28
edits
Changes
From Gramps
Synced from repo@41d611f via publish.py
<!--
Data access is the first non-trivial thing every addon does: read the
family tree the user has open. This chapter covers the shape of the
database API, not the full API reference (that's
[07-api-reference](07-api-reference.md)). Aim: enough to write a working
addon without first having to read gramps/gen/db source.
Authoritative cross-reference: the standalone wiki page
[[Using_database_API|Using database API]] covers internals and
performance in more depth; this chapter sticks to what addon authors
actually need.
-->
== Overview ==
Every addon that does anything useful with a family tree reads or writes through the '''database API''' — the <code>DbReadBase</code> / <code>DbWriteBase</code> interface implemented by Gramps' database backends (BSDDB historically, SQLite from 6.0 onward).
You don't instantiate a database yourself. The plugin loader hands you a <code>DbState</code> object; the live database is <code>dbstate.db</code>. Everything below is methods on that handle.
<syntaxhighlight lang="python">db = dbstate.db # this is your entry point</syntaxhighlight>
The same <code>db</code> works for read-only addons (reports, gramplets, quick views) and for tools that mutate data. Mutation goes through transactions; see [[#mutating-data|Mutating data]] below.
== Identifying objects: handles vs Gramps IDs ==
Every primary object (Person, Family, Event, Place, Source, Citation, Repository, Media, Note, Tag) has '''two identifiers''':
{|
! Identifier
! Stable
! Format
! Used for
|-
| '''Handle'''
| Yes (internal, never reused)
| 32-char hex string
| Cross-references in the database
|-
| '''Gramps ID'''
| User-renameable
| <code>I0001</code>, <code>F0001</code>, <code>E0001</code>, ...
| User-visible labels and external interop
|}
'''Rule of thumb:''' use handles inside your code; show Gramps IDs to the user. Handles never change; Gramps IDs do (the user can edit them, the "Reorder Gramps IDs" tool can rewrite them in bulk).
<syntaxhighlight lang="python"># Right: traverse by handle
person = db.get_person_from_handle(handle)
# Right: show the user a Gramps ID
print(f"Working on {person.gramps_id}")
# Wrong: traverse by Gramps ID (works, but slower and breaks under reorder)
person = db.get_person_from_gramps_id("I0001")</syntaxhighlight>
Each object class has both lookup methods (<code>get_<type>_from_handle</code> and <code>get_<type>_from_gramps_id</code>); see [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] for the full list.
== Reading: one object at a time ==
The fastest pattern, when you have a handle in hand:
<syntaxhighlight lang="python">person = db.get_person_from_handle(person_handle)
family = db.get_family_from_handle(family_handle)
event = db.get_event_from_handle(event_handle)</syntaxhighlight>
Each returns <code>None</code> if the handle isn't in the database (deleted, broken reference). Always guard:
<syntaxhighlight lang="python">person = db.get_person_from_handle(handle)
if person is None:
return # silently skip, or raise a HandleError if the caller expects one</syntaxhighlight>
For <code>HandleError</code> and friends, import from <code>gramps.gen.errors</code>.
== Reading: iterating all objects ==
For reports and surveys you'll want every object of a given type. The database exposes one generator per object class:
<syntaxhighlight lang="python">for person in db.iter_people():
...
for family in db.iter_families():
...</syntaxhighlight>
These are '''generators''', not lists — they stream through the database without loading everything into memory. Don't call <code>list(db.iter_people())</code> on a 50,000-person tree unless you have a reason.
To iterate just the handles (cheaper when you only need to count or filter):
<syntaxhighlight lang="python">for handle in db.iter_person_handles():
...</syntaxhighlight>
Counts come without iteration:
<syntaxhighlight lang="python">db.get_number_of_people()
db.get_number_of_families()
db.get_number_of_events()</syntaxhighlight>
== Following references ==
[[File:data-model.svg|Fig. 1 — Gramps primary objects and the most-traversed relationships. Edges labelled <code><Type>Ref</code> (e.g. <code>EventRef</code>, <code>CitationRef</code>, <code>MediaRef</code>) go through a ref object that carries metadata such as the role or relationship; bare-labelled edges are direct handle references. Notes and Tags can be attached to any primary object and are omitted to keep arrows readable. Reverse traversals — "who refers to this object?" — go through <code>db.find_backlink_handles()</code> instead of these forward links; see Backlinks below.]]
Most addons don't visit objects in isolation — they follow the relationships between them. Gramps' object model exposes references as '''handle lists''' on the parent object.
Person → families they're a parent in:
<syntaxhighlight lang="python">for family_handle in person.get_family_handle_list():
family = db.get_family_from_handle(family_handle)
...</syntaxhighlight>
Person → events:
<syntaxhighlight lang="python">for ref in person.get_event_ref_list():
event = db.get_event_from_handle(ref.ref)
role = ref.get_role()
...</syntaxhighlight>
<code>event_ref</code> carries more than the handle — also the role (Primary, Witness, etc.) and any private flag. Read the ref, then dereference if you need the event itself.
Family → children:
<syntaxhighlight lang="python">for child_ref in family.get_child_ref_list():
child = db.get_person_from_handle(child_ref.ref)
...</syntaxhighlight>
The full handle-list / ref-list inventory per object class lives in the [[Gramps_6.0_Developer_Reference|Gramps API docs]]; see also [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] for the addon-facing subset.
== Backlinks: who refers to this object? ==
The forward direction (person → events they participated in) lives on the object. The reverse direction (event → people who participated in it) lives on the database:
<syntaxhighlight lang="python">for (obj_type, obj_handle) in db.find_backlink_handles(event.handle):
if obj_type == "Person":
person = db.get_person_from_handle(obj_handle)
...</syntaxhighlight>
<code>find_backlink_handles</code> returns <code>(class_name, handle)</code> tuples for every primary object that references the given handle. Use it for:
* Finding all sources that cite a given place
* Finding all people present at a given event
* Detecting orphaned objects (no backlinks → unreferenced)
Note that <code>obj_type</code> is the '''class name as a string''' (<code>"Person"</code>, <code>"Family"</code>, ...), not the Python class itself.
== Filters ==
For non-trivial selection (e.g. "all people born in Hamburg between 1850 and 1900"), use Gramps' filter framework rather than hand-rolling predicates:
<syntaxhighlight lang="python">from gramps.gen.filters import GenericFilterFactory
GenericFilter = GenericFilterFactory("Person")
filt = GenericFilter()
filt.add_rule(SomeRule([arg1, arg2]))
handles = filt.apply(db, db.iter_person_handles())</syntaxhighlight>
Filters compose, cache, and integrate with the GUI's filter sidebar — a report that defines its own filter gets it as a sidebar option for free. The rule catalogue lives under <code>gramps.gen.filters.rules</code>; the user-facing counterpart is documented in [[Gramps_6.0_Wiki_Manual_-_Filters|Filters]].
== Mutating data ==
Write addons (mainly tools) modify data through '''transactions'''. The pattern is always the same:
<syntaxhighlight lang="python">with DbTxn(_("Tool name: what it did"), db) as trans:
person = db.get_person_from_handle(handle)
person.set_privacy(True)
db.commit_person(person, trans)</syntaxhighlight>
Three things matter:
# '''The transaction message is user-visible''' in the Undo History. Make it descriptive and translated.
# '''Always <code>db.commit_<type>(obj, trans)</code>''' after mutating — the object is a copy; commit writes it back.
# '''Group related changes''' in one transaction so the user can undo as a single step.
Creating a new object follows the same shape:
<syntaxhighlight lang="python">from gramps.gen.lib import Person, Name
with DbTxn(_("Add unknown spouse"), db) as trans:
person = Person()
name = Name()
name.set_surname(surname)
person.set_primary_name(name)
person.gramps_id = db.find_next_person_gramps_id()
db.add_person(person, trans)</syntaxhighlight>
<code>find_next_<type>_gramps_id()</code> allocates an unused ID; <code>add_<type>()</code> inserts and assigns the handle.
== Testing data access ==
Two complementary approaches:
* '''Real-data tests''' — load <code>example.gramps</code> (shipped with Gramps, canonical test fixture) and exercise your code against it. Best for catching real-world data quirks (cross-typed backlinks, ID normalisation, unusual character sets). See [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Testing|08-testing]].
* '''Mocked tests''' — substitute the database with a stub that returns fixed objects. Best for tight unit-test loops that don't need a database on disk.
The lesson, learned the hard way: mocked DB tests can pass while the real-DB code is broken, because the mock doesn't reproduce the cross-typed backlinks and ID quirks of a populated tree. Prefer example.gramps for anything that traverses the DB; reserve mocks for pure helpers.
== Performance notes ==
The database API is fast enough that most addons don't need to think about performance. When you do:
* Iterating '''handles''' is cheaper than iterating '''objects''' — only dereference when you need the object's contents.
* <code>get_number_of_<type>()</code> is O(1); <code>len(list(db.iter_<type>()))</code> is O(n).
* Backlinks aren't free — they read an index but still scan it. Don't call <code>find_backlink_handles</code> in a tight inner loop.
* The 5.x → 6.0 SQLite backend is roughly comparable to BSDDB for reads, faster for writes. Avoid backend-specific assumptions; addons should work on either.
== See also ==
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Fundamentals|05-fundamentals]] — the plugin lifecycle that wraps this DB access in
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] — the addon-facing API surface
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Testing|08-testing]] — testing strategies, real-data vs mocks
* [[Using_database_API|Using database API]] — the standalone wiki reference, covers backends and internals in more depth
* [[Gramps_6.0_Developer_Reference|Gramps Developer Reference]] — the full API docs
Data access is the first non-trivial thing every addon does: read the
family tree the user has open. This chapter covers the shape of the
database API, not the full API reference (that's
[07-api-reference](07-api-reference.md)). Aim: enough to write a working
addon without first having to read gramps/gen/db source.
Authoritative cross-reference: the standalone wiki page
[[Using_database_API|Using database API]] covers internals and
performance in more depth; this chapter sticks to what addon authors
actually need.
-->
== Overview ==
Every addon that does anything useful with a family tree reads or writes through the '''database API''' — the <code>DbReadBase</code> / <code>DbWriteBase</code> interface implemented by Gramps' database backends (BSDDB historically, SQLite from 6.0 onward).
You don't instantiate a database yourself. The plugin loader hands you a <code>DbState</code> object; the live database is <code>dbstate.db</code>. Everything below is methods on that handle.
<syntaxhighlight lang="python">db = dbstate.db # this is your entry point</syntaxhighlight>
The same <code>db</code> works for read-only addons (reports, gramplets, quick views) and for tools that mutate data. Mutation goes through transactions; see [[#mutating-data|Mutating data]] below.
== Identifying objects: handles vs Gramps IDs ==
Every primary object (Person, Family, Event, Place, Source, Citation, Repository, Media, Note, Tag) has '''two identifiers''':
{|
! Identifier
! Stable
! Format
! Used for
|-
| '''Handle'''
| Yes (internal, never reused)
| 32-char hex string
| Cross-references in the database
|-
| '''Gramps ID'''
| User-renameable
| <code>I0001</code>, <code>F0001</code>, <code>E0001</code>, ...
| User-visible labels and external interop
|}
'''Rule of thumb:''' use handles inside your code; show Gramps IDs to the user. Handles never change; Gramps IDs do (the user can edit them, the "Reorder Gramps IDs" tool can rewrite them in bulk).
<syntaxhighlight lang="python"># Right: traverse by handle
person = db.get_person_from_handle(handle)
# Right: show the user a Gramps ID
print(f"Working on {person.gramps_id}")
# Wrong: traverse by Gramps ID (works, but slower and breaks under reorder)
person = db.get_person_from_gramps_id("I0001")</syntaxhighlight>
Each object class has both lookup methods (<code>get_<type>_from_handle</code> and <code>get_<type>_from_gramps_id</code>); see [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] for the full list.
== Reading: one object at a time ==
The fastest pattern, when you have a handle in hand:
<syntaxhighlight lang="python">person = db.get_person_from_handle(person_handle)
family = db.get_family_from_handle(family_handle)
event = db.get_event_from_handle(event_handle)</syntaxhighlight>
Each returns <code>None</code> if the handle isn't in the database (deleted, broken reference). Always guard:
<syntaxhighlight lang="python">person = db.get_person_from_handle(handle)
if person is None:
return # silently skip, or raise a HandleError if the caller expects one</syntaxhighlight>
For <code>HandleError</code> and friends, import from <code>gramps.gen.errors</code>.
== Reading: iterating all objects ==
For reports and surveys you'll want every object of a given type. The database exposes one generator per object class:
<syntaxhighlight lang="python">for person in db.iter_people():
...
for family in db.iter_families():
...</syntaxhighlight>
These are '''generators''', not lists — they stream through the database without loading everything into memory. Don't call <code>list(db.iter_people())</code> on a 50,000-person tree unless you have a reason.
To iterate just the handles (cheaper when you only need to count or filter):
<syntaxhighlight lang="python">for handle in db.iter_person_handles():
...</syntaxhighlight>
Counts come without iteration:
<syntaxhighlight lang="python">db.get_number_of_people()
db.get_number_of_families()
db.get_number_of_events()</syntaxhighlight>
== Following references ==
[[File:data-model.svg|Fig. 1 — Gramps primary objects and the most-traversed relationships. Edges labelled <code><Type>Ref</code> (e.g. <code>EventRef</code>, <code>CitationRef</code>, <code>MediaRef</code>) go through a ref object that carries metadata such as the role or relationship; bare-labelled edges are direct handle references. Notes and Tags can be attached to any primary object and are omitted to keep arrows readable. Reverse traversals — "who refers to this object?" — go through <code>db.find_backlink_handles()</code> instead of these forward links; see Backlinks below.]]
Most addons don't visit objects in isolation — they follow the relationships between them. Gramps' object model exposes references as '''handle lists''' on the parent object.
Person → families they're a parent in:
<syntaxhighlight lang="python">for family_handle in person.get_family_handle_list():
family = db.get_family_from_handle(family_handle)
...</syntaxhighlight>
Person → events:
<syntaxhighlight lang="python">for ref in person.get_event_ref_list():
event = db.get_event_from_handle(ref.ref)
role = ref.get_role()
...</syntaxhighlight>
<code>event_ref</code> carries more than the handle — also the role (Primary, Witness, etc.) and any private flag. Read the ref, then dereference if you need the event itself.
Family → children:
<syntaxhighlight lang="python">for child_ref in family.get_child_ref_list():
child = db.get_person_from_handle(child_ref.ref)
...</syntaxhighlight>
The full handle-list / ref-list inventory per object class lives in the [[Gramps_6.0_Developer_Reference|Gramps API docs]]; see also [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] for the addon-facing subset.
== Backlinks: who refers to this object? ==
The forward direction (person → events they participated in) lives on the object. The reverse direction (event → people who participated in it) lives on the database:
<syntaxhighlight lang="python">for (obj_type, obj_handle) in db.find_backlink_handles(event.handle):
if obj_type == "Person":
person = db.get_person_from_handle(obj_handle)
...</syntaxhighlight>
<code>find_backlink_handles</code> returns <code>(class_name, handle)</code> tuples for every primary object that references the given handle. Use it for:
* Finding all sources that cite a given place
* Finding all people present at a given event
* Detecting orphaned objects (no backlinks → unreferenced)
Note that <code>obj_type</code> is the '''class name as a string''' (<code>"Person"</code>, <code>"Family"</code>, ...), not the Python class itself.
== Filters ==
For non-trivial selection (e.g. "all people born in Hamburg between 1850 and 1900"), use Gramps' filter framework rather than hand-rolling predicates:
<syntaxhighlight lang="python">from gramps.gen.filters import GenericFilterFactory
GenericFilter = GenericFilterFactory("Person")
filt = GenericFilter()
filt.add_rule(SomeRule([arg1, arg2]))
handles = filt.apply(db, db.iter_person_handles())</syntaxhighlight>
Filters compose, cache, and integrate with the GUI's filter sidebar — a report that defines its own filter gets it as a sidebar option for free. The rule catalogue lives under <code>gramps.gen.filters.rules</code>; the user-facing counterpart is documented in [[Gramps_6.0_Wiki_Manual_-_Filters|Filters]].
== Mutating data ==
Write addons (mainly tools) modify data through '''transactions'''. The pattern is always the same:
<syntaxhighlight lang="python">with DbTxn(_("Tool name: what it did"), db) as trans:
person = db.get_person_from_handle(handle)
person.set_privacy(True)
db.commit_person(person, trans)</syntaxhighlight>
Three things matter:
# '''The transaction message is user-visible''' in the Undo History. Make it descriptive and translated.
# '''Always <code>db.commit_<type>(obj, trans)</code>''' after mutating — the object is a copy; commit writes it back.
# '''Group related changes''' in one transaction so the user can undo as a single step.
Creating a new object follows the same shape:
<syntaxhighlight lang="python">from gramps.gen.lib import Person, Name
with DbTxn(_("Add unknown spouse"), db) as trans:
person = Person()
name = Name()
name.set_surname(surname)
person.set_primary_name(name)
person.gramps_id = db.find_next_person_gramps_id()
db.add_person(person, trans)</syntaxhighlight>
<code>find_next_<type>_gramps_id()</code> allocates an unused ID; <code>add_<type>()</code> inserts and assigns the handle.
== Testing data access ==
Two complementary approaches:
* '''Real-data tests''' — load <code>example.gramps</code> (shipped with Gramps, canonical test fixture) and exercise your code against it. Best for catching real-world data quirks (cross-typed backlinks, ID normalisation, unusual character sets). See [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Testing|08-testing]].
* '''Mocked tests''' — substitute the database with a stub that returns fixed objects. Best for tight unit-test loops that don't need a database on disk.
The lesson, learned the hard way: mocked DB tests can pass while the real-DB code is broken, because the mock doesn't reproduce the cross-typed backlinks and ID quirks of a populated tree. Prefer example.gramps for anything that traverses the DB; reserve mocks for pure helpers.
== Performance notes ==
The database API is fast enough that most addons don't need to think about performance. When you do:
* Iterating '''handles''' is cheaper than iterating '''objects''' — only dereference when you need the object's contents.
* <code>get_number_of_<type>()</code> is O(1); <code>len(list(db.iter_<type>()))</code> is O(n).
* Backlinks aren't free — they read an index but still scan it. Don't call <code>find_backlink_handles</code> in a tight inner loop.
* The 5.x → 6.0 SQLite backend is roughly comparable to BSDDB for reads, faster for writes. Avoid backend-specific assumptions; addons should work on either.
== See also ==
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Fundamentals|05-fundamentals]] — the plugin lifecycle that wraps this DB access in
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_API_Reference|07-api-reference]] — the addon-facing API surface
* [[User:Eduralph/Sandbox/Gramps_6.0_Wiki_Manual_-_Addon_Development_-_Testing|08-testing]] — testing strategies, real-data vs mocks
* [[Using_database_API|Using database API]] — the standalone wiki reference, covers backends and internals in more depth
* [[Gramps_6.0_Developer_Reference|Gramps Developer Reference]] — the full API docs
