5,414
edits
Changes
m
We can develop backends that work directly on Exported formats. For example, our SQLite export would make a fine '''Branch geps/gep-032-backend. Others to consider: GEDCOM and CSV. These would probably use a DbDictionary, and simply import/export on load/close (would lose data if power outage; would be fast as in-memory, slow on start/stop)database is complete.'''
→See Also
{{GEPS-Finished}}
{{man menu|'''This GEP is now complete and has been implemented for [[Gramps_5.0_Wiki_Manual_-_What%27s_new%3F|Gramps 5.0]], now in github master (Aug 14, 2015).'''}}
This proposal defines a complete Database Backend API so that we can have plug-in replacements for BSDDB. This would allow the use of other databases.
==Plan==
===Step 1===
Create a Database Backend plugin. Create a functioning BSDDB plugin, that is used by default. Identify which plugin to use via a file in the database directory.
This was developed under the branch geps/gep-032-database-backend here:
https://github.com/gramps-project/gramps/tree/geps/gep-032-database-backend
It has now been committed to gramps50 (aka master as of this writing).
Identify the items that:
* need to be fixed
* are too BSDDB specific (some tools?)
* could be abstracted away from details
'''Step one is completed.'''
===Step 2===
Develop alternative plugins that support the db API.
Exactly what is required for a class to implement a fully functional database for Gramps is only determined by examination of the existing BSDDB class. This involves the following components:
Once a full Gramps Database class is created, there needs to be a way of:
# selecting which backend to use for new databases[DONE]# selecting the database to load (Family Tree Manager)[DONE]
We will use the directory structure, as we do now. In each directory, the type of database needs to be identified. This could be done in two ways:
# well-defined database backend types. These could be registered, like any plugin/addon.[DONE]# is there really any reason for Gramps to have to have the code for the db backend? All that is necessary is for the backend to create the Database instance.[DONE- Backend code is either in gramps/plugins/database or in ~/.gramps/gramps50/plugins)
It makes sense that we will reuse and share the backends, so we should use option 1, and develop a database backend plugin type.
We should factor-out all BSDDB dependencies, and make BSDDB the first database backend plugin. The plugin API should include functions for:
* making a new database, given a directory[DONE]* loading the database, given the directory[DONE]
Other things that might need to be changed:
Listing the databases (-l and -L). That might include changing the current listing:[DONE]
<pre>
Schema version: Unknown
</pre>
'''DjangoDb, DictionaryDb, and DBAPI are complete.'''
=== Step 3 ===
Develop a fully-tested alternative to BSDDB.
Proposal is to develop a [https://www.python.org/dev/peps/pep-0249/ DB-API 2.0 database backend], testing with sqlite, postgresql, and mysql. [COMPLETE]
The database directory will have a small initialize program to create the database, and return a class that can create the connection.
==Transactions==
Because BSDDB didn't support transaction nativelybut required a two-step transactional system, Gramps developed these this support in Python. Most database system have transaction built into the system. Most database backends will perhaps not use transactions as BSDDB does.
There still could be uses for the transactions, However. For example, we can use an abstraction for the History undo/redo. Although the current system only exists in the current session, and is limited. We can probably create a better method with more features (such as diff between versions, lifetime changes, etc).
'''COMPLETE'''
==Complications==
Some systems, like Django, have a single database used per Python session. That makes sense, given that it is designed to be a webserver with fixed settings. However, what to do if you want to use the Django ORM and switch settings?
I have implemented a solution based on the following:
<pre>
import sys
class ModulesCheckpoint(object):
def __init__(self):
self.original = sys.modules.copy()
def reset(self):
# clear modules:
for key in list(sys.modules.keys()):
del(sys.modules[key])
# load previous:
for key in self.original:
sys.modules[key] = self.original[key]
checkpoint = ModulesCheckpoint()
# do stuff
checkpoint.reset()
# do stuff again
</pre>
However, this has to be done in the right place, and can't (as far as I can see) be embedded in an the imported database plugin. [DONE]
'''It would be nice to do away with hack, but is the only method I can find to unload Django. It works fine, so far.'''
==Progress==
The first step is to separate all of the gramps.gen.db code into reusable and extendable components. [DONE]
This has begun with the [http://sourceforge.net/p/gramps/source/ci/master/tree/gramps/gen/db/dictionary.py DictionaryDB] {{bug|4972}}, which is a in-memory replacement for the BSDDB. Still needs the indexes, and metadata support (gender names, bookmarks, etc). Also, the Dictionary transaction is non-existent.
Currently, the best working replacement backend is gramps"dictionarydb".webapp[DONE] We can develop backends that work directly on Exported formats.dbdjangoOthers to consider: GEDCOM and CSV. These would probably use a DbDictionary, and simply import/export on load/close (would lose data if power outage; would be fast as in-memory, slow on start/stop).
==Other Backends==
= See Also =
* [[ExportSql.pyAddon:SQLite Export Import]]
* [[GEPS 010: Relational Backend]]
* [[GEPS 013: Gramps Webapp]]
* [[Database Backends]]
* Maillist thread: [Gramps-devel] [https://sourceforge.net/p/gramps/mailman/gramps-devel/thread/226cf688-5eba-d3a1-a971-31491541b2e2%40gramps-project.org/#msg36907575 Proposal to remove the BSDDB backend] - 23 Jan 2020
* Doug Blank YouTube channel: [https://www.youtube.com/watch?v=BoGkTypiQbA GEP-032: Gramps Database Backend API]
[[Category:GEPS|S]]