Difference between revisions of "Relational database comparison"

From Gramps
Jump to: navigation, search
m
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page if for a comparison of different database, and is specific for how they might be used for GRAMPS. It was started to help with [[GEPS 010: SQL Backend]].  
+
This page is for a comparison of different relational database engines, and is specific for how they might be used for Gramps. (Gramps has used the [[Gramps_Glossary#bsddb|BSDDB]] database engine from the 2.0 to 5.0 versions.) This comparison was started to help with [[GEPS 010: SQL Backend]].  
  
 
=SQLite=
 
=SQLite=
Line 31: Line 31:
 
*many "traditional" relational db things are lacking.
 
*many "traditional" relational db things are lacking.
 
*while tools exist they aren't as fleshed out and solid as the mysql ones.
 
*while tools exist they aren't as fleshed out and solid as the mysql ones.
 
+
*there may be some limits on size mostly due to ram and rom limts. [http://www.sqlite.org/limits.html Limits In SQLite] Since the whole db doesn't need to be loaded on reads and rights these limits should be far larger than for bsddb.
  
 
=MySQL=
 
=MySQL=
Line 37: Line 37:
 
*far better tools for management and reporting
 
*far better tools for management and reporting
 
*a true enterprise level database capable of handling serious loads
 
*a true enterprise level database capable of handling serious loads
*far more is built into the db. ie auto incrementing fields, stored procedures and on and on.
+
*far more is built into the db. ie auto incrementing fields, stored procedures and on and on. (sqlite may not even have triggers but I can't remember)
(sqlite may not even have triggers but I can't remember)
 
 
*far more extensive user base and support.
 
*far more extensive user base and support.
  
Line 44: Line 43:
 
*install size (bloat)
 
*install size (bloat)
 
*an actual server to setup run and maintain.
 
*an actual server to setup run and maintain.
** there are tools that can do this automatically though and make things almost none
+
** there are tools that can do this automatically though and make things almost none existent for an end user. also the embeddable mysql might be an option.
existent for an end user. also the embeddable mysql might be an option.
+
*may be difficult to manage / share multiple databases. more difficult but very do able. maybe not even that difficult. it should just take some planning.
*may be difficult to manage / share multiple databases. more difficult but very do able.
 
  maybe not even that difficult. it would just take some planning.
 
  
 
=MySQL Embeded=
 
=MySQL Embeded=
Line 98: Line 95:
 
http://pybsddb.sourceforge.net/ref/intro/dbisnot.html
 
http://pybsddb.sourceforge.net/ref/intro/dbisnot.html
  
From previous GRAMPS discussions:
+
[[Category:Developers/General]]
 
 
http://mlblog.osdir.com/genealogy.gramps.devel/2005-02/msg00092.shtml&ei=2MYxSanZNaCgesqz0KQB&usg=AFQjCNG1l3yKZ4YP_L7Yo0cQ8bqWmoJKTQ&sig2=H8x1qf4YrFYlsLFlJUsZ-w
 
 
 
From the GRAMPS archives:
 
<pre>
 
> Now, sometimes we get a request for a major architectural change that we
 
> will accept. A good example is the new database backend for the upcoming
 
> GRAMPS 2.0. The request came in to support a real database backend so we
 
> could support larger databases. We analyzed the request, and felt that
 
> it matched the goals of the project and would provide a significant step
 
> forward in the usability of the program. The result was a major redesign
 
> effort that will soon be released.
 
 
 
I think I and few others are the ones that impacted this decision.  Having an
 
850,000 person database tends to be deadly to the XML architecture that we
 
were with.  I've been the main person to test the integrity of the system
 
with my Gedcom file importing.  When I found that I couldn't import my file
 
without extensive data loss, I came to Don and Alex and we all sought for
 
solutions.  We found that the XML interface was taking huge amounts of
 
memory, and we looked for database backends that would handle the load.  Don
 
and Alex came through with the BSDDB backend, and ever since 1.1.3, I've been
 
happy as a clam with the Gramps project, because I'm one step closer to
 
killing Windows.
 
 
 
I personally want to do away with it, but I need it for other applications. 
 
I've also come to the realization that both Windows and Linux are good, but
 
in their own realms.  I don't want this to become a huge flame war about
 
Linux and Windows. so if you have other questions as to why I feel this way,
 
email me.
 
 
 
> So, would we accept a mySQL database backend? There is a good chance we
 
> would (depending on the implementation), as long did not impact Aunt
 
> Martha. We have even architected the backend to support this, since we
 
> can see that higher end databases could provide additional functionality
 
> such as versioning and multiuser support.
 
 
 
We could accept mySQL because of this, but I agree with Don.  If it negatively
 
impacts the end user, why would we want to proceed with it?  I have a friend
 
that wondered about mySQL interaction, but he can see the impact that BSDDB
 
has had on my database, and he has sided with me as well as the rest of the
 
team.  Not to say that this is not a possibility, but we need to remain
 
focused on the tasks at hand.
 
 
 
> So, in summary, the project is going in a direction that seems to meet
 
> the needs of our users. If we changed directions, we might or might not
 
> be able to reach a larger audience, but numbers are not our goal. We
 
> fully support others submitting patches and other contributions, but
 
> they will be weighed on how they match the goals of the project (and
 
> most of the patches we've received to date do match the goals). If
 
> someone wants us take the project in a different direction, we may or
 
> may not be receptive depending if the direction matches our goals.
 
> However, we will support your efforts if you decide to fork the project.
 
> Who knows, maybe a remerge will occur in the future, or a forked project
 
> will make us irrelevent.
 
 
 
I agree with Don on this, numbers don't matter as long as the users are happy. 
 
Getting things appropriately nailed down and ready for the end user's use is
 
what is paramount.  After all, if there were no users, why would we even have
 
a project with which to collaborate in the first place?
 
 
 
We are here for the users, especially Aunt Martha, because of the fact that
 
many people are just moving over to Linux and having something familiar to
 
them, like a genealogical program is what matters to them.  Making the
 
transition to Linux is hard, don't get me wrong.  But we are making it one
 
step easier by not complicating the user's experience in their move.
 
 
 
Like I said before, I'm just a bug finder.  I'm not really a Python
 
programmer, or anything, but I like to find bugs.  Even if that's all I do on
 
this project, I'm rather content.  Everyone else that wants to port over to
 
other toolkits and whatnot is free to do so.
 
 
 
But also as an end user that's still a greenie to Linux in general, I can say
 
that this program has helped my move over to Linux that much easier.  Even if
 
I have only contributed a little in the way of feedback (mostly from the
 
end-user perspective).
 
 
 
-Jason
 
</pre>
 
 
 
From http://osdir.com/ml/genealogy.gramps.user/2004-06/msg00078.html:
 
 
 
<pre>
 
Alex said:
 
 
 
    SQLite might be better or it might not, we haven't tried it. A great factor
 
    speaking for BSDDB is that it is supported by a standard Python module,
 
    bsddb.
 
 
 
 
 
Don said:
 
 
 
This is an important factor here - ease of setup and use. GRAMPS is
 
difficult enough to get installed on some platforms (especially
 
KDE-centric systems). Requiring someone to get an SQL database up and
 
running to try out the program is probably too much effort. What I've
 
discovered is that GRAMPS is one of the first programs that a lot of
 
new users want to get running - usually before they have a lot of
 
Linux experience. So we can't make the barriers to entry too high.
 
</pre>
 
 
 
<pre>
 
"Requiring someone to get an SQL database up and
 
running to try out the program is probably too much effort." This simply isn't true of sqlite
 
at all. The program would simply write to the db file. No server setup, no user accounts, no
 
connection settings. Just a file name. users wouldn't even know. The embeded version of MySQL
 
may be similar but I haven't tried it out. This might be true of MySQL though. However, I believe
 
it's possible to use scripts and or code to manage launching and stoping the server. It might
 
be possible to make it seamless for the user but would depend on the implementation.
 
--AaronS
 
</pre>
 

Latest revision as of 04:32, 23 December 2020

This page is for a comparison of different relational database engines, and is specific for how they might be used for Gramps. (Gramps has used the BSDDB database engine from the 2.0 to 5.0 versions.) This comparison was started to help with GEPS 010: SQL Backend.

SQLite

Advantages

  • far easier to setup. just start writing to the file! no connection or user accounts.
  • smaller install (code) size.
  • easier for users to manage / and share sepperate db's
  • single file
  • good support.

Transportable Trees

From http://www.sqlite.org/onefile.html:

Single-file Cross-platform Database

A database in SQLite is a single disk file. Furthermore, the file format is cross-platform. A database that is created on one machine can be copied and used on a different machine with a different architecture. SQLite databases are portable across 32-bit and 64-bit machines and between big-endian and little-endian architectures.

The SQLite database file format is also stable. All releases of of SQLite version 3 can read and write database files created by the very first SQLite 3 release (version 3.0.0) going back to 2004-06-18. This is "backwards compatibility". The developers promise to maintain backwards compatibility of the database file format for all future releases of SQLite 3. "Forwards compatiblity" means that older releases of SQLite can also read and write databases created by newer releases. SQLite is usually, but not completely forwards compatible.

The stability of the SQLite database file format and the fact that the file format is cross-platform combine to make SQLite database files an excellent choice as an Application File Format.

The Single disk file of sqlite db would be a major selling point for sqlite
for genealogy software since users share and compare db's all the time.
--Aarons

Disadvantages

  • while great for what it is it's not an enterprise level database
  • many "traditional" relational db things are lacking.
  • while tools exist they aren't as fleshed out and solid as the mysql ones.
  • there may be some limits on size mostly due to ram and rom limts. Limits In SQLite Since the whole db doesn't need to be loaded on reads and rights these limits should be far larger than for bsddb.

MySQL

Advantages

  • far better tools for management and reporting
  • a true enterprise level database capable of handling serious loads
  • far more is built into the db. ie auto incrementing fields, stored procedures and on and on. (sqlite may not even have triggers but I can't remember)
  • far more extensive user base and support.

Disadvantages

  • install size (bloat)
  • an actual server to setup run and maintain.
    • there are tools that can do this automatically though and make things almost none existent for an end user. also the embeddable mysql might be an option.
  • may be difficult to manage / share multiple databases. more difficult but very do able. maybe not even that difficult. it should just take some planning.

MySQL Embeded

Needs to be looked at, may be more powerful than sqlite but easier for end users than full mysql

Comparing BSDDB to SQLite

A sqlite shelve interface for Python: http://bugs.python.org/issue3783

From http://www.sqlite.org/cvstrac/wiki?p=SqliteCompetitors: SQLite versus Berkeley DB:

      Berkeley DB (BDB) is just the data storage layer - it does not
      support SQL or schemas. In spite of this, BDB is twice the size
      of SQLite. A comparison between BDB and SQLite is similar to a
      comparison between assembly language and a dynamic language like
      Python or Tcl. BDB is probably much faster if you code it
      carefully. But it is much more difficult to use and considerably
      less flexible.

      On the other hand BDB has very fine grained locking (although
      it's not very well documented), while SQLite currently has only
      database-level locking. -- fine grain locking is important for
      enterprise database engines, but much less so for embedded
      databases. In SQLite, a writer gets a lock, does an update, and
      releases the lock all in a few milliseconds. Other readers have
      to wait a few milliseconds to access the database, but is that
      really ever a serious problem?


A company that justifies a switch from BSDDB to SQLite; see http://www.tribler.org/DatabaseMigration:

Oracle's description of BSDDB; see http://www.oracle.com/database/docs/Berkeley-DB-v-Relational.pdf. Excerpt:

Berkeley DB Offers APIs, not Query Languages 

Berkeley DB was designed for software developers, by software
developers.  Relational database systems generally provide SQL access
to the data that they manage, and usually offer some SQL abstraction,
like ODBC or JDBC, for use in applications.

What BSDDB is not:

http://pybsddb.sourceforge.net/ref/intro/dbisnot.html