Addon:Import Merge Tool
This is a Third-party Addon. Please use carefully on data that is backed up, and help make it better by reporting any comments or problems to the author, or issues to the bug tracker |
Released for Gramps 5.0.x or greater versions |
The Import and Merge tool will Import and merge a Gramps XML family tree derived from your own Gramps XML family tree backup file.
Contents
Purpose of this tool
This tool was specifically written to make importing and merging a Gramps Tree derived from your own Tree easier.
Make no mistake, merging is hard! And this tool is not magic, it does not make the process simple but it is easier than the alternatives.
Important Note This tool this requires that the Family Tree being merged must be a Gramps Family Tree derived from a version of the currently loaded Family Tree. |
A situation occurs occasionally after giving someone a copy of your tree in our native Gramps XML format. Then each person makes changes to the copy in their possession using Gramps locally. (You can verify the Trees differ and log those revisions with the Database Differences addon report.)
Programmers call this a 'fork.' (As in: when travelers come to a 'fork in the road' and each chooses different paths.) But the bottom line is that you now have two tree variants derived from a common Tree and you want share your updated Tree. You can't just give the person a fresh copy of your Tree because then their changes would be overwritten.
At some point, you may want to merge the forks into a common tree... without losing the useful changes from either tree. That synchronization is what this tool is designed to facilitate. It depends on the two Trees sharing common internal handles (not just the user-editable IDs) to simplify ignoring unchanged records. Then the tool focuses you on just the changes.
But say the other user doesn't use Gramps, so you exported with GEDCOM (or another format) and got it back? Sorry... this tool won't be any easier than a manual merge after a standard import. You'd still need to hand compare every record.
The derived trees need not be limited to those edited by someone else. I often take a copy of my tree on my laptop when traveling and sometimes I make updates while on the road. But then I may forget to copy it back to my home desktop computer until after I've made more updates. This tool helps to merge these back together.
Warning: Data Corruption Risk Mass changes mean more potential for mangling the data. Back up your current tree BEFORE using this tool to merge. |
Usage
The tool can be installed like any other Addon; see Installing Addons in Gramps
Once installed, it is run by using the Tools -> Family Tree Processing -> Import and Merge tool....
Running the Import Merge tool
The tool will start with an 'Undo history warning' that suggest that you back up your work.
You are then presented with the import file dialog, which shows '.gramps' files from your import location.
When you have selected a file to import, the main tool window comes up and some progress meters pop up indicating the import and comparison process. When this is complete you end up with a three part window. The bottom part provides control and 'Action' buttons and a hint that changes depending on what you are doing.
Top Pane
The top pane is a scrollable and selectable view of individual items that were found to be different in your import. Items that were the same (since they were derived from the common Gramps tree) are not shown.
The top pane shows a 'Status' which is one of Different, Added, or Missing. Different items were found in both the current and imported trees, but had been changed in some way. Added items are found only in your imported tree. And Missing items were found in your current tree, but not in the imported tree.
The top pane also shows the Object type (Person, Family, Media etc.), its Gramps ID, an Object Name or Description, and the selected Action (more later).
You can search for items by Name by just typing characters to match. The top pane must be active for this to work. For example if you are looking for 'Smith, Magnes' in the 'sample.gramps' tree, you could type 'smi' which would jump the selection to the first of the lines containing these characters, and then use cursor up/down keys to find the 'Smith' you are looking for. Or type 'magn' which is more likely to land on the right place.
You can search for items by 'ID' field by typing '#' and then the characters to match. For example type '#I0103' to search for the individual with ID I0103.
You can sort the top pane by Status (Object will be a sub-sort), Object, ID, Object Name/Description (will be grouped by Object) and Action. Click on the column header to change the sort.
Lower Pane
When an item in the top pane is selected, the lower pane shows the specific differences. If no Action is made yet, the pane will show the 'Original >>' and 'Imported >>' state for each sub-object and item that has a difference. If an Action has been selected, the pane also includes a 'Result >>' for each sub-object and item that has a difference.
Sometimes it may be difficult to determine enough about the difference to figure out what is going on. If so, you can check the Show more details checkbox to see more about the difference.
The lower pane tries to present the information as concisely but informatively as possible. Apologies if the information seems a bit hard to interpret at times. One area that might be confusing is when there are differences in 'lists'. Many of Gramps objects have lists of other objects, families have lists of children, persons have lists of events or notes etc. When a list has changes, the left side will have a list position number preceded by '#'. For example:
Person, Event References #1, Event | Original >> | your tree Event: [E0012] Death: Smith Gus |
Imported >> | your tree Event: [E0011] Birth: Smith Gus | |
Person, Event References #2, Event | Original >> | your tree Event: [E0011] Birth: Smith Gus |
Imported >> | your tree Event: [E0012] Death: Smith Gus |
In this example list of event references, the imported tree somehow got the Birth and Death events swapped, compared to the original tree.
Control buttons
Help brings up this wiki.
Edit import allows editing of the selected item. Any changes are temporarily saved in your imported data, and will be kept only if that data is merged into your current tree. If your editing creates new items, they will appear in the top pane list when you finish editing. Changes here cause the 'Action' to be cleared, so be sure to choose an Action when finished.
Unmark clears any current Action. If an item remains unmarked when you are done, it will be treated as if 'Ignored', that is, there will be no changes to your tree for that item.
Ignore sets the action for the item so that there will be no changes to your tree for that item. This is the same as leaving the item unmarked, but makes it clear that you have looked at that item.
Done tells the tool that you are finished marking the various actions. The tool will bring up a confirmation dialog, allowing you to save your work, cancel and return to marking, or abandon it.
The remaining buttons are considered 'Actions' and depend on the Status of the item.
Add appears for items with an 'Added' status. When added, the imported data will be added to your current tree.
Delete appears for items with a 'Missing' status. When selected, the data in your tree for this item will be deleted.
Replace appears for items with a 'Different' status. When replaced, the imported data for the item will completely replace the same data in your current tree.
Merge Original appears for items with a 'Different' status. When selected, the imported data for the item will be merged with the original tree and the IDs of the original item are retained. The portions of the data that cannot be merged will remain as it is in your current tree data.
Merge Import appears for items with a 'Different' status. When selected, the imported data for the item will be merged with the original tree and the IDs of the imported item are used. The portions of the data that cannot be merged will be replaced by the import data.
When you select an 'Action' with one of the buttons, a hint will appear that states the action in more detail.
You can change the 'Action' at any time on any item, it may be instructive to see the results of choosing the various choices in the lower pane.
In general I recommend that you start marking your items with the 'Different' items first. Within these, doing Families and Persons first will usually minimize the amount of work to merge your tree. This tool starts up with the sort set to put these items first, so starting at the top is easy.
Automatic Mark
This tool includes the ability to automatically mark those items that are 'referred to' by the item you are currently marking. For Example, if you Add a person, the events, notes, media, citations and families that the person refers to will also be added. The same process is applied for each of the various 'Action' buttons.
The marks that are automatically generated are shown in the top pane action column with a '*' preceding the action.
Occasionally you may mark items that create a conflict in the automatic marking. For example you Add a person that refers to a media, and then later Ignore a person that also refers to the same media. In this case, the media item will be marked with a '?' and the action that tends to conserve more data will be used. This means that Adds have priority over Ignore, Ignore over Delete etc.
You should review any items marked this way to make sure that you agree with the program.
Note that families and persons in particular usually have a lot of references to other families or persons. In fact if your imported file contains a whole new branch of your family tree, selecting any one of the people or families in that branch will automatically mark all the new people and families at once. This is usually desirable, but not always.
Automatic mark with Families
This tool is not ideal for selecting only parts of a tree for merging, but it can be done. If you wish to include a person, but not his family (spouse and children) then you can uncheck the Automark Families checkbox. This applies to all persons found via the Automark as well. The effect is that you can get ancestors (if Automark Parent Families is checked) included but not all the descendants of those ancestors.
If you wish to get a set of spouses and descendants included when adding a person but not all the ancestors of the spouses, then you can uncheck the Automark Parent Families checkbox and leave the Automark Families unchecked. If you don't want any family or parent families automatically included, leave both Automark Parent Families and Automark Families unchecked.
Finishing up
When you have marked up as many items as you want and reviewed the automatic marks to make sure they are what you desire, you can press the Done button.
The tool will bring up a confirmation dialog, allowing you to save your work, cancel and return to marking, or abandon it. If you abandon, your markings are lost and no changes are made to your current tree.
If you decide to save your work, the tool will perform a series of checks as it saves your work. These checks make sure that any conflicts that may have been created will not result in errors in your database. For example, if you selected a person to add, but decided to ignore a note attached to that person, the tool will 'clean up' the reference to the note on the person before saving it.
FAQ
- Why would I want to use the Edit import button?
- If you had an item with a difference, where both the current tree and the imported tree made changes. Because of some limitations of Gramps and this tool, you cannot choose between current and imported for every detail within an object. For these details, you can only choose to keep your current data with Ignore or Merge Original, or use the imported data with Replace or Merge Import. In either case some changes will be lost. For example, if you have edited a Person and added a Nickname, and the imported tree had edited the same person to add a LDS Ordinance, you can only keep the information for one side. In this case you might want to edit the imported person to change it to add the Nickname so that data will not be lost.
- Why does the import file have to be derived from the same original source?
- If it is not, then the matching process will not find any items that are the same or even different. Every item in the import file will end up showing as 'Added', and every item in the current tree will show as 'Missing'. You can use this tool to add in part or all of the imported data, but it is not ideal for this purpose. You would be better importing the data into a new tree, using all the Gramps views and filters to find what you do or don't want, make any deletions or edits and export to a new XML before importing that into your tree the usual way.
- When would it make sense to use 'Merge Original' vs 'Merge Imported'?
- The difference between 'Merge Original' or 'Merge Imported' is which Gramps IDs are kept as the final object. If you would like to keep your starting family tree the way you have set it up, use Merge Original. If you think that the imported tree is better, has better IDs etc, then use the Merge Imported.
- Why can't you use a GEDCOM file as the import source?
- One of the things that make this tool work is that every 'object' in the Gramps database has a unique ID called a handle (this is different than the ID shown in the GUI for each object). The tool can compare objects with matching handles relatively easily, much more easily than guessing if John Smith in your tree is the one from 1722, 1854, or 2012.
- At this time Gramps GEDCOM import and export has no way to save these handles to make this sort of matching possible. A few of the other family tree tools can import and export the so called '_UID' GEDCOM tag (which is not standardized) which might be usable for this, but most tools will simply discard them or modify them to be unusable. Even if these were available and not lost in translation, Gramps data model does not always match with GEDCOM data model. So a GEDCOM export/import, even without any changes made by the user, would have a lot of differences.
Notes
If you are not sure exactly what an imported tree contains, you should use Gramps to examine it before using this tool. Import the tree into a new family tree, and look it over with the various charts (I like the relationship chart myself) to see if there are branches that you don't want. It is frequently easier to use Gramps filters or editing tools to remove items than to use the tool to exclude them, although the tool will suffice for simpler cases. Even then you should take notes on the IDs of people or families that you don't want, to make it easier to find them with this tool.
This is a new tool and fairly complex. Please report any issues or desired enhancements to paulr2787 at gmail.com
I have included a French translation of the tool, mostly as a test of the internationalization code. Since I don't speak French (USA Texas version of English only), please excuse my French.
See also
- Database Differences Report - allows a mostly complete comparison between the currently opened database (Family Tree) and a Gramps XML file.