Difference between revisions of "Testing Gramps"
(update sourceforge links to github) |
|||
| Line 15: | Line 15: | ||
=Tests and frameworks used= | =Tests and frameworks used= | ||
== Manual test plan == | == Manual test plan == | ||
| − | See [ | + | See [https://github.com/gramps-project/gramps/blob/master/TestPlan.txt TestPlan.txt] in gramps toplevel. (I believe this is only done at a major release (like Gramps version 4.0.0)). |
== Specialized scripts for testing == | == Specialized scripts for testing == | ||
| − | See more specialized scripts in [ | + | See more specialized scripts in [https://github.com/gramps-project/gramps/tree/master/test test/], status unknown. |
| − | Contents of [ | + | Contents of [https://github.com/gramps-project/gramps/tree/master/test test/] : |
* GrampsLogger/ | * GrampsLogger/ | ||
** ErrorReportAssistant_Test.py | ** ErrorReportAssistant_Test.py | ||
| Line 37: | Line 37: | ||
==== test/runtest.sh ==== | ==== test/runtest.sh ==== | ||
| − | '''[ | + | '''[https://github.com/gramps-project/gramps/blob/master/test/runtest.sh test/runtest.sh]''' - Report test for Gramps: Generate every report in every format. Runs all possible reports using the report cli interface, based on the [[example.gramps]] database. This test is not fully self-contained, in that it depends on various environment settings, such as your locale, your preferred name display formats, and your report options. Last, but not the least, the verification of the resulting reports is entirely manual. |
Bugs tagged as [http://www.gramps-project.org/bugs/search.php?tag_string=found-by-runtest.sh found-by-runtest.sh] | Bugs tagged as [http://www.gramps-project.org/bugs/search.php?tag_string=found-by-runtest.sh found-by-runtest.sh] | ||
===test/impex.sh=== | ===test/impex.sh=== | ||
| − | '''[ | + | '''[https://github.com/gramps-project/gramps/blob/master/test/impex.sh test/impex.sh]''' - Import/export test for Gramps. |
From the file header: | From the file header: | ||
| Line 57: | Line 57: | ||
== Unit testing == | == Unit testing == | ||
=== test/RunAllTests.py === | === test/RunAllTests.py === | ||
| − | '''[ | + | '''[https://github.com/gramps-project/gramps/blob/master/test/RunAllTests.py test/RunAllTests.py]''' - Testing framework for performing a variety of unittests for Gramps. Runs out-of-tree (not in gramps/) testing code, by looking for any test/*_Test.py files and executing the test suites therein. See the current code in test/*_Test.py for example and python standard unittest docs. |
{{man note|Starting with gramps4.x branch|these tests include non-automated unit tests only. The automated unit tests are all under gramps/.}} | {{man note|Starting with gramps4.x branch|these tests include non-automated unit tests only. The automated unit tests are all under gramps/.}} | ||
| Line 216: | Line 216: | ||
! Comments | ! Comments | ||
|- | |- | ||
| − | |[ | + | |[https://github.com/gramps-project/gramps/tree/master/gramps/gen/relationship.py#l1889 gramps/gen/relationship.py] |
|bgcolor="#ff8080"| To Do | |bgcolor="#ff8080"| To Do | ||
|Relationship calculator | |Relationship calculator | ||
Revision as of 03:10, 16 May 2020
Test status
As of Jan 2015 we don't have a record of tests executed, the platforms and environments they were run upon, and what code they covered. The only indirect evidence is available in open bugs, when people care to fill in these details. :-(
Possible improvements
- Unify running all the tests, in- or out- tree
- Try switching the feature tests and non-automated unit tests in test/ from our runner scripts to python native unittest discovery mechanism.
- We have a stale gramps/test/regrtest.py runner, notable for logging init. - should we revive that, or maybe integrate into setup.py test runner?
- Coverage analysis
- Continuous test status report, coverage, automatic deployment into win/mac/linux VMs (needs server capacity to be hosted online)(I can dream, can't I?)8294
- Automated regression tests for our GUI. The following links look interesting:
Tests and frameworks used
Manual test plan
See TestPlan.txt in gramps toplevel. (I believe this is only done at a major release (like Gramps version 4.0.0)).
Specialized scripts for testing
See more specialized scripts in test/, status unknown.
Contents of test/ :
- GrampsLogger/
- ErrorReportAssistant_Test.py
- GtkHandler_Test.py
- LosHawlos_bsddbtest.py
- LosHawlos_dbtest.py
- RunAllTests.py
- dates.sh
- det_ancestor_report.sh
- det_descendant_report.sh
- impex.sh
- runtest.sh
- tools.sh
testing of reports
test/runtest.sh
test/runtest.sh - Report test for Gramps: Generate every report in every format. Runs all possible reports using the report cli interface, based on the example.gramps database. This test is not fully self-contained, in that it depends on various environment settings, such as your locale, your preferred name display formats, and your report options. Last, but not the least, the verification of the resulting reports is entirely manual.
Bugs tagged as found-by-runtest.sh
test/impex.sh
test/impex.sh - Import/export test for Gramps.
From the file header:
- Import example XML data and create internal Gramps DB
- Open produced Gramps DB, then
- check data for integrity
- output in all formats
- Check resulting XML for well-formedness and validate it against DTD and RelaxNG schema.
- Import every exported file produced if the format is also supported for import, and run a text summary report.
- Diff each report with the summary of the produced example DB.
Bugs tagged as found-by-impex.sh
Unit testing
test/RunAllTests.py
test/RunAllTests.py - Testing framework for performing a variety of unittests for Gramps. Runs out-of-tree (not in gramps/) testing code, by looking for any test/*_Test.py files and executing the test suites therein. See the current code in test/*_Test.py for example and python standard unittest docs.
|
Starting with gramps4.x branch these tests include non-automated unit tests only. The automated unit tests are all under gramps/. |
Bugs tagged as found-by-RunAllTests.py
|
GtkHandler testing code pops up the Gramps error dialog but this is actually for testing the error reporting itself. Don't be scared by the dialog, it's expected. Your manual work is required to close the dialogs with the "Cancel" button. The relevant tests still pass (unless there's another bug there)... |
unit tests in the main tree
See Unit Test Quickstart for detailed running instructions.
using - python setup.py test
python setup.py test
| File | Test | Status | Comments |
|---|---|---|---|
| gramps/cli/test/cli_test.py | 3 | OK | |
| gramps/cli/test/user_test.py | 8 | OK | (6 test require mocking to run) |
| gramps/gen/test/config_test.py | 1 | OK | |
| gramps/gen/test/constfunc_test.py | 1 | OK | (linux only, skipped elsewhere) |
| gramps/gen/test/user_test.py | 1 | OK | See #7013 for context |
| gramps/gen/datehandler/test/datehandler_test.py | 9 | OK | |
| gramps/gen/db/test/cursor_test.py | 4 | OK | 2 skipped |
| gramps/gen/db/test/db_test.py | 6 | OK | |
| gramps/gen/db/test/reference_map_test.py | 5 | OK | |
| gramps/gen/lib/test/date_test.py | 8 | OK | (locale-based, OK for en, fails for some other locales) |
| gramps/gen/lib/test/grampstype_test.py | 3 | OK | |
| gramps/gen/lib/test/merge_test.py | 201 | FAIL | 2 failed skipped (7027- Bug shows resolved) |
| gramps/gen/merge/test/merge_ref_test.py | 53 | OK | |
| gramps/gen/utils/test/callback_test.py | 10 | OK | |
| gramps/gen/utils/test/keyword_test.py | 4 | OK | |
| gramps/gen/utils/test/place_test.py | 28 | FAIL | 4 failures (bug #7044 - Bug shows resolved) |
| gramps/gui/logger/test/rotate_handler_test.py | 2 | OK | |
| gramps/gui/test/user_test.py | 1 | OK | (requires mocking to run) |
| gramps/plugins/export/test/exportvcard_test.py | 28 | OK | |
| gramps/plugins/importer/test/importvcard_test.py | 66 | OK | |
| gramps/plugins/importer/test/importxml_test.py | 4 | OK | |
| gramps/plugins/importer/test/importgeneweb_test.py | 2 | OK | |
| gramps/test/test/gedread_util_test.py | No longer required? | ||
| gramps/test/test/test_util_test.py | No longer required? | ||
| gramps/webapp/grampsdb/view/png.py |
|
|
Note from Nick Hall Some of the merge tests are slow because they actually run Gramps from the command line to import and export files. |
semi-interactive
There is also semi-interactive testing via __main__ in some code:
| File | Status | Comments |
|---|---|---|
| gramps/gen/relationship.py | To Do | Relationship calculator |
| gramps/gen/db/bsddbtxn.py | To Do | |
| gramps/gen/db/txn.py | To Do | |
| gramps/gen/db/undoredo.py | To Do | |
| gramps/gen/db/write.py | To Do | |
| gramps/gen/db/txn.py | To Do | |
| gramps/gen/lib/styledtext.py | To Do | |
| gramps/gen/utils/place.py | OK | Done |
| gramps/gen/utils/docgen/odstab.py | To Do | |
| gramps/gen/utils/docgen/csvtab.py | To Do | |
| gramps/gui/ddtargets.py | To Do | Not worth running? |
| gramps/gui/widgets/undoablebuffer.py | To Do | Not worth running? |
| gramps/gui/dialog.py | To Do | Interactive |
| gramps/gui/widgets/validatedmaskedentry.py | To Do | Interactive |
| gramps/gui/widgets/progressdialog.py | To Do | Interactive |
| gramps/gui/widgets/statusbar.py | To Do | Interactive |
| gramps/plugins/lib/libsubstkeyword.py | To Do | |
| gramps/plugins/rel/rel_*.py | To Do | Relationship calculator plugins |
| gramps/plugins/tool/phpgedviewconnector.py | To Do |
