DGCov Documentation
The DGCov tool is used to test that a patch for the MySQL server is fully covered by the testsuite. This can be used to help determine if further test cases needs to be added to the patch before pushing to the main source tree. At MySQL AB, we also run this tool automatically on every code change to check for coverage (using or daily build tool PushBuild).
Contents |
[edit] Background
DGCov uses GCC code instrumentation and gcov to determine code coverage of the previously executed test run(s). It then matches this against a BitKeeper report (bk annotate) about which lines have been changed locally. Any lines touched and not executed during test runs are reported.
Typical usage is to run it to check local changes for coverage before committing/pushing. First, clean out any old coverage results and rebuild with proper GCC instrumentation:
bk citool perl ~/internals/dev/dgcov/dgcov.pl --purge BUILD/compile-pentium-gcov
Note: On some systems it could be necessary to remove $static_link from extra_configs in BUILD/compile-pentium-gcov.
Note: If you change the code and want to build again, do not just run 'make'. You need to prepare the environment like BUILD/compile-pentium-gcov does. So run
CCACHE_DISABLE=1 LDFLAGS="-fprofile-arcs -ftest-coverage" make
Then run the test suite and check for any missing coverage:
make test-force perl ~/internals/dev/dgcov/dgcov.pl --uncommitted
This will check all lines changed (but not committed) for code coverage. This are the lines that occur in 'bk -r diffs'.
Or to check all changesets committed locally, but not yet pushed to the parent:
make test-force perl ~/internals/dev/dgcov/dgcov.pl --local
It is also possible to list specific changeset(s) to check on the command line, or to combine --uncommitted, --local, and explicitly listed changesets.
Run dgcov.pl --help for a quick overview of usage and available command line arguments.
This tool is an implementation of [http://forge.mysql.com/worklog/task.php?id=3285 WL#3285] [Internal MySQL link].
[edit] Annotating dead code
Parts of a patch may not be possible to cover by test suite, eg. dead code or error conditions than cannot be made to occur in normal usage. Such lines can be excluded from reports using annotations:
if((p= malloc(10)) == NULL) return 0; /* purecov: inspected */
/* purecov: begin deadcode */ tmp= x; x= y; y= tmp; /* purecov: end */
Note: If a code statement spans multiple lines, use begin-end annotations. Otherwise the annotation would have to be on the first line of the statement.
Available annotations are:
- inspected
- For code that cannot be covered (like out of memory conditions), but which has been reviewed and is considered correct.
- deadcode
- Unreachable code.
- tested
- Code that is not covered by automatic tests, but which has been manually tested.
[edit] Interpreting the output
Here is some sample output from the tool:
File: sql/ha_ndbcluster_binlog.cc
-------------------------------------------------------------------------------
. -: 138:
. +++++: 139: for (int i= 0; i < n && pos < 20; i++)
. -: 140: {
| #####: 141: pos+= sprintf(&buf[[[pos]]]," %x", (int) (uchar) field_ptr[[[i]]]);
. -: 142: }
. +++++: 143: buf[[[pos]]]= 0;
. +++++: 144: DBUG_PRINT("info",("[[[%u]]]field_ptr[[[0->%d]]]: %s", j, n, buf));
The lines prefixed with '|' are changed lines, while lines prefixed with '.' are just context lines that were not changed in any of the changesets. The number of context lines printed can be adjusted with the --context=N option.
The line marked with '#####' denotes a line that was changed, but not executed during the test runs. Such lines thus indicate parts of a patch that are not covered by tests. In code that was not changed (ie. context lines), such missing coverage is marked with +++++ to easily distinguish them.
By default, only changed lines with no coverage are reported. With the --verbose switch, all lines are printed.
Another example:
File: sql/filesort.cc
-------------------------------------------------------------------------------
| 21542: 699: diff=(int) (sort_field_length - length);
| 21542: 700: if (diff < 0)
| -: 701: {
| 2034: 702: diff=0; /* purecov: inspected */
| 2034: 703: length= sort_field_length;
| -: 704: }
| 21542: 705: if (sort_field->suffix_length)
Here, line number 702 has been annotated as not covered, but has nevertheless been executed 2034 times during the tests. So the annotation is redundant, and is therefore reported as an error.
The lines annotated with '-' are lines for which no code is generated, and which therefore has no coverage information.
Note: The final count of not covered lines may not match the number of '#####' marked lines. Even though some lines may have purecov annotations, they can be shown if they are close to other lines which are reported.
[edit] Ideas for future improvements
The checking of not yet committed changesets works by applying the diff obtained from 'bk -r diffs' to the list of changed lines obtained from 'bk annotate'. So it does not consider checked-in, but not yet committed deltas.
This could be fixed by running 'bk pending', and add any lines changed in checked-in, but not committed file revisions. This should be done before applying the diff from 'bk -r diffs'.