QCP Project Rules
Contents |
Quality Contribution Program - Project Rules
Accepted Contributions
Bug reports
A verified Bug report is submitted through the MySQL bugs web interface and consists of:
- a concise definition of the bug
- a clear sequence of steps to repeat the bug
For additional value, a bug report may also have
- a test case pluggable to the test suite with its expected result;
- a workaround to get the normal functionality;
- a knowledgeable suggestion of how to fix the bug.
Bug reports are considered more valuable if they can be verified quickly, and that usually means that they contain all elements to reproduce the bug, determine how serious it is and which priority it should receive in our tasks.
Code patches
Code patches are associated with bug reports. Contributors may submit patches to fix a bug or to improve an existing feature, or even to create a new feature.
When there is no bug associated to a patch, a feature request must be submitted. Therefore, a patch is always dependent on a bug report. It will be possible to submit patches for bugs reported by other contributors. Test material
Test cases can be accepted in three different ways.
- The preferred form is the format used by the MySQL test suite, consisting of pair of files: a test sequence and the expected result, as explained in the manual;
- Also acceptable are test cases using the Test Anything Protocol as explained by Mats Kindahl in his testing docs;
- Finally, we will welcome complex cases that test the database in a concurrency scenario, such as the internal system stress test.
To be accepted, test cases must be easily pluggable into the test suite (or, in well defined cases, easily runnable outside the suite). A non-pluggable one can be accepted with lower evaluation if it can be converted with minimal effort. Preferably, test cases should not have platform specific requirements.
In the QA Contributor rules we will add details to specify exactly what makes a test well accepted and rewarded. For now, it suffices to say that a test case will receive additional points if:
- it finds a bug;
- it is well documented;
- it is engine agnostic, i.e. it can be adapted to another engine by simply changing one variable (it will require a tutorial that explains how to do it).
- It is systematic and covers many data types/combination of transactions, permissions, etc);
Test cases that are hard to understand can be refused (e.g. a test containing hundreds of queries with no evident purpose).
Test cases can be submitted in two circumstances:
- to complement a bug (or feature) report. Thus, a test case can be submitted as comment to an existing bug.
- to satisfy a test plan published by the QA department (see action item below);
Points Calculation
bugs related issues
1 if the bug is verified - else no further evaluation + 5 if there is a clearly repeatable test + 2 if the test is a script + 2 to 15 if the script is pluggable to the test suite (see below) + 1 * (5 - Severity) (but severity 5 is more valuable than 4) + 2 * (4 - Priority) + 1 to 3 if there is a workaround + 2 if there are clear indications on how to solve it + 3 to 20 for an *accepted* patch that fixes the bug or matches a feature request + 5 to 20 for an associated functional test + up to 10 bonus points for exceptionally good contributions
Patches should possibly come with an associated functional test. While we will accept them without a test, we will encourage the practice of submitting a patch + a test, by rewarding the inclusion with a higher reward.
Testing related issues
For a test case that is associated to a bug report, the following evaluation apply:
1 if the test is not pluggable or 2 to 5 if the test is pluggable or accepted as extra-suite + 0 to 3 for the quality of the documentation + 10 if it is engine agnostic + 10 for a concurrency test
Additionally, test cases submitted on a test plan defined by our QA department will receive 3 extra points for each.
Submission of QA material
QA contributions will be submitted mainly through the bugs system. Code patches and complex test cases will need to be discusses in the internals mailing list (as described in Contributing). However, they need to be associated to a bug report. If the patch fixes a bug, the bug report is there. For a new feature, there must exist a bug report marked as a "feature request". The same is feasible for complex test cases. When in doubt, discuss first in the internals list.
Levels and rewards
Quality contributors will be ranged into levels that mirror the Enterprise ones.
- Candidate : someone who we have selected as potential contributor (based on previous contributions), or someone who has expressed interest to the plan
- Basic : a proven contributor, who has submitted material, earning at least 50 QA points;
- Silver : a more skilled contributor, earning at least 200 QA points
- Gold : a super contributor, who has earned at least 500 QA points
- Platinum : The contributor that makes the headlines, having delivered contributions up to 1000 QA points
To be part of the project, each contributor must agree to the terms of the project itself.
Points will be calculated on a yearly basis. The total points accrued must be referred to bugs submitted within the last 12 months.
Contributors who agree to have their name published will be listed in the Quality Contributor Program pages. Contributors from level Basic to Platinum will receive a free subscription to the corresponding level of MySQL Enterprise. Existing customers who are also contributors will be rewarded with alternative prizes.
Legal Issues
Contributors supplying test cases and bug patches must agree to the MySQL Contributor License Agreement. Each contribution becomes legal property of MySQL AB, which will then be free to use the contributed material in its source code.