Storage Engine API Testing Framework
This page is under progress
Original Author: Nidhi Shrotriya (nidhi.shrotriya@sun.com)
Contents |
[edit] Version History
Date Comments 2008-07-14 Created and Posted to wiki (First draft) 2008-09-09 Updated by 'Nidhi Shrotriya' (Working draft)
[edit] What is API Testing
An API (Application Programming Interface) is a collection of software functions and procedures, called API calls, that can be executed by other software applications.
You need to setup initial environment, invoke API with required set of parameters and then finally analyze the result. Initial condition in API testing also involves creating conditions under which API will be called. Probably, API can be called directly or it can be called because of some event or in response of some exception. Output of API could be some data or status or it can just wait for some other call to complete in a-synchronized environment. Most of the test cases of API will be based on the output.
[edit] Context ?
API in our context -> handler.h (this is the minimum, we might also need to test some critical parts from mysql_priv.h, plugin.h or mysql.h)
- In handler.h, we have:
- virtual private methods (for the inheritance interface) implemented by Storage Engines.
- ex: write_row()
- public wrappers (interface to users of handler class) over these methods.
- ex: ha_write_row()
[edit] What we plan to test here
What we plan to test here is the API implementation independent of SEs.
- In particular,
- On performing some operation, the appropriate flow(order) of calls made b/w API & SE.
- We are testing the output of API while mocking the output from SE. This could be some data or status (permissible/not permissible).
- API performs the job it is supposed to do before & after making the call to SE. To cover all conditional blocks.
[edit] Purpose to Test
- Factorizing the failures from server side or SE side.
- To test all API calls (not implemented by any engine).
- Test new additions/changes to API before any SE (for ex: AQT).
[edit] What we *do not* plan to test here
- We are not testing the SEs (or the TestSE) here.
- This is also independent of the API definition check carried out at the build time (https://intranet.mysql.com/worklog/?tid=4380).
[edit] How to Test?
[edit] Framework
Test Framework would consist of following units -
- TestSE (Dummy Storage Engine for API Testing).
- It would contain all APIs.
- It would contain some good/bad data (return values for ex: 0, -1 for return status). Might also use some kind of definition file.
- It would contain some logging.
- Logging into handler.cc (ifdef block could be added to enable the logging only for testing)- Gotcha: In scenarios such as a call to close() function call which are called from places other than handler.cc, such logging would not reflect the method calls made from handler.cc
- Client program (Client tests would be added to tests/mysql_client_test.c and called from mysql-test/t/mysql_client_test.test wrapper)
[edit] How it works
