MySQL Forge

Resources for the MySQL Community

ParallelMySQLDump

Contents 1 Adding parallelism to mysqldump 1.1 Synopsys 1.2 Parallelization 1.3 Timeline 1.4 Deliverables

[edit] Adding parallelism to mysqldump

This is part of the Summer of Code 2008 project proposals.

[edit] Synopsys

The idea of this project is to parallelize 'mysqldump' program. Currently, MySQL is fully multi-threaded and can handle larger databases. But dumping these worst-case scenarios can be really painful. By parallelizing the mysqldump tool, significant performance improvements can be achieved for both power and small users of MySQL.

[edit] Parallelization

Incremental steps to achieve parallelizaton:

1. For each table, fork a process that will dump it into its own file (considering a single machine running the program).
2. Limit the number of process created in a single machine.
3. Create a limited number of concurrent threads inside each process.
4. Limit the number of concurrent threads dumping.
5. Add support to multi-computer parallelism.
6. [OPTIONAL] Enable across-machine dumps (dumps generated by one machine are moved to another).

[edit] Timeline

1. Examine mysqldump code - until May, 23
2. Parallel version of mysqldump (step S4 completed) working and tested - until mid-June*
3. Parallel version of mysqldump (step S7 completed) working and tested - until mid-August

• To be delivered also to Multi-core Systems course

[edit] Deliverables

1. A parallelized version of mysqldump
2. Updated version of mysqldump page at MySQL Reference Manual