ReplicationFeatures/ParallelSlave
Contents |
[edit] Parallel execution of row-based replication events on slave
[edit] Provides:
Increases throughput of replication in terms of transactions per second. Reduces replication latency.
[edit] Use case:
Currently, replication does not scale well on multi-core machines. The single slave thread execute replication events one by one and have sometimes problems with a load produced on the master by concurrent multiple client connections. MySQL Cluster Replication benefits from parallelization as well because workload on the master side is generated by multiple machines.
[edit] Methods of parallelization:
Intra-transactional parallelization. We have chosen to implement first the intra-transaction parallelization. That means events of a transaction (or a sequence of transactions) are executed by multiple threads on the slave side. Each worker thread starts a subtransaction when it gets its the first hashed-out rows event.
The subtransactions commit using 2PC when the parent transaction reaches its COMMIT. The model provides strict consistency (the slave never gets into a state of data that has not existed on the master), only if slave transactions are executed in SERIALIZABLE isolation level (if another isolation level is used, then the slave transaction may see non-committed data).
Inter-transactional parallelization. The inter-transaction parallelization is at design phase.
[edit] Limitations:
Limitations that will be removed in later versions:
- May crash
- Tables must have primary key of an integer type
- Only parallel execution for INSERTs and DELETEs, not for UPDATEs. (No support for Update_rows event)
- If slave crashes during commit, then only part of a transaction can have been committed.
Limitations that will not be removed soon:
- Only works with transactional storage engines
- Only works with row-based events
- All transactions on slave should use SERIALIZABLE isolation level
[edit] User interface:
- The server configuration parameter
--slave_parallel_workers=..., zero by default
- SHOW PROCESSLIST displays all engaged workers
- Parallel mode is not active by default, to set it invoke
set @@global.slave_exec_mode='strict,parallel';
[edit] References
- WL#4648 Multi-threaded slave for row-based replication
[edit] Downloads
- Pre-release 0.1
[edit] Thanks:
MySQL Replication and MySQL Cluster Team, including Mats, Lars, Andrei, Tomas, Jonas, Mikael.