By rudrap on Apr 05, 2014
MySQL Fabric Testing :
MySQL Fabric is a simple and easy to use management system to handle High Availability and Sharding. MySQL SysQA team is responsible to test the MySQL Fabric at a system level.We do
1) Feature/Functional/Regression Testing
2) Complex/Integration/concurrent Testing
3) Scalability testing
4) Performance/Atomic Testing
5) Fabric Recovery/Security Testing
There are few test scenario examples below that the team is performed to get a stable,robust application.
MySQL Fabric Testing Examples:
1) multiple activities performed at same time. e.g.
promote a slave
insert records in a shard
demote another group
remove master from the group
add old master to the group
2) promote and demote for 100 times
3) scalability testing for 20 mn and 50 mn records and run activities to ensure fabric does not crash
4)1000 connections at the same time to check if there is crash.
5)comparing performance of conn/python,conn/j fabric support against conn/python,conn/j
6) fabric server setup using ndbCluster
7) DDLs on a sharded table
8) 100-500 servers in multiple machines with 50 groups and 45 shards (will change based on topology) to ensure there is no crash
9) 10+ activities in parallel to check if there is any hangs in activities
10) 20+ activites one after another
11) Fabric tests integration with mysqlutilities (failover, mysqlreplicate,etc) (future scope)
12) Backup of shard table (using multisource replication) (future scope)
13) 10 tables sharded using a single mapping and do DDLs and DMLs in each shard
14) Fabric test with DDLs and DMLs in backend
15) Minor updown of servers for each master or slave (future scope)
How We did it?
Created/updated an Automated and Multi Threaded program to run the Fabric Activities and transactions in parallel.
- Creating Hudson jobs to run the tests regularly
- Crash testing and Randomize testing for MySQL Fabric server farm.