loader - parallel data loading

Loader splits the file into -s splits and loads the data in parallel from -s threads on the database nodes specified. Parallelism can be increased by specifying a higher number for -s (e.g. 32), we will then have 32 threads applying data. If for e.g. 4 database nodes are specified, each node will have 8 applier threads inserting the data.

Loader is good when you have a large dataset and want to reduce the time it takes to import data compared to importing data sequentially using methods like mysqlimport or LOAD DATA INFILE.

For MySQL Cluster, it is faster and less error prone than doing sequential

ALTER TABLE t1 engine=ndb

on innodb/myisam tables or importing the db schema and data using a single large mysqldump.



Loader is available on GitHub
Drop an email to [email protected] if you have problems or suggestions how to improve this utility.