SQL Latency and Throughput

Our approach to SQL performance benchmarking is to use sysbench, an industry standard benchmkarking tool.

Performance Roadmap

Dolt is slower than MySQL. The goal is to get Dolt to within 2-4 times the speed of MySQL for common operations. If a query takes MySQL 1 second, we expect it to take Dolt 2-4 seconds. Or, if MySQL can run 8 queries in 10 seconds, then we want Dolt to run 2-4 queries in 10 seconds. The multiple column represents this relationship with regard to a particular benchmark.

It's important recognize that these are industry standard tests, and are OLTP oriented. Many Dolt use-cases are non-OLTP, and Dolt is fast for bulk operations common in, for example, data pipeline contexts.

Benchmark Data

Below are the results of running sysbench MySQL tests against Dolt SQL Server for the most recent release of Dolt. We will update this with every release. The tests attempt to run as many queries as possible in a fixed 2 minute time window. The Dolt and MySQL columns show the median latency of each query during that 2 minute time window.

The Dolt version is 0.26.9.

Read Tests

Dolt

MySQL

Multiple

covering_index_scan

9.91

1.44

7.0

index_scan

132.49

34.95

4.0

oltp_point_select

1.67

0.14

12.0

oltp_read_only

34.33

2.66

13.0

select_random_points

2.86

0.27

11.0

select_random_ranges

3.68

0.3

12.0

table_scan

142.39

34.95

4.0

mean

9.0

Write Tests

Dolt

MySQL

Multiple

bulk_insert

0.001

0.001

1.0

oltp_delete

10.27

0.14

73.0

oltp_insert

12.52

3.25

4.0

oltp_read_write

73.13

7.17

10.0

oltp_update_index

14.46

3.49

4.0

oltp_update_non_index

8.28

3.62

2.0

oltp_write_only

40.37

4.65

9.0

mean

14.71

Overall Mean Multiple

11.86