Debugging a running Dolt server can be challenging. This document covers the debugging basics and how to diagnose what is happening from common symptoms.
Dolt is constantly evolving. We release a new Dolt approximately once a week. Connect to the SQL server and run
select dolt_version(). Make sure the version matches the latest as seen on the GitHub releases page.
To upgrade the server, download the latest Dolt binary for your platform and replace the Dolt binary on your
PATHwith the downloaded one. Running the install process on most platforms again will do this for you. Restart the Dolt server using
dolt sql-serverto have your running server start using the latest binary.
Dolt consumes CPU, Memory, and Disk. Consuming more of any of these resources than the host has available can lead to degraded performance. Use your system's built in resource monitoring systems to inspect Dolt's usage of these resources. You may need a larger host or additional read replicas to support your load.
To see queries being run against the server, query results, and query latency set your Dolt log level to
DEBUG. This can be done by starting the server like so
dolt sql-server --loglevel=debugor by setting
log_level: debugin your
Dolt supports the SQL
EXPLAINoperation in order for you to see the plan for complex queries. Rearranging your query to perform fewer
JOINs or make better use of indexes can help speed up complex queries.
If you run into any issues requiring engineering attention, please submit a GitHub Issue to the Dolt project. Please be as detailed as possible in your report. Note the schema of the database and query or queries that can be used to trigger the issue. If possible, push the database to DoltHub so we can use a clone to reproduce the issue.
Dolt operational issues usually manifest as slow SQL queries. In rare occasions, Dolt may consume more of your system's resources than you expect. In these cases, this document has some recommendations.
Dolt creates disk garbage on write. This can sometimes become a substantial portion of the disk Dolt is consuming. Dolt ships with a garbage collection function. Running the garbage collection function can free disk.
To run garbage collection, you should first stop your running server. It is not a safe operation to run concurrently with server write operations. Once stopped, navigate to the Dolt directory where your database is stored and run
dolt gc. Once the operation is complete, restart your server using
Online garbage collection is on the roadmap and we will be implementing it in the back half of 2022.
- Using primary keys with random values. Inserts into indexes with random values guarantees that edits will occur all throughout the index instead of being clustered around the same key space. This results in a rewrite of the prolly tree thereby increasing storage disproportionately to the delta of the changes.
- Adding a column to a table. A new column forks the storage of the table resulting in a loss of structural sharing. Dolt is row major and builds chunks for each primary key, row values pair. The row values encodes the schema length so every row now requires a new chunk.
A Dolt server requires approximately 1% of the disk size of the database in memory at minimum. So, a 100GB database should have at least 1GB of RAM but preferably more. We recommend provisioning approximately 10% of the disk size of the database as memory.
A query may cause Dolt to grow memory use unbounded and then eventually crash the server. If you discover one such queries, please submit a GitHub Issue. Such queries should be rare but not impossible, especially with complex queries containing multiple
Dolt may not free memory efficiently. If your Dolt server grows memory use unbounded over time and then frees the memory upon restart, you have discoverd a memory leak. Again, please submit a GitHub issue. Memory leaks should be rare and we treat memory leak fixes as high priority.
Under too much concurrent load, Dolt may consume all the CPU on a host. This is likely caused by too much read concurrency. In this case, create more read replicas and load balance your reads among your replicas.
Currently, Dolt is not a high throughput write database. Given the current transaction model, each transaction is treated as a
mergeand requires a global write lock to perform that merge. Dolt can easily become overwhelmed with too many writes. This is an issue we will be tackling in the back half of 2022 using row level locking and other common transactional database strategies.