LogoLogo
DoltHubBlogDiscordGitHubDolt
  • Introduction
    • What Is Dolt?
    • Installation
      • Linux
      • Windows
      • Mac
      • Build from Source
      • Application Server
      • Docker
      • Upgrading
    • Getting Started
      • Version Controlled Database
      • Git For Data
      • Versioned MySQL Replica
    • Use Cases
      • Data Sharing
      • Data and Model Quality Control
      • Manual Data Curation
      • Version Control for your Application
      • Versioned MySQL Replica
      • Audit
      • Configuration Management
      • Offline First
  • Concepts
    • Dolt
      • Git
        • Commits
        • Log
        • Diff
        • Branch
        • Merge
        • Conflicts
        • Remotes
        • Working Set
      • SQL
        • Databases
        • Schema
        • Tables
        • Primary Keys
        • Types
        • Indexes
        • Views
        • Constraints
        • Triggers
        • Procedures
        • Users/Grants
        • Transactions
        • System Variables
      • RDBMS
        • Server
        • Backups
        • Replication
    • DoltHub/DoltLab
      • Permissions
      • Pull Requests
      • Issues
      • Forks
  • SQL Reference
    • Running the Server
      • Configuration
      • Access Management
      • Branch Permissions
      • Backups
      • Garbage Collection
      • Metrics
      • Replication
      • Troubleshooting
    • Version Control Features
      • Using Branches
      • Merges
      • Querying History
      • Using Remotes
      • Procedures
      • Functions
      • System Tables
      • System Variables
      • Saved Queries
    • SQL Language Support
      • Data Description
      • Expressions, Functions, Operators
      • Supported Statements
      • MySQL Information Schema
      • Collations and Character Sets
      • System Variables
      • Miscellaneous
    • Supported Clients
      • Programmatic
      • SQL Editors
    • Benchmarks and Metrics
      • Correctness
      • Latency
      • Import
  • CLI Reference
    • Commands
    • Git Comparison
  • Architecture
    • Overview
    • Storage Engine
      • Commit Graph
      • Prolly Trees
      • Block Store
    • SQL
      • Go MySQL Server
      • Vitess
  • Guides
    • Cheat Sheet
    • Contributing
      • dolt
      • go-mysql-server
    • MySQL to Dolt Replication
    • Importing Data
    • Integrations
  • Other
    • FAQ
    • Roadmap
    • Versioning
  • Products
    • Hosted Dolt
      • Getting Started
      • Notable Features
      • SQL Workbench
      • Cloning a Hosted Database
      • Using DoltHub as a Remote
      • Infrastructure
    • DoltHub
      • Data Sharing
      • API
        • Authentication
        • SQL
        • CSV
        • Database
        • Hooks
      • Continuous Integration
        • Getting Started
        • Workflow Reference
      • Transform File Uploads
      • Workspaces
    • DoltLab
    • Dolt Workbench
    • DoltgreSQL
Powered by GitBook
On this page
  • Setup
  • Identifying the Bug
  • Fixing the Bug
  • Testing
  • Submit Changes

Was this helpful?

Edit on GitHub
Export as PDF
  1. Guides
  2. Contributing

dolt

PreviousContributingNextgo-mysql-server

Last updated 3 months ago

Was this helpful?

Here, we will go through a fix for this to demonstrate how to make a change to .

Setup

Follow these .

Identifying the Bug

Dolt has the ability to host a SQL server and take in user queries to create, modify, and drop tables. The server allows users to provide a config file through the --config option. A customer claimed that the read_only flag in their config file was not being respected when using the dolt server; this means that the server was allowing users to run queries that modified data on the server.

First off, let's try to reproduce the issue. Create a dolt database:

$ cd ~/dolt_workspace
$ mkdir test_db
$ cd test_db
$ dolt init

Create a config file named config.yml and put this in it:

log_level: debug
listener:
  host: "0.0.0.0"
  port: $PORT
behavior:
  read_only: true

We have a file called pytest.py located in ~/dolt_workspace/dolt/integration-tests/bats/helper, which contains many useful helper methods for testing dolt sql-server. I wrote a quick Python script using methods from pytest.py to send queries and look at the results manually.

from pytest import *
# Create a new connection
dc = DoltConnection(port=3000, database="test_db", user="root", auto_commit=1)
dc.connect()
try:
    actual_rows, num_rows = dc.query("create table t(a int)", False)
    print("no problems")
except BaseException as e:
    print('caught exception:', str(e))

In one terminal, start the dolt sql-server:

$ dolt sql-server --host 0.0.0.0 --port=3000 --config ./config.yml

In another, run the python script.

$ python sqltest.py
no problems

In this case "no problems" is actually bad, since we expected the server to return an error.

Fixing the Bug

After poking around in the code, we see that the config file containing user permissions is used to create a new sqlengine.

serverConf.Address = hostPort
serverConf.Auth = userAuth
serverConf.ConnReadTimeout = readTimeout
serverConf.ConnWriteTimeout = writeTimeout
serverConf.MaxConnections = serverConfig.MaxConnections()
serverConf.TLSConfig = tlsConfig
serverConf.RequireSecureTransport = serverConfig.RequireSecureTransport()
sqlEngine, err := engine.NewSqlEngine(ctx, mrEnv, engine.FormatTabular, "", serverConfig.AutoCommit())

For some reason, the NewSqlEngine constructor creates a new authenticator using auth.None, which always gives users full permissions. Instead, we should be passing in the authenticator already created that is based on permissions specified in the config file.

// NewSqlEngine returns a SqlEngine
func NewSqlEngine(
	ctx context.Context,
	mrEnv *env.MultiRepoEnv,
	format PrintResultFormat,
	initialDb string,
+	au auth.Auth,
	autocommit bool) (*SqlEngine, error) {
-	au := new(auth.None)

The method call to NewSqlEngine now looks like this:

sqlEngine, err := engine.NewSqlEngine(ctx, mrEnv, engine.FormatTabular, "", serverConf.Auth, serverConfig.AutoCommit())

Testing

$ python sqltest.py
caught exception: 1105 (HY000): not authorized: user does not have permission: write
$ npm install -g bats

This test basically creates a config file (with the read-only flag set to true), starts a dolt sql-server using the config file, sends a query to create a table, and checks to see if that table was created. So, it's an automated way to do everything we did earlier.

@test "sql-server: read-only flag prevents modification" {
    skiponwindows "Has dependencies that are missing on the Jenkins Windows installation."
    cd repo1
    DEFAULT_DB="$1"
    let PORT="$$ % (65536-1024) + 1024"
    cat >config.yml <<EOF
log_level: debug
listener:
  host: "0.0.0.0"
  port: $PORT
behavior:
  read_only: true
EOF
    dolt sql-server --host 0.0.0.0 --port=$PORT --config ./config.yml &
    SERVER_PID=$!
    wait_for_connection $PORT 5000
    # No tables at the start
    run dolt ls
    [ "$status" -eq 0 ]
    [[ "$output" =~ "No tables in working set" ]] || false
    # attempt to create table (autocommit on), expect either some exception
    server_query repo1 1 "CREATE TABLE i_should_not_exist (
            c0 INT
        )" "" "not authorized: user does not have permission: write"
    # Expect that there are still no tables
    run dolt ls
    [ "$status" -eq 0 ]
    [[ "$output" =~ "No tables in working set" ]] || false
}

Then, I ran the test using this shell script placed in the ~/dolt_workspace/dolt directory:

#!/bin/bash
cd go/cmd/dolt && go install . && cd -
cd go/cmd/git-dolt && go install . && cd -
cd go/cmd/git-dolt-smudge && go install . && cd -
cd go/store/cmd/noms && go install . && cd -
cd integration-tests/bats && bats sql-server.bats && cd -

As expected, this test passes.

$ test.sh
...
 ✓ sql-server: read-only flag prevents modification
...

Submit Changes

Use git to commit and push our changes.

I highly recommend using an IDE like especially when debugging larger projects.

After dolt and running the python script, we get an exception as expected.

A better way to test this is to use a tests, which are located in . You can install bats through npm

If you are unfamiliar with how to create a Pull Request, follow the instructions .

bug where Dolt wasn't respecting a config flag
Dolt
setup instructions
Goland
bats
~/dolt_workspace/dolt/integration-tests/bats
reinstalling
here