Install DoltLab
The latest version of DoltLab is v2.0.8
and to get started running your own DoltLab instance, you can follow the steps below. To see release notes for DoltLab's releases or to report and track DoltLab issues, visit DoltLab's issues repository.
Please note, that to upgrading to a newer version of DoltLab will require you to kill the older version of DoltLab and install the newer one, which may result in data loss.
DoltLab versions lower than v2.0.0
allowed users to use certain enterprise features for free. However, DoltLab >= v2.0.0
has removed these features for the free version of DoltLab, as they are now exclusive to DoltLab Enterprise customers.
For more information about which features have been removed for free from DoltLab v2.0.0
, please see Administrator Guide.
If you'd like more information about DoltLab Enterprise, please reach out to us on Discord, or email Brain Fitzgerald at brianf@dolthub.com.
Recommended Minimum Hardware
DoltLab is currently available for Linux and we recommend the following minimum hardware for running your own DoltLab instance:
4 CPU and 16 GB of memory
100 GBs of disk (DoltLab's container images alone require about 4 GBs of disk). Depending on your use case, this may not be enough to back all database data, and user uploaded files.
Ubuntu 18.04/20.04 Operating System
Host IP must be discoverable by the Dolt CLI and web browser.
Host should allow egress
TCP
connections.The following
TCP
ports must be open on the host:22
, forssh
connections.80
, for ingressHTTP
connections, or443
for ingressHTTPS
connections (supported in DoltLab >=v1.0.6
).100
, for ingress connections to DoltLab's remote data file server.50051
, for ingress connections to DoltLab's remote API.4321
, for ingress connections to DoltLab's file upload service API.
Starting with DoltLab
v0.7.0
, DoltLab uses more variable disk and memory on its host in order to run DoltLab Jobs. As a result, the minimum memory and disk requirements listed above might be too low for your use case. Please read the section on DoltLab Jobs in the Administrator Guide to find out more about how Jobs can impact your DoltLab instance.
Step 1: Install DoltLab's Dependencies on the Host
Once you've provisioned a Linux host and properly configured it's networking interface, you can now install DoltLab's dependencies.
If your host is running Ubuntu 18.04/20.04, the quickest way to install these dependencies is with this ubuntu-bootstrap.sh script. CentOS 7 users can use the centos-bootstrap.sh script. These scripts will also download and unzip DoltLab at the specified DOLTLAB_VERSION
to a local doltlab
directory.
To use them:
Otherwise, install the following dependencies on your host:
curl unzip docker docker-compose amazon-ecr-credential-helper
Once these are installed, follow the post-installation steps for Docker to ensure you can run it without sudo
, then verify with:
Step 2: Download DoltLab
Next, download and unzip DoltLab. To install the latest version of DoltLab run:
To install a specific version, run:
Inside the unzipped doltlab
directory, you'll find the following items:
templates
envoy.tmpl
envoy-tls.tmpl
config_loader
gentokenenckey
send_doltlab_deployed_event
smtp_connection_helper
get_machine_id
gen_saml_key
gen_saml_cert
gent_saml_certs.sh
migrate_postgres_dolt.sh
dolt_db_cli.sh
shell-db.sh
docker-compose.yaml
docker-compose-tls.yaml
start-doltlab.sh
prometheus.yaml
prometheus-alert.rules
alertmanager.yaml
templates
contains email templates used by doltlabapi
to send automated emails to users of your DoltLab instance. You can customize emails by editing these files before starting your DoltLab instance. For more information on the contents of these files and how to change them, see the Customize automated emails section of the Administrator guide.
envoy.tmpl
is an template file used to create the Envoy proxy configuration file called envoy.yaml
.
envoy-tls.tmpl
is included in DoltLab >= v1.0.6
and is used to create an envoy.yaml
file that uses TLS.
gentokenenckey
, short for "generate token encryption key" is a binary used to generate token encryption keys used by DoltLab. The code is available here.
config_loader
is a binary used to process the admin-config.yaml
file, if one exists.
send_doltlab_deployed_event
is a binary that sends a single request to our metrics server, letting us track how many DoltLab instances get deployed each day. This information helps us properly fund and staff our DoltLab team. The source for this binary is here.
smtp_connection_helper
is a binary used to help troubleshoot any issues your DoltLab instance might have when establishing a connection to your existing SMTP server. This tool uses similar code to DoltLab's email service and should successfully send a test email if the connection to the SMTP server was configured correctly. The source code for the tool is available here and basic instructions for using the tool are here.
get_machine_id
is a binary used to determine the hardware ID of the DoltLab host, used only for DoltLab Enterprise.
gen_saml_key
is a binary used to generate a private key for configuring SAML single-sign-on, used only for DoltLab Enterprise.
gen_saml_cert
is a binary used to generate a signing certificate for SAML single-sign-on, used only for DoltLab Enterprise.
gen_saml_certs.sh
is a script that uses gen_saml_key
and gen_saml_cert
to create a signing certificate for SAML single-sign-on, used only for DoltLab Enterprise. The script requires a single argument, the common name
to use for the certificate.
migrate_postgres_dolt.sh
is a script available in DoltLab v1.0.0
+. Prior to DoltLab v1.0.0
, DoltLab used PostgreSQL as its database. But now, starting with v1.0.0
, DoltLab uses Dolt as its database. This script is used when upgrading from an older DoltLab instance to DoltLab v1.0.0
and will copy the data from the existing, older DoltLab instance into the new Dolt database backing DoltLab v1.0.0
. Please see the guide for using this script to copy existing data during upgrade.
dolt_db_cli.sh
is a script available in DoltLab v1.0.0
+, useful for restoring DoltLab's Dolt server from a backup.
shell-db.sh
is a script that will open a shell to your running DoltLab's database server. For DoltLab versions lower than v1.0.0
, this database server is PostgreSQL. To use this script with those versions, supply the POSTGRES_PASSWORD
value you set when starting DoltLab as the environment variable PGPASSWORD
. A successful connection will display a dolthubapi=#
prompt.
For DoltLab v1.0.0
and later, Dolt is the database server. To connect to it, supply the DOLT_PASSWORD
environment variable value you set when starting DoltLab. A successful connection will display a mysql>
prompt.
docker-compose.yaml
is a complete Docker Compose configuration file that will spin up all the services required to run DoltLab.
docker-compose-tls.yaml
is included in DoltLab >= v1.0.6
and will spin up DoltLab using TLS.
start-doltlab.sh
is a helper script designed to quickly and easily start DoltLab. See the following section for more information about how to use this script.
prometheus.yaml
is a Prometheus configuration file that can be used for observing real-time DoltLab service metrics. Used for DoltLab Enterprise automated backups.
prometheus-alert.rules
is an alert rules file used for sending emails to DoltLab admins in the event of a backup failure. Used for DoltLab Enterprise automated backups.
alertmanager.yaml
is a AlertManager configuration file for sending email alerts to DoltLab admins based on Prometheus metrics. Used for DoltLab Enterprise automated backups.
Step 3: Start DoltLab
The recommended way to run DoltLab is with the start-doltlab.sh
script included in DoltLab's zip folder. This script requires the following environment variables to be set in your DoltLab host environment/shell.
Note, the
./start-doltlab.sh
script contains references to some of these environment variables, but not all, as some are referenced elsewhere.
For DoltLab version <=
v0.8.4
includeexport POSTGRES_USER="dolthubapi"
and renameDOLT_PASSWORD
toPOSTGRES_PASSWORD
.
HOST_IP
should be the IP address or DNS name of the Linux host running DoltLab.
DOLT_PASSWORD
and DOLTHUBAPI_PASSWORD
may be set to any valid Dolt password.
EMAIL_USERNAME
should be a valid username authorized to use existing SMTP server.
EMAIL_PASSWORD
should be the password for the aforementioned username of the SMTP server.
EMAIL_PORT
a STARTTLS
port to the existing SMTP server is assumed by default. To use an implicit TLS port, please follow these steps.
EMAIL_HOST
should be the host of the existing SMTP server.
NO_REPLY_EMAIL
should be the email address that receives no-reply messages.
DOLTLAB_ENTERPRISE_ONLINE_PRODUCT_CODE
, provided by us to DoltLab Enterprise customers. DOLTLAB_ENTERPRISE_ONLINE_SHARED_KEY
, provided by us to DoltLab Enterprise customers. DOLTLAB_ENTERPRISE_ONLINE_API_KEY
, provided by us to DoltLab Enterprise customers. DOLTLAB_ENTERPRISE_ONLINE_LICENSE_KEY
, provided by us to DoltLab Enterprise customers. DOLTLAB_ENTERPRISE_HARDWARE_ID
, use ./get_machine_id
binary to get the hardware ID and use the output as this value (DoltLab Enterprise only). TLS_CERT_CHAIN
required if running DoltLab >= v1.0.6
with TLS, should be the the absolute path to a TLS certificate chain.
TLS_PRIVATE_KEY
required if running DoltLab >= v1.0.6
with TLS, should be the the absolute path to a TLS private key.
To use DoltLab with TLS ensure the certificate is for the HOST_IP
of your DoltLab host. We recommend creating a TLS certificate with certbot.
Supported SMTP Authentication methods
Starting in DoltLab v0.3.1
, admins can use different SMTP authentication protocols to connect to an existing SMTP server. By default, ./start-doltlab.sh
sets the environment variable EMAIL_AUTH_METHOD
to plain
.
Supported EMAIL_AUTH_METHOD
options are plain
, login
, anonymous
, external
, oauthbearer
, or disable
.
plain
requires the environment variables EMAIL_USERNAME
and EMAIL_PASSWORD
to be set and uses the optional environment variable EMAIL_IDENTITY
. login
requires the environment variables EMAIL_USERNAME
and EMAIL_PASSWORD
to be set. This is used by Microsoft 365 (smtp.office365.com
). anonymous
uses the optional environment variable EMAIL_TRACE
. external
uses the optional environment variable EMAIL_IDENTITY
. oauthbearer
requires the environment variables EMAIL_USERNAME
and EMAIL_OAUTH_TOKEN
to be set. disable
will result in an unauthenticated SMTP server connection.
If you are experiencing any SMTP server connection issues (or DoltLab account creation issues) please see the SMTP troubleshooting guide.
Default user `admin`
Starting with DoltLab v0.4.1
, the default user admin
is created, when DoltLab's API server starts.
This default user allows DoltLab admins to immediately sign in to DoltLab and begin using the product, even if their DoltLab instance is not successfully connected to an SMTP server.
By default, the ./start-doltlab.sh
script will create a default user DEFAULT_USER=admin
with password DEFAULT_USER_PASSWORD=DoltLab1234
and the email address DEFAULT_USER_EMAIL=$NO_REPLY_EMAIL
, which gets its value from the supplied NO_REPLY_EMAIL
environment variable. To overwrite these default values, simply change the values of their corresponding environment variables.
Once these variables are set, simply run the start-doltlab.sh
script:
The running DoltLab processes can be viewed with docker ps
:
And navigating to http://${HOST_IP}:80
in a web browser should show the DoltLab homepage.
To run DoltLab with TLS instead run:
And navigating to https://${HOST_IP}:443
in a web browser should show the DoltLab homepage.
Last updated