cockroach start command, which you use to start a new multi-node cluster or add nodes to an existing cluster.
Node-level settings are defined by flags passed to the
cockroach start command and cannot be changed without stopping and restarting the node. In contrast, some cluster-wide settings are defined via SQL statements and can be updated anytime after a cluster has been started. For more details, see Cluster Settings.Synopsis
Start a node to be part of a new multi-node cluster:Flags
Thecockroach start command supports the following general-use, networking, security, and logging flags.
Many flags have useful defaults that can be overridden by specifying the flags explicitly. If you specify flags explicitly, however, be sure to do so each time the node is restarted, as they will not be remembered. The one exception is the --join flag, which is stored in a node’s data directory. We still recommend specifying the --join flag every time, as this will allow nodes to rejoin the cluster even if their data directory was destroyed.
General
Networking
Security
Locality
The--locality flag accepts arbitrary key-value pairs that describe the location of the node. Locality should include a region key-value if you are using CockroachDB’s Multi-region SQL capabilities.
Depending on your deployment you can also specify country, availability zone, etc. The key-value pairs should be ordered into locality tiers from most inclusive to least inclusive (e.g., region before availability zone as in region=eu-west-1,zone=eu-west-1a), and the keys and order of key-value pairs must be the same on all nodes. It’s typically better to include more pairs than fewer.
-
Specifying a region with a
regiontier is required in order to enable CockroachDB’s multi-region capabilities. - CockroachDB spreads the replicas of each piece of data across as diverse a set of localities as possible, with the order determining the priority. Locality can also be used to influence the location of data replicas in various ways using high-level multi-region SQL capabilities or low-level replication zones.
- When there is high latency between nodes (e.g., cross-availability zone deployments), CockroachDB uses locality to move range leases closer to the current workload, reducing network round trips and improving read performance, also known as “follow-the-workload”. In a deployment across more than 3 availability zones, however, to ensure that all data benefits from “follow-the-workload”, you must increase your replication factor to match the total number of availability zones.
- Locality is also a prerequisite for using the Multi-region SQL abstractions, table partitioning, and Node Map Enterprise features.
Example
The following shell commands use the--locality flag to start 9 nodes to run across 3 regions: us-east-1, us-west-1, and europe-west-1. Each region’s nodes are further spread across different availability zones within that region.
This example follows the conventions required to use CockroachDB’s multi-region capabilities.
us-east-1:
us-west-1:
europe-west-1:
Storage
Storage engine
The--storage-engine flag is used to choose the storage engine used by the node. Note that this setting applies to all stores on the node, including the temp store.
As of v21.1 and later, CockroachDB always uses the Pebble storage engine. As such, pebble is the default and only option for the --storage-engine flag.
Store
The--store flag allows you to specify details about a node’s storage.
To start a node with multiple disks or SSDs, you can use either of these approaches:
- Configure the disks or SSDs as a single RAID volume, then pass the RAID volume to the
--storeflag when starting thecockroachprocess on the node. - Provide a separate
--storeflag for each disk when starting thecockroachprocess on the node. For more details about stores, see Start a Node.
If you start a node with multiple
--store flags, it is not possible to scale back down to only using a single store on the node. Instead, you must decommission the node and start a new node with the updated --store.--store flag supports the following fields. Note that commas are used to separate fields, and so are forbidden in all field values.
In-memory storage is not suitable for production deployments at this time.
Logging
By default,cockroach start writes all messages to log files, and prints nothing to stderr. This includes events with INFO severity and higher. However, you can customize the logging behavior of this command by using the --log flag:
Defaults
See the default logging configuration.Standard output
When you runcockroach start, some helpful details are printed to the standard output:
Examples
Start a multi-node cluster
To start a multi-node cluster, run thecockroach start command for each node, setting the --join flag to the addresses of the initial nodes.
For a cluster in a single region, set 3-5 --join addresses. Each starting node will attempt to contact one of the join hosts. In case a join host cannot be reached, the node will try another address on the list until it can join the gossip network.
When starting a multi-region cluster, set more than one --join address per region, and select nodes that are spread across failure domains. This ensures high availability.
- Secure
- Insecure
Start a multi-region cluster
In this example we will start a multi-node local cluster with a multi-region setup that uses the same regions (passed to the--locality flag) as the multi-region MovR demo application.
-
Start a node in the
us-east1region: -
Start a node in the
us-west1region: -
Start a node in the
europe-west1region: -
Initialize the cluster:
-
Connect to the cluster using
cockroach sql: -
Issue the
SHOW REGIONSstatement to verify that the list of regions is expected:
For more information about the
--locality flag, see Locality.Start a multi-node cluster across private networks
Scenario:- You have a cluster that spans GCE and AWS.
- The nodes on each cloud can reach each other on public addresses, but the private addresses aren’t reachable from the other cloud.
-
Start each node on GCE with
--localityset to describe its location,--locality-advertise-addrset to advertise its private address to other nodes in on GCE,--advertise-addrset to advertise its public address to nodes on AWS, and--joinset to the public addresses of 3-5 of the initial nodes: -
Start each node on AWS with
--localityset to describe its location,--locality-advertise-addrset to advertise its private address to other nodes on AWS,--advertise-addrset to advertise its public address to nodes on GCE, and--joinset to the public addresses of 3-5 of the initial nodes: -
Run the
cockroach initcommand against any node to perform a one-time cluster initialization:
Add a node to a cluster
To add a node to an existing cluster, run thecockroach start command, setting the --join flag to the same addresses you used when starting the cluster:
- Secure
- Insecure
Create a table with node locality information
Start a three-node cluster with locality information specified in thecockroach start commands:
crdb_internal.locality_value built-in function to return the current node’s locality information from inside a SQL shell. The example below uses the output of crdb_internal.locality_value('zone') as the DEFAULT value to use for the zone column of new rows. Other available locality keys for the running three-node cluster include region and cloud.
zone column has the zone of the node on which the row was created.
In a separate terminal window, open a SQL shell to a different node on the cluster:
Start a cluster with separate RPC and SQL networks
Separating the network addresses used for intra-cluster RPC traffic and application SQL connections can provide an additional level of protection against security issues as a form of defense in depth. This separation is accomplished with a combination of the--sql-addr flag and firewall rules or other network-level access control (which must be maintained outside of CockroachDB).
For example, suppose you want to use port 26257 for SQL connections and 26258 for intra-cluster traffic. Set up firewall rules so that the CockroachDB nodes can reach each other on port 26258, but other machines cannot. Start the CockroachDB processes as follows:
26258 (the value for listen-addr, not sql-addr) in the --join flag. Also, if your environment requires the use of the --advertise-addr flag, you should probably also use the --advertise-sql-addr flag when using a separate SQL address.
Clusters using this configuration with client certificate authentication may also wish to use split client CA certificates.

