- Data is migrated to the target in phases.
- This approach does not utilize continuous replication.
- Rollback is manual. If you wish to roll back before the target has received any writes that are not present on the source database, nothing needs to be done. If you wish to roll back after the target has received writes that are not present on the source database, you must manually replicate these new rows on the source.
Example scenario
You have a moderately-sized (500GB) database that provides the data store for a web application. You want to migrate the entirety of this database to a new CockroachDB cluster. You will divide this migration into four geographic regions (A, B, C, and D). You schedule a maintenance window for each region over four subsequent evenings, and you announce them to your users (per region) several weeks in advance. The application runs on a Kubernetes cluster with an NGINX Ingress Controller. Estimated system downtime: 4 hours per region.Before the migration
- Install the MOLT (Migrate Off Legacy Technology) tools.
- Review the MOLT Fetch documentation.
- Develop a migration plan and prepare for the migration.
- Recommended: Perform a dry run of this full set of instructions in a development environment that closely resembles your production environment. This can help you get a realistic sense of the time and complexity it requires.
- Announce the maintenance window to your users on a per-region basis.
- Understand the prerequisites and limitations of the MOLT tools:
Limitations
MOLT Fetch limitations
- Only tables with primary key types of
INT,FLOAT, orUUIDcan be sharded with--export-concurrency. GEOMETRYandGEOGRAPHYtypes are not supported.
MOLT Verify limitations
-
MOLT Verify compares 20,000 rows at a time by default, and row values can change between batches, potentially resulting in temporary inconsistencies in data. To configure the row batch size, use the
--row_batch_sizeflag. -
MOLT Verify checks for collation mismatches on primary key columns. This may cause validation to fail when a
STRINGis used as a primary key and the source and target databases are using different collations. - MOLT Verify might give an error in case of schema changes on either the source or target database.
- Geospatial types cannot yet be compared.
-
MOLT Verify only supports comparing one MySQL database to a whole CockroachDB schema (which is assumed to be
public).
Step 1: Prepare the source database
In this step, you will:Create migration user on source database
Create a dedicated migration user (for example,MIGRATION_USER) on the source database. This user is responsible for reading data from source tables during the migration. You will pass this username in the source connection string.
Step 2: Prepare the target database
In this step, you will:- Provision and run a new CockroachDB cluster.
- Define the tables on the target cluster to match those on the source.
- Create a SQL user on the target cluster with the necessary write permissions.
Provision a CockroachDB cluster
Use one of the following options to create and run a new CockroachDB cluster. This is your migration target.Option 1: Create a secure cluster locally
If you have the CockroachDB binary installed locally, you can manually deploy a multi-node, self-hosted CockroachDB cluster on your local machine. Learn how to deploy a CockroachDB cluster locally.Option 2: Create a CockroachDB Self-Hosted cluster on AWS
You can manually deploy a multi-node, self-hosted CockroachDB cluster on Amazon’s AWS EC2 platform, using AWS’s managed load-balancing service to distribute client traffic. Learn how to deploy a CockroachDB cluster on AWS.Option 3: Create a CockroachDB Cloud cluster
CockroachDB Cloud is a fully-managed service run by Cockroach Labs, which simplifies the deployment and management of CockroachDB. Sign up for a CockroachDB Cloud account and create a cluster using trial credits.Define the target tables
Convert the source table definitions into CockroachDB-compatible equivalents. CockroachDB supports the PostgreSQL wire protocol and is largely compatible with PostgreSQL syntax.-
The source and target table definitions must match. Review Type mapping to understand which source types can be mapped to CockroachDB types.
MySQL tables belong directly to the database specified in the connection string. A MySQL source table defined as
CREATE TABLE tbl (id INT PRIMARY KEY);should map to CockroachDB’s defaultpublicschema:-
MOLT Fetch can automatically define matching CockroachDB tables using the
drop-on-target-and-recreateoption. - If you define the target tables manually, review how MOLT Fetch handles type mismatches. You can use the MOLT Schema Conversion Tool to create matching table definitions.
-
MOLT Fetch can automatically define matching CockroachDB tables using the
-
Every table must have an explicit primary key.
Avoid using sequential keys. To learn more about the performance issues that can result from their use, refer to the guidance on indexing with sequential keys. If a sequential key is necessary in your CockroachDB table, you must create it manually, after using MOLT Fetch to load and replicate the data.
- Review Transformations to understand how computed columns and partitioned tables can be mapped to the target, and how target tables can be renamed.
-
By default on CockroachDB,
INTis an alias forINT8, which creates 64-bit signed integers. PostgreSQL and MySQL default to 32-bit integers. Depending on your source database or application requirements, you may need to change the integer size to4. For more information, refer to Considerations for 64-bit signed integers.
Schema Conversion Tool
The MOLT Schema Conversion Tool (SCT) converts source table definitions to CockroachDB-compatible syntax. It requires a free CockroachDB Cloud account.-
Upload a source
.sqlfile to convert the syntax and identify unimplemented features and syntax incompatibilities in the table definitions. -
Import the converted table definitions to a CockroachDB cluster:
- When migrating to CockroachDB Cloud, the Schema Conversion Tool automatically applies the converted table definitions to a new Cloud database.
- When migrating to a self-hosted CockroachDB cluster, export a converted DDL file and pipe the data definition language (DDL) directly into
cockroach sql.
String case sensitivity
Strings are case-insensitive in MySQL and case-sensitive in CockroachDB. You may need to edit your MySQL data to get the results you expect from CockroachDB. For example, you may have been doing string comparisons in MySQL that will need to be changed to work with CockroachDB. For more information about the case sensitivity of strings in MySQL, refer to Case Sensitivity in String Searches from the MySQL documentation. For more information about CockroachDB strings, refer toSTRING.
Identifier case sensitivity
Identifiers are case-sensitive in MySQL and case-insensitive in CockroachDB. When using the Schema Conversion Tool, you can either keep case sensitivity by enclosing identifiers in double quotes, or make identifiers case-insensitive by converting them to lowercase.AUTO_INCREMENT attribute
The MySQL AUTO_INCREMENT attribute, which creates sequential column values, is not supported in CockroachDB. When using the Schema Conversion Tool, columns with AUTO_INCREMENT can be converted to use sequences, UUID values with gen_random_uuid(), or unique INT8 values using unique_rowid(). Cockroach Labs does not recommend using a sequence to define a primary key column. For more information, refer to Unique ID best practices.
Changing a column type during table definition conversion will cause MOLT Verify to identify a type mismatch during data validation. This is expected behavior.
ENUM type
MySQL ENUM types are defined in table columns. On CockroachDB, ENUM is a standalone type. When using the Schema Conversion Tool, you can either deduplicate the ENUM definitions or create a separate type for each column.
TINYINT type
TINYINT data types are not supported in CockroachDB. The Schema Conversion Tool automatically converts TINYINT columns to INT2 (SMALLINT).
Geospatial types
MySQL geometry types are not converted to CockroachDB geospatial types by the Schema Conversion Tool. They should be manually converted to the corresponding types in CockroachDB.FIELD function
The MYSQL FIELD function is not supported in CockroachDB. Instead, you can use the array_position function, which returns the index of the first occurrence of element in the array.
Example usage:
NULL. So if you are using the ORDER BY clause in a statement with the array_position function, the caveat is that sort is applied even when the element is not found. As a workaround, you can use the COALESCE operator.
Drop constraints and indexes
To optimize data load performance, drop all non-PRIMARY KEY constraints and indexes on the target CockroachDB database before migrating:
FOREIGN KEYUNIQUE- Secondary indexes
CHECKDEFAULTNOT NULL(you do not need to drop this constraint when usingdrop-on-target-and-recreatefor table handling)
Do not drop
PRIMARY KEY constraints.Create the SQL user
Create a SQL user in the CockroachDB cluster that has the necessary privileges. To create a usercrdb_user in the default database (you will pass this username in the target connection string):
migration_schema schema and internal MOLT tables like _molt_fetch_exceptions in the public CockroachDB schema:
Ensure that you are connected to the target database.
drop-on-target-and-recreate will not be used), grant the following privileges on the schema:
drop-on-target-and-recreate, and the target schema has pre-existing tables that were created by a different user, you may need to change table ownership to allow the migration user to drop those tables. Make the following alteration for each table:
IMPORT INTO or COPY FROM on the target tables:
IMPORT INTO privileges
Grant SELECT, INSERT, and DROP (required because the table is taken offline during the IMPORT INTO) privileges on all tables being migrated:
EXTERNALIOIMPLICITACCESS system-level privilege:
COPY FROM privileges
Grant admin privileges to the user:
Set session variable
Ensure that thestatement_timeout session variable is set to 0s for the user:
Migrate each phase
Steps 3-7 are run for each phase of the data migration. Within the first migration downtime window, you will run through these steps for Region A. You will repeat these steps for the other regions during each subsequent downtime window.Step 3: Stop application traffic
With both the source and target databases prepared for the data load, it’s time to stop application traffic to the source for a particular region. If the Kubernetes cluster that deploys the application has pre-region deployments (for example,app-us, app-eu, app-apac), you can scale down only the deployment for that region.
Application downtime begins now, for users in the given region.It is strongly recommended that you perform a dry run of this migration in a test environment. This will allow you to practice using the MOLT tools in real time, and it will give you an accurate sense of how long application downtime might last.
Step 4: Load data into CockroachDB
In this step, you will:- Configure MOLT Fetch with the flags needed for your migration.
- Run MOLT Fetch.
- Understand how to continue a load after an interruption.
Configure MOLT Fetch
The MOLT Fetch documentation includes detailed information about how to configure MOLT Fetch, and how to monitor MOLT Fetch metrics. When you runmolt fetch, you can configure the following options for data load:
- Specify source and target databases: Specify URL‑encoded source and target connections.
- Select data to migrate: Specify schema and table names to migrate. Important for a phased migration.
- Define intermediate file storage: Export data to cloud storage or a local file server.
- Define fetch mode: Specifies whether data will only be loaded into/from intermediate storage.
- Shard tables: Divide larger tables into multiple shards during data export.
- Data load mode: Choose between
IMPORT INTOandCOPY FROM. - Table handling mode: Determine how existing target tables are initialized before load.
- Define data transformations: Define any row-level transformations to apply to the data before it reaches the target.
- Monitor fetch metrics: Configure metrics collection during initial data load.
molt fetch command and its flags. Follow best practices, especially those related to security.
At minimum, the molt fetch command should include the source, target, data path, and --ignore-replication-check flags. For a phased migration, you may also choose to include --schema-filter or --table-filter flags:
Run MOLT Fetch
Perform the bulk load of the source data.-
Run the MOLT Fetch command to move the source data into CockroachDB. This example command passes the source and target connection strings as environment variables, writes intermediate files to S3 storage, and uses the
truncate-if-existstable handling mode to truncate the target tables before loading data. It limits the migration to a single schema and filters for three specific tables. The data load mode defaults toIMPORT INTO. Include the--ignore-replication-checkflag to skip replication checkpoint queries, which eliminates the need to configure the source database for logical replication. -
Check the output to observe
fetchprogress. Astarting fetchmessage indicates that the task has started:data extractionmessages are written for each table that is exported to the location in--bucket-path:data importmessages are written for each table that is loaded into CockroachDB:Afetch completemessage is written when the fetch task succeeds:
Continue MOLT Fetch after an interruption
If MOLT Fetch fails while loading data into CockroachDB from intermediate files, it exits with an error message, fetch ID, and continuation token for each table that failed to load on the target database.- All data has been exported from the source database into intermediate files on cloud or local storage.
- The initial load of source data into the target CockroachDB database is incomplete.
- The load uses
IMPORT INTOrather thanCOPY FROM.
Only one fetch ID and set of continuation tokens, each token corresponding to a table, are active at any time. See List active continuation tokens.
Step 5: Verify the data
In this step, you will use MOLT Verify to confirm that the source and target data is consistent. This ensures that the data load was successful. Use MOLT Verify’s--schema-filter or --table-filter to select only the tables that are relevant for the given phase.
Run MOLT Verify
-
Run the MOLT Verify command, specifying the source and target connection strings and the tables to validate.
-
Check the output to observe
verifyprogress. Averification in progressindicates that the task has started:starting verifymessages are written for each specified table:Afinished row verificationmessage is written after each table is compared. Ifnum_successequalsnum_truth_rowsand the error counters (num_missing,num_mismatch,num_extraneous, andnum_column_mismatch) are all0, the table verified successfully. Any non-zero values in the error counters indicate data discrepancies that need investigation. For details on each field, refer to the MOLT Verify page.Averification completemessage is written when the verify task succeeds:
Step 6: Finalize the target schema
Add constraints and indexes
Add any constraints or indexes that you previously removed from the CockroachDB schema to facilitate data load.If you used the
--table-handling drop-on-target-and-recreate option for data load, only PRIMARY KEY and NOT NULL constraints are preserved. You must manually recreate all other constraints and indexes.ALTER TABLE ... ADD CONSTRAINT and CREATE INDEX. Review the best practices for creating secondary indexes on CockroachDB.

