MOLT Releases

​

November 10, 2026

Breaking changes

• MySQL source connection strings now use MySQL’s TLS parameter names. In mysql:// source URLs, rename sslmode to ssl-mode, sslrootcert to ssl-ca, sslcert to ssl-cert, and sslkey to ssl-key. For details, refer to Connection security.
• MOLT log files are now readable only by the user who runs MOLT (permissions 0600), and log directories that MOLT creates are 0700. Previously, log files were readable by all users. If another user or process reads MOLT log files, such as a log shipping agent running under its own account, run it as the same user as MOLT or as root, or pass --log-file stdout and capture the output.
• MOLT Fetch now fails when the tables selected from a PostgreSQL source include a parent table that uses table inheritance (CREATE TABLE ... INHERITS), and the error names each parent and child table. Previously, MOLT Fetch reported success but loaded each child row twice: into the child table and again into the parent table. There is no override flag, so migrate inheritance hierarchies manually. Refer to Known limitations.
• When --bucket-path points to Amazon S3, MOLT Fetch no longer gets credentials from the AWS EC2 instance metadata service (IMDS) by default. This prevents multi-minute hangs on hosts outside AWS. If you run MOLT Fetch on EC2 and use the instance profile for credentials, set AWS_EC2_METADATA_DISABLED=false. Other credential sources, such as environment variables and the shared credentials file, still work. Refer to Cloud storage security.
• For Oracle sources, when --oracle-application-users is not set, MOLT Fetch now finds the application users automatically (users with ORACLE_MAINTAINED='N') and checks only their active transactions when it chooses the replication start point (SCN). Previously, it checked the transactions of all users. On Oracle 11g, or if the MOLT user cannot query ALL_USERS, MOLT Fetch now fails unless you set --oracle-application-users. The flag still overrides the automatic list.
• MOLT now rejects an Oracle --source connection string whose charset parameter is not UTF-8 or AL32UTF8. Previously, other values could corrupt character data or be ignored. MOLT Fetch also logs a warning when the Oracle source database character set is not Unicode.
• MOLT Verify now records telemetry, as MOLT Fetch already does. It writes run summary events to the MOLT log, and creates and writes to a _molt_telemetry_runs table on the target database. Previously, MOLT Verify did not create or write tables on the target. Telemetry is on by default. To turn it off, pass --opt-out-telemetry. If MOLT Verify cannot write the table, it logs a warning and continues. MOLT does not send this telemetry over the network.

Enhancements

MOLT Fetch

• Added the --require-empty-target flag. If set, then before it moves any data, MOLT Fetch checks that every target table in scope is empty, and stops if any table has rows. This flag cannot be combined with --table-handling truncate-if-exists, --table-handling drop-on-target-and-recreate, --fetch-id, --continuation-token, or --mode export-only.
• Added the --dropped-constraints-file and --dropped-indexes-file flags. With --table-handling drop-on-target-and-recreate, MOLT Fetch writes the constraints and secondary indexes that it dropped to SQL files, so that you can recreate them after the data load. The files contain the source definitions, adjusted to CockroachDB syntax where needed (for example, LIKE in place of the PostgreSQL ~~ operator). Review the files before you run them.
• Added the --flatten-partitioned-tables flag for PostgreSQL sources. MOLT Fetch loads each declaratively partitioned table into one unpartitioned target table. With this flag, --table-filter matches only the partitioned table’s name, not the names of its partitions.
• On PostgreSQL 13 and later, you can use --flatten-partitioned-tables with logical replication setup (--pglogical-replication-slot-name). With --pglogical-publication-and-slot-drop-and-recreate, MOLT Fetch creates the publication with publish_via_partition_root enabled. Otherwise, the existing publication must have publish_via_partition_root enabled and must include each partitioned table.
• Added the --fail-on-no-tables flag. MOLT Fetch exits with an error when no tables are left to migrate, for example because the filters match no tables or the tables are missing on the target.
• --assume-role now accepts CockroachDB ASSUME_ROLE syntax for Amazon S3, including an external ID (<role>;external_id=<id>) and comma-separated role chains. The new --import-assume-role flag sets a different role chain for IMPORT INTO. It works only with s3:// paths and requires --use-implicit-auth.
• MOLT Fetch now logs the progress of each IMPORT INTO job during the data load, including the job ID, status, and percent complete.
• After the data load, MOLT Fetch now advances each target sequence past the largest value it loaded, so the first insert after the migration no longer conflicts with a migrated row. To turn this off, pass --skip-sequence-sync.
• With --table-handling drop-on-target-and-recreate, MOLT Fetch now keeps explicit per-column collations when it creates target tables from a PostgreSQL source. For example, name text COLLATE "en_US.utf8" becomes name STRING COLLATE en_US. Previously, MOLT Fetch silently dropped these collations, which could change sort order, string comparison, and uniqueness on the target. Collations that CockroachDB cannot represent, such as custom collations, are dropped with a warning.
• When MOLT Fetch creates target tables, its column type warnings now name the column as table.column.
• For MySQL sources, MOLT Fetch now distinguishes between GTID being disabled and GTID being enabled with no transactions committed yet, and explains how to fix each case.

MOLT Verify

• Added the --fail-on-mismatch flag. MOLT Verify exits with a non-zero code when it finds missing or extra tables, table definition mismatches, or missing, mismatched, or extra rows. Tables that cannot be compared row by row (no primary key, or a primary key collation mismatch) are reported as warnings and do not cause a failure. With or without this flag, MOLT Verify now also exits with a non-zero code if it cannot finish comparing a table, for example because of a statement timeout. Previously, it logged the error and exited with code 0, leaving rows unchecked.
• MOLT Verify no longer reports false missing and extra rows when string primary keys use collations that sort differently on the source and on CockroachDB. It now logs a warning that the collation prevents verification, and skips row verification for that table.
• MOLT Verify no longer reports CockroachDB’s hidden rowid column (added to tables without a primary key) or MOLT’s own tables (_molt_fetch_status, _molt_fetch_exceptions, and _molt_telemetry_runs) as mismatches.
• Column exclusions in a --transformations-file now work correctly in MOLT Verify. An excluded column that exists on both the source and the target is no longer reported as a mismatch, and exclusions also apply to columns that exist only on the target. In MOLT Fetch and MOLT Verify, anchored patterns (^ and $) now match quoted identifiers, such as mixed-case column names. If an anchored pattern now matches a primary key column, MOLT Fetch and MOLT Verify stop with an error, because primary key columns cannot be excluded.

General

• MOLT Fetch and MOLT Verify now read connection strings from the MOLT_SOURCE, MOLT_TARGET, and MOLT_SOURCE_CDB environment variables, which correspond to --source, --target, and --source-cdb. A flag takes precedence over its variable. Use the variables to keep credentials out of the process list. Refer to Connection security.
• MOLT now prints a HINT: line on stderr for each remediation hint attached to an error, such as the ALTER PUBLICATION statements that MOLT Fetch suggests for a partitioned PostgreSQL source.

Security

• Connection string parse errors no longer print the connection string.
• MOLT Fetch and MOLT Verify now log a warning when a PostgreSQL or CockroachDB connection with sslmode=prefer (the default when sslmode is not set) falls back to an unencrypted connection because the server refused TLS.
• MOLT log files and directories now have restricted permissions. For details, refer to Breaking changes.

Oracle sources

• Added the --oracle-tablespace-filter flag to MOLT Fetch and MOLT Verify. It limits the run to Oracle tables in tablespaces that match a POSIX regular expression.
• Before it exports data, MOLT Fetch now checks the Oracle source’s redo and archive log settings. It stops on settings that would break replication, such as NOARCHIVELOG mode or missing primary key supplemental logging, and logs warnings for weaker settings, such as single-member redo log groups or a short retention window. To skip the check, pass --ignore-replication-check.
• Added the --oracle-narrow-snapshot-scn-by-table-touch flag. When it is set, MOLT Fetch moves the replication start point (SCN) later by checking only the active transactions that touch the tables being migrated. The flag requires SELECT access to V$LOCK and ALL_OBJECTS.

Data types

• PostgreSQL sources: PostGIS GEOMETRY and GEOGRAPHY columns are now supported, including when PostGIS is installed in a schema other than public. Previously, these columns failed for PostgreSQL sources. Geometry values with a non-zero SRID (for example, 4326) no longer cause a panic.
• PostgreSQL sources: DOMAIN columns (migrated as their base type), money, tsvector, tsquery, and pgvector vector (migrated to VECTOR) are now supported. Built-in types that CockroachDB does not have, such as the geometric types, xml, and macaddr, now migrate as STRING instead of stopping the run, and cidr migrates as INET. Arrays of these types and composite types can migrate as STRING with a --type-map-file entry.
• MySQL sources: MOLT Fetch and MOLT Verify now migrate spatial columns (GEOMETRY, POINT, POLYGON, and others) to CockroachDB GEOMETRY.
• MOLT Fetch and MOLT Verify now support tables with a TEXT primary key on the source and a UUID primary key on the target. Previously, both skipped these tables.

Type map files

• MOLT Fetch and MOLT Verify now apply --type-map-file when they compare source and target column types, so an intended type override no longer stops the data load. MOLT Verify now accepts --type-map-file. Both reject a malformed file, and warn about entries that name a table that is not on the source.
• You can now map Oracle NUMBER columns to INT2, INT4, INT8, FLOAT4, or FLOAT8 with --type-map-file. MOLT rejects integer types for columns with a non-zero scale, and warns about possible precision loss for floating-point types.

Bug fixes

• Fixed a bug where, for a CockroachDB source, rows committed in the second before MOLT Fetch started could be left out of the initial data load.
• Fixed a bug where, with --pglogical-publication-and-slot-drop-and-recreate on a PostgreSQL source, a transaction committed while MOLT Fetch created the replication slot could be both copied and replicated, which could cause unique constraint failures during replication. The export now reads at the slot’s consistent point. MOLT Fetch now opens one additional replication connection to the source, so a connection pooler that rejects replication connections now rejects MOLT Fetch. MOLT Fetch also no longer ignores errors when it drops the previous slot, other than a slot that does not exist.
• Fixed a bug where, with a PostgreSQL 18 source and --table-handling drop-on-target-and-recreate, MOLT Fetch warned that the recreated tables were missing NOT NULL constraints.
• Fixed a bug where MOLT Fetch failed with array element OID not registered when migrating PostgreSQL enum array columns.
• Fixed a bug where Oracle tables with a RAW primary key lost rows in MOLT Fetch and showed false extra rows in MOLT Verify when the table spanned more than one batch.
• Fixed a bug where Oracle tables with a string primary key could fail with ORA-00904 in MOLT Fetch and MOLT Verify after the first batch.
• Fixed a bug where MOLT Fetch created FLOAT4 target columns for Oracle FLOAT, REAL, and DOUBLE PRECISION columns, which rounded double-precision values. When it creates target tables, MOLT Fetch now creates FLOAT8 columns, and MOLT Verify expects FLOAT8.
• Fixed a bug where MOLT Fetch dropped the time of day from Oracle DATE values, and MOLT Verify did not report the loss. Oracle DATE columns are now mapped to the CockroachDB TIMESTAMP type, and fetching into an existing DATE column fails if any value has a time of day.
• Fixed a bug where, when you resumed MOLT Fetch with --fetch-id, a transient error while reading the earlier run’s failure records could cause MOLT Fetch to skip a table that had failed.

​

July 31, 2026

Enhancements

• Stats-based sharding is now automatic for PostgreSQL sources. MOLT Fetch turns this on by default, so migrations from PostgreSQL are faster without any extra flags. Run ANALYZE on your source tables first so the statistics are current. To turn it off, pass --use-stats-based-sharding=false.
• Oracle wallets for TLS connections are now supported. Use the new --source-oracle-wallet-location flag to connect to an Oracle source secured with SSL/TLS through an Oracle wallet.

Bug fixes

• Fixed out-of-memory failures when fetching tables without a primary key. molt fetch --direct-copy --skip-pk-check no longer loads an entire table into memory before copying it. Rows are now streamed in batches, so very large tables (hundreds of millions of rows) migrate without running the host out of memory.
• Fixed errors being silently ignored while reading the source. Temporary errors while listing source tables or reading table details could previously be missed, letting molt verify and molt fetch continue with an incomplete set of tables. These errors are now reported.

​

June 26, 2026

• Fixed a bug where MOLT Fetch and MOLT Verify, run against a MySQL source with --schema-filter set to anything other than .*, would silently filter out all tables, completing with zero rows migrated and a success (exit 0) status. The schema filter is now correctly reset for MySQL sources.
• MOLT Verify now honors column_exclusion rules from a --transformations-file. Previously these rules were silently ignored, producing false missing column on target warnings for columns intentionally excluded during MOLT Fetch.
• MOLT Fetch now handles empty (0-row) tables in export-only to import-only workflows. Previously, an empty table produced no output during export, causing a subsequent import-only run to fail with a no such file or directory error. Empty tables are now marked and skipped gracefully on import (from local filesystem, S3, GCP, and Azure).
• MOLT Fetch now validates flag values at startup and reports all invalid flags at once, instead of failing one flag at a time across re-runs.
• At the end of a fetch run, MOLT Fetch now prints the exact commands to drop the PostgreSQL logical replication slot and publication that it created on the source, since replication is complete and WAL no longer needs to be retained.
• Reduced per-object and per-batch log volume at default (info) logging in MOLT Fetch and MOLT Verify for large migrations. Verbose per-table and per-batch progress is now logged at debug level.

​

May 28, 2026

• MOLT Fetch now logs structured telemetry events. Opt out with the --opt-out-telemetry flag.
• MOLT Fetch no longer counts tables that cannot be migrated (no primary key or source/target collation mismatch) toward the fetch complete summary’s num_tables and tables fields. Such tables are now reported separately in a new skipped_unmigratable_tables field.

​

April 30, 2026

• MOLT Verify now supports PostgreSQL declarative partitioned tables.

​

March 26, 2026

• Fixed a bug in MOLT Fetch v1.3.3-v1.3.6 where a transient error when scanning rows from the source database could cause retried queries to resume from an incorrect position, resulting in rows missing from the target database. Migrations from PostgreSQL and CockroachDB databases may have been exposed to this issue. Fetch retry attempts are recorded in the MOLT Fetch logs, and MOLT Verify will have detected any data discrepancies introduced by this bug.

​

February 26, 2026

• Added support for selective data verification on PostgreSQL and MySQL sources, allowing users to compare a subset of data from both source and target databases based on filter predicates.
• Fixed an issue where MOLT Fetch failed to copy data from PostgreSQL when numeric arrays contained negative values.

​

January 22, 2026

• Binary downloads now include version-compatible Grafana dashboards.

​

December 18, 2025

• Deprecated data-load-and-replication, replication-only, and failback modes in Fetch. Replication should now be performed separately with Replicator. Refer to the MOLT Replicator documentation.
• Oracle-compatible MacOS builds are now available for arm64 and amd64. Refer to Installation.
• Fixed a bug where the --filter-path flag for MOLT Fetch was hidden. --filter-path is now present for selective data movement in MOLT Fetch.
• MOLT Fetch now logs a warning when --schema-filter is used with MySQL sources, which do not support schemas.
• Fixed a character escaping bug where MOLT Verify was incorrectly reporting mismatching rows for JSON data in MySQL.
• Added better edge case handling for cleaning up the target database after IMPORT runs so that a fresh connection is opened, which mitigates against stale or already broken long-lived connections.
• Added better logging to handle the case where the recreation of a connection fails during a retry.
• Removed fixup mode, live verify mode, and continuous verify mode from MOLT Verify, including the --continuous and --continuous-pause-between-runs flags.
• Fixed a bug in retry logic to cleanly exit (without hanging) when the context is canceled and report the proper error.

​

October 24, 2025

• MOLT Fetch replication modes are deprecated in favor of a separate replication workflow using replicator. For details, refer to MOLT Replicator.
• Added retry logic to the export phase for CockroachDB and PostgreSQL sources to handle transient errors while maintaining a consistent snapshot point. Not currently supported for Oracle or MySQL sources.
• Added --export-retry-max-attempts and --export-retry-max-duration flags to control retry behavior for source query exports.

​

September 25, 2025

• MOLT Fetch replication modes will be deprecated in favor of a separate replication workflow in an upcoming release. This includes the data-load-and-replication, replication-only, and failback modes.
• Fixed a bug with stats-based sharding where PostgreSQL queries were generated incorrectly when the first primary key column did not have sufficient unique values.
• Fixed a bug in escape-password where passwords that start with a hyphen were not handled correctly. Users must now pass the --password flag when running escape-password. For example, molt escape-password --password 'a$52&'.
• Added support for assume role authentication during data export with MOLT Fetch.
• Added support to replicator for retrying unique constraint violations on the target database, which can be temporary in some cases.
• Added exponential backoff to replicator for retryable errors when applying mutations to the target database. This reduces load on the target database and prevents exhausting retries prematurely. The new replication flags --retryInitialBackoff, --retryMaxBackoff, and --retryMultiplier control backoff behavior. The new --maxRetries flag configures the maximum number of retries. To retain the previous “immediate retry” behavior, set --retryMaxBackoff 1ns --retryInitialBackoff 1ns --retryMultiplier 1.
• Added support to replicator for the source_lag_seconds, target_lag_seconds, apply_mutation_age_seconds, and source_commit_to_apply_lag_seconds metrics for replication from PostgreSQL and MySQL, and introduced histogram metrics source_lag_seconds_histogram and target_lag_seconds_histogram for replication from CockroachDB. source_lag_seconds measures the delay before data is ready to be processed by replicator, while target_lag_seconds measures the “end-to-end” delay until replicator has written data to the target. A steady increase in source_lag_seconds may indicate replicator cannot keep up with the source workload, while a steady increase in target_lag_seconds may indicate replicator cannot keep up with the source workload or that writes on the target database are bottlenecked.

​

August 21, 2025

• MOLT Fetch now supports sharding of primary keys of any data type on PostgreSQL 11+ sources. This can be enabled with the --use-stats-based-sharding flag.
• Added the --ignore-replication-check flag to allow data loads with planned downtime and no replication setup. The --pglogical-ignore-wal-check flag has been removed.
• Added the --enableParallelApplies replication flag to enable parallel application of independent table groups during replication. By default, applies are synchronous. When enabled, this increases throughput at the cost of increased target pool and memory usage.
• Improved cleanup logic for scheduled tasks to ensure progress reporting and prevent indefinite hangs.
• Added parallelism gating to ensure the parallelism setting is smaller than the targetMaxPoolSize. This helps prevent a potential indefinite hang.
• Added new metrics that track start and end times for progress reports ( core_progress_reports_started_count and core_progress_reports_ended_count ) and error reports ( core_error_reports_started_count and core_error_reports_ended_count ). These provide visibility into the core sequencer progress and help identify hangs in the applier and progress tracking pipeline.
• Added new metrics target_apply_queue_utilization_percent and target_apply_queue_depth to track target apply queue utilization, indicating when the queue should be resized. These metrics apply only to PostgreSQL, Oracle, and CockroachDB sources.
• Oracle sources now support configurable backpressure between the source and target applier. The new target_apply_queue_depth metric tracks queue depth.
• Improved visibility into queue depth between each source frontend and the backend applier to the target. Updated logging to use lower log levels with clearer, less alarming messages.
• Improved throughput for tables in the replication stream that have no dependencies on one another. This increases parallelism and minimizes blocking of transactions that are mutually independent.
• The best effort window is now disabled by default to prevent unexpected mode switches that could cause consistency issues or stall replication due to failed target applies.

Bug fixes

• Fixed a panic that could occur with ENUM types.

​

July 24, 2025

• Oracle is now supported on Fetch Docker images and standalone binaries.
• Oracle is now supported on Replicator Docker images and standalone binaries.
• Azure Blob Storage is now supported as an intermediate data store for Fetch.
• Added a --skip-pk-check flag that lets you run the initial data load even when the source or target table is missing a primary key, or the keys do not match. When this flag is set, Fetch treats every table as keyless, disables sharding, and exports each table in a single batch, ignoring export-concurrency and row-batch-size. Querying the entire table at once may lead to high memory usage or long-running queries. Duplicate source rows are automatically removed during import when the target has primary key or unique constraints.
• Replication now uses checkpoint polling by default. This is because polling (which periodically queries the staging table for updates) performs comparably to checkpoint streaming (which uses an internal changefeed from the staging table to broadcast real-time updates). To use checkpoint streaming, set --enableCheckpointStream. --disableCheckpointStream is deprecated and should be removed from replication commands.
• Replaced the --copierChannel and --stageCopierChannelSize replication flags with a single --targetApplyQueueSize flag, which controls downstream apply throughput and memory usage. This feature applies only to CockroachDB and PostgreSQL ( pglogical ) sources.
• Added support for compressed changefeed payloads, improving performance of changefeed-to-Replicator communication. This only affects failback workflows from CockroachDB v25.2 and later.
• Improved apply-side performance by switching to a faster JSON decoder.
• Improved batch-accumulation performance by removing unnecessary sorting.

Bug fixes

• Fixed a bug where some error logs were not displayed when replicating to the target database.

​

June 19, 2025

• Updated the MOLT Grafana dashboard with the following timing metrics to better diagnose performance: Source Side Queries, Convert Source Side Queries to Datums, Writing Datums to Pipe, Preparing CSV files to be uploaded, Uploading CSV files to S3/GCP/local.
• Upgraded the MOLT parser to support new syntax that is valid in CockroachDB v25.2.
• Added more granular replication counter metrics to track data counts at each stage of the mutation pipeline, helping to diagnose data correctness issues.

Bug fixes

• MOLT Fetch failback now reliably creates changefeeds with a sorted list of table names so that create changefeed operations can be properly deduplicated.
• Fixed an issue where shard connections failed to recognize custom types (e.g., ENUM ) in primary keys during table migration. This occurred because the type map from the original pgx.Conn was not cloned. The type map is now properly cloned and attached to each shard connection.
• Fixed a bug that could cause an integer overflow, which impacts retrieving the correct shards for exporting data.

​

May 22, 2025

• Fixed a bug in --direct-copy mode that occurred when --case-sensitive was set to false (default). Previously, the COPY query could use incorrect column names in some cases during data transfer, causing errors. The query now uses the correct column names.
• Fixed a bug in how origin messages were handled during replication from PostgreSQL sources. This allows replication to successfully continue.
• ENUM types can now be replicated from MySQL 8.0 sources.

​

April 25, 2025

• During data export, MOLT Fetch now treats empty STRING values on source Oracle databases as NULL values on the target database. This is because Oracle does not differentiate between empty STRING and NULL values.

​

April 7, 2025

• MOLT Fetch now supports PostgreSQL 11.
• MOLT Fetch failback to CockroachDB is now disallowed.
• MOLT Verify can now compare tables that are named differently on the source and target schemas.
• The molt logging date format is now period-delimited for Windows compatibility.
• During replication, an index is now created on all tables by default, improving replication performance. Because index creation can cause the replication process to initialize more slowly, this behavior can be disabled using the --stageDisableCreateTableReaderIndex replication flag.
• Added a failback metric that tracks the time to write a source commit to the staging schema for a given mutation.
• Added a failback metric that tracks the time to write a source commit to the target database for a given mutation.

​

February 26, 2025

• MOLT Fetch users can now set --table-concurrency and --export-concurrency to values greater than 1 for MySQL sources.
• MOLT Fetch now supports case-insensitive comparison of table and column names by default. Previously, case-sensitive comparisons could result in no matching table on target errors. To disable case-sensitive comparisons explicitly, set --case-sensitive=false. If = is not included (e.g., --case-sensitive false ), this is interpreted as --case-sensitive (i.e., --case-sensitive=true ).

​

February 5, 2025

• Added an --import-region flag that is used to set the AWS_REGION query parameter explicitly in the s3 URL.
• Fixed the truncate-if-exists schema mode for cases where there are uppercase table or schema names.
• Fixed an issue with unsigned BIGINT values overflowing in replication.
• Added a --schemaRefresh replication flag that is used to configure the schema watcher refresh delay in the replication phase. Previously, the refresh delay was set to a constant value of 1 minute. Set the flag as follows: --replicator-flags "--schemaRefresh {value}".

​

December 13, 2024

• MOLT Fetch users now can use --assume-role to specify a service account for assume role authentication to cloud storage. --assume-role must be used with --use-implicit-auth, or the flag will be ignored.
• MySQL 5.7 and later are now supported with MOLT Fetch replication modes.
• Fetch replication mode now defaults to a less verbose INFO logging level. To specify DEBUG logging, pass in the --replicator-flags '-v' setting, or --replicator-flags '-vv' for trace logging.
• MySQL columns of type BIGINT UNSIGNED or SERIAL are now auto-mapped to DECIMAL type in CockroachDB. MySQL regular BIGINT types are mapped to INT type in CockroachDB.
• The pglogical replication workflow was modified in order to enforce safer and simpler defaults for the data-load, data-load-and-replication, and replication-only workflows for PostgreSQL sources. Fetch now ensures that the publication is created before the slot, and that replication-only defaults to using publications and slots created either in previous Fetch runs or manually.
• Fixed scan iterator query ordering for BINARY and TEXT (of same collation) PKs so that they lead to the correct queries and ordering.
• For a MySQL source in replication-only mode, the --stagingSchema replicator flag can now be used to resume streaming replication after being interrupted. Otherwise, the --defaultGTIDSet replicator flag is used to start initial replication after a previous Fetch run in data-load mode, or as an override to the current replication stream.

​

October 29, 2024

• Added failback mode to MOLT Fetch, which allows the user to replicate changes on CockroachDB back to the initial source database. Failback is supported for MySQL and PostgreSQL databases.
• The --pprof-list-addr flag, which specifies the address of the pprof endpoint, is now configurable. The default value is '127.0.0.1:3031'.
• Fetch modes involving replication now state that MySQL 8.0 and later are supported for replication between MySQL and CockroachDB.
• Partitioned tables can now be moved to CockroachDB using IMPORT INTO.
• Improved logging for the Fetch schema check phases under the trace logging level, which is set with --logging trace.
• Added a sample Grafana dashboard for monitoring MOLT tasks.
• Fetch now logs the name of the staging database in the target CockroachDB cluster used to store metadata for replication modes.
• String primary keys that use C collations on PostgreSQL can now be compared to the default en_US.utf8 on CockroachDB.
• MOLT is now distributed under the Cockroach Labs Product License Agreement, which is bundled with the binary.

​

August 26, 2024

• When a Fetch transformation rule is used to rename a table or map partitioned tables, a script in the format partitionTableScript.{timestamp}.ts is now automatically generated to ensure that replication works properly if enabled.

​

August 19, 2024

• Fixed a Fetch bug where --table-exclusion-filter was ignored when --table-filter and --schema-filter were set to the default ( '.*' ).

​

August 15, 2024

• Deprecated the --ongoing-replication flag in favor of --mode data-load-and-replication, using the new --mode flag. Users should replace all instances of --ongoing-replication with --mode data-load-and-replication.
• Fetch can now be run in an export-only mode by specifying --mode export-only. This will export all the data in csv or csv.gz format to the specified cloud or local store.
• Fetch can now be run in an import-only mode by specifying --mode import-only. This will load all data in the specified cloud or local store into the target CockroachDB database, effectively skipping the export data phase.
• Strings for the --mode flag are now word-separated by hyphens instead of underscores. For example, replication-only instead of replication_only.

​

August 8, 2024

• Added a replication-only mode for Fetch that allows the user to run ongoing replication without schema creation or initial data load. This requires users to set --mode replication_only and --replicator-flags to specify the defaultGTIDSet ( MySQL ) or slotName ( PostgreSQL ).
• Partitioned tables can now be mapped to renamed tables on the target database, using the Fetch transformations framework.
• Added a new --metrics-scrape-interval flag to allow users to specify their Prometheus scrape interval and apply a sleep at the end to allow for the final metrics to be scraped.
• Previously, there was a mismatch between the errors logged in log lines and those recorded in the exceptions table when an IMPORT INTO or COPY FROM operation failed due to a non-PostgreSQL error. Now, all errors will lead to an exceptions table entry that allows the user to continue progress from a certain table’s file.
• Fixed a bug that will allow Fetch to properly determine a GTID if there are multiple source_uuids.

​

July 31, 2024

• 'infinity'::timestamp values can now be moved with Fetch.
• Fixed an issue where connections were not being closed immediately after sharding was completed. This could lead to errors if the maximum number of connections was set to a low value.
• Fetch users can now exclude specific tables from migration using the --table-exclusion-filter flag.

​

July 18, 2024

• Fetch users can now specify columns to exclude from table migrations in order to migrate a subset of their data. This is supported in the schema creation, export, import, and direct copy phases.
• Fetch now automatically maps a partitioned table from a PostgreSQL source to the target CockroachDB schema.
• Fetch now supports column exclusions and computed column mappings via a new transformations framework.
• The new Fetch --transformations-file flag specifies a JSON file for schema/table/column transformations, which has validation utilities built in.

​

July 10, 2024

• Fixed a bug that led to incorrect list continuation file behavior if a trailing slash was provided in --bucket-path.
• Fixed a bug with extracting the filename from a failed import URL. Previously, an older filename was being used, which could result in duplicated data. Now, the filename that is used in import matches what is stored in the exceptions log table.
• Added a --use-implicit-auth flag that determines whether implicit authentication is used for cloud storage import URIs.

​

July 8, 2024

• Fixed a bug where integer values for JSONB / JSON types were not correctly handled when migrating from a PostgreSQL or CockroachDB source.
• Optimized the logic for clearing continuation tokens. Now, the clearing option is only presented in the relevant modes and when there are tokens to clear.
• CockroachDB sources now exclusively accept connections from admin or root users.
• The statement timeout can now be adjusted during the export phase. This guards against hanging queries.

​

May 31, 2024

• Renamed the --table-splits flag to --concurrency-per-table, which is more descriptive.
• Increased the default value of --import-batch-size to 1000. This leads to better performance on the target post-migration. Each individual import job will take longer, since more data is now imported in each batch, but the sum total of all jobs should take the same (or less) time.

​

May 29, 2024

• Added an --import-batch-size flag, which configures the number of files to be imported in each IMPORT job.
• In some cases on the previous version, binaries would not work due to how molt was being built. Updated the build method to use static linking, which creates binaries that should be more portable.
• VARBIT <> BOOL conversion is now allowed for Fetch and Verify. The bit array is first converted to UINT64. A resulting 1 or 0 is converted to true or false accordingly. If the UINT64 is another value, an error is emitted.

​

May 20, 2024

• MOLT tools now enforce secure connections to databases as a default. The --allow-tls-mode-disable flag allows users to override that behavior if secure access is not possible.
• When using MySQL as a source, --table-concurrency and --export-concurrency are strictly set to 1.
• Fixed a bug involving history retention for DECIMAL values.

​

May 3, 2024

• Fetch now supports CockroachDB multi-region tables.
• Fetch now supports continuous replication for PostgreSQL and MySQL source databases via the --ongoing-replication flag. When Fetch finishes the initial data load phase, it will start the replicator process as a subprocess, which runs indefinitely until the user ends the process with a SIGTERM ( ctrl-c ).
• Replicator flags for ( PostgreSQL and MySQL ) are now supported, allowing users to further configure the --ongoing-replication mode for their use case.
• Added the --type-map-file flag, which enables custom type mapping for schema creation.
• Fixed a bug where primary key positions could be missed when creating a schema with multiple primary keys.
• Added a default mode for MySQL sources that ensures consistency and does not leverage parallelism. New text is displayed that alerts the user and links to documentation in cases where fetching from MySQL might not be consistent.
• Logging for continuation tokens is now omitted when data export does not successfully complete.

​

April 10, 2024

• Fixed an inconsistency where inserting 'null' as a string into JSON / JSONB columns did not align with PostgreSQL’s case-sensitive null handling.

​

March 22, 2024

• Fixed a bug where running Verify with a PostgreSQL or CockroachDB source would cause an error when there are dropped columns on the table.

​

March 13, 2024

• An error is now displayed if the original column is a PostgreSQL nested array.
• Added sharded data export functionality for faster performance.
• Added support for the --table-handling 'drop-on-target-and-recreate' flag for MySQL sources, enabling automatic schema recreation.
• Added support for ENUM data types in schema creation.
• Deprecated the --s3-bucket and --gcp-bucket flags in favor of a single --bucket-path flag.

​

February 22, 2024

• Added implicit authentication for GCP sources.
• Resolved a CSV write deadlock issue for large file exports.
• Added new dashboard metrics: molt_verify_num_tables (categorizes tables as verified, missing, or extraneous ) and molt_verify_duration_ms (how long a Verify run took to complete, in milliseconds).
• Previously, if a comparison between heterogeneous column types occurred, Verify would panic and stop execution. Now, Verify no longer panics, but records the error and reports it to logging, and also marks the row as having mismatching columns. The error message is logged at most once per shard to avoid log spewing. Additionally, introduced the concept of conditional success, which means that all comparable row columns were compared and succeeded with the same value. Upon conditional success, the user is instructed to check all warnings and verify the data.
• Marked the --live and --continuous flags as dependencies for other sub-configuration flags. If a flag is set without its dependency, MOLT will indicate which dependency flag is not correctly set.
• Added the RFC for MOLT Fetch, which explains the architectural decisions and considerations of how Fetch was built. The RFC also outlines limitiations of the current system and further improvements around usability, performance, and integration with replication tools.
• Added logging annotations for data-level errors and information that surfaced during the Verify process, as well as summary status logs about the overall functioning of the job (e.g., 10000 truth rows seen, 5000 mismatches, etc.). These can be filtered so that exact tables and rows can be triaged in one place.
• Changed the output for the running summary statistics so that they are now more easily indexable by log processing tools. Previously, it was more human-readable, but fields were not easily indexable out of the box. Now, there is a balance between readability and downstream processing.
• Added tagging for summary and data logs so that users can filter the logs in a variety of log processing tools, or grep in the command line. This solves the pain point of all the logs going to the same location without any differentiation. Now, it is possible to view types of logs one at a time. Summary logs report aggregated results on either a task or step in a task (e.g., task took 10 minutes; 500 tables processed overall). Data logs report on individual or row-level operations and quantitative markers (e.g., 1000 rows completed so far; failed to verify item with PK 1000)
• Marked the --s3-bucket, --gcp-bucket, and --local-path flags as mutually exclusive because they all represent different intermediary stores.
• Adjusted settings for the number of rows and the size of data to be flushed so that only one rule is applied at any given time.
• Fixed a bug where outputs containing special characters were not formatted correctly.
• When running the import query for Fetch, the import statement is now printed if the logger is in debug mode.
• Updated the MOLT README with an OS + Docker-based edge case with Fetch.
• Logging for Fetch and Verify now defaults to structured JSON logging, which contains at least the message, level, and formatted time string. The --use-console-writer flag enables prettier console writing that has colored and human-readable log output. Structured logging is the default because it is easier to parse with log ingestion tools and is more efficient for the application to output.
• Task/data logs can now be written to specified log files using --log-file. If the flag is left empty, the MOLT tool will only write to stdout, which is more efficient.