Apply and restore use an atomic BSON before-image comparison with simple (binary) collation. A scalar changed to an array, a missing field changed to null, extra fields, or case-only string drift fails the comparison. Numeric BSON values follow MongoDB numeric equality; nested document key order remains significant. A string _id under a non-simple collection collation may require more scanning; review the query plan for large collections.
dataTasks uses one DataTaskJob configuration for release-scoped index checks, filtered data synchronization, local field edits, reviewed preview, affected-scope backup, and restore.
Backup manifests and data rows are read as canonical EJSON. Only manifest control fields (version, entryCount, maxBytes) are converted from safe numeric BSON values to JavaScript numbers. Identity, target IDs, before/after images, and index keys retain their BSON types and key order. An unsafe or ambiguous legacy control value fails backup validation instead of being guessed.
Start with Production Data Migration to choose the right tool. See Production Rollout for the complete release sequence.
Use the named ESM export:
import MonSQLize, { dataTasks, type DataTaskJob } from 'monsqlize';Read the same export from the CommonJS package object:
const { dataTasks } = require('monsqlize');| Method | Database writes | Purpose |
|---|---|---|
preview(job, options?) |
No | Read indexes and data, then return the diff, backup estimate, and short-lived approval |
apply(job, { approval }) |
Yes | Replan, validate approval, back up, then apply reviewed index and data changes |
previewRestore(backup, options?) |
No | Validate the manifest and target state, then return restore actions and approval |
restore(backup, { approval, target? }) |
Yes | Create a restore safety backup, then restore documents and task-created indexes |
dataTasks is not an instance method. There is no task runner or second step-based configuration.
const development = new MonSQLize({
type: 'mongodb',
databaseName: 'development',
config: { uri: process.env.SOURCE_MONGODB_URI! }
});
const production = new MonSQLize({
type: 'mongodb',
databaseName: 'production',
config: { uri: process.env.TARGET_MONGODB_URI! }
});
await Promise.all([development.connect(), production.connect()]);
const job = {
name: 'release-2026-07-feature-modules',
description: 'Publish the feature modules required by this release',
source: development,
target: production,
targetEnvironment: 'production',
collections: [{
name: 'feature_modules',
targetName: 'feature_modules',
indexes: [
{ key: { code: 1 }, options: { unique: true } },
{ key: { release: 1, enabled: 1 } }
],
data: {
filter: { release: '2026-07' },
projection: { _id: 1, code: 1, legacyName: 1, enabled: 1, developmentOnly: 1 },
identity: { mode: 'fields', fields: ['code'] },
strategy: 'upsert',
rename: { legacyName: 'name' },
set: { schemaVersion: 2 },
unset: ['developmentOnly'],
batchSize: 500,
maxDocuments: 10_000
},
verify: {
mode: 'full',
fields: ['code', 'name', 'enabled', 'schemaVersion']
}
}],
backup: {
dir: './.monsqlize/data-tasks',
format: 'extended-jsonl',
compression: 'gzip',
retentionDays: 7,
maxBytes: 256 * 1024 * 1024
},
lock: { ttlMs: 120_000, waitTimeoutMs: 0 }
} satisfies DataTaskJob;SDK jobs can use two connected monSQLize instances; the caller owns their lifecycle. CLI jobs must use two MonSQLizeOptions values. dataTasks closes connections that it creates.
Close both SDK instances in finally before the process exits. See examples/docs/data-tasks.ts for the complete runnable lifecycle.
| Path | Required | Default | Meaning |
|---|---|---|---|
name |
Yes | - | Stable name included in the job hash, approval, and manifest |
description |
No | None | Human-readable context without execution semantics |
source |
Yes | - | Read-only source runtime or MongoDB MonSQLizeOptions |
target |
Yes | - | Independent target runtime or MongoDB MonSQLizeOptions |
targetEnvironment |
Yes | - | development/test/staging/production/prod/live; production aliases require durable backup |
collections |
Yes | - | Non-empty array; a target collection can appear only once |
backup |
Production | System temp outside production | Affected-scope rollback package settings |
lock |
No | false |
true or lease options to prevent concurrent data task writers |
Every collection needs non-empty indexes, data, or both.
| Path | Required | Default | Meaning |
|---|---|---|---|
collections[].name |
Yes | - | Source collection name |
collections[].targetName |
No | Same as name |
Target collection name |
collections[].indexes |
Conditional | [] |
Exact indexes required by this release; source indexes are not copied wholesale |
indexes[].key |
Yes | - | Ordered MongoDB index key |
indexes[].name |
No | Generated by MongoDB | Set for readability when useful; recovery uses the actual returned database name |
indexes[].options |
No | {} |
Driver index options; do not put name inside options |
Preview calls listIndexes() first for every target collection and classifies each declaration as existing, missing, or conflict. Apply creates only missing indexes. It never drops, renames, or rebuilds a conflicting index.
identity.mode: 'fields' requires an exact non-partial unique index in the target or in the same collection's declarations. Proving a missing unique index requires a final-target-image scan. The task blocks when that scan exceeds 10,000 documents or an index shape cannot be proved exactly.
| Path | Required | Default | Meaning |
|---|---|---|---|
data.filter / data.all |
Exactly one | - | Non-empty filter or explicit all: true; an empty filter is rejected |
data.identity |
Yes | - | Target matching policy |
data.strategy |
No | upsert |
upsert updates or inserts; insert fails when identity already exists |
data.projection |
No | All fields | Source projection; must preserve identity, conflictBy, and rename source fields |
data.rename |
No | {} |
Rename only listed paths, as { oldPath: 'newPath' } |
data.set |
No | {} |
Set only listed BSON literal values; dotted paths are supported |
data.unset |
No | [] |
Remove only listed paths |
data.batchSize |
No | 500 |
Write-ahead checkpoint batch size, from 1..10000 |
data.maxDocuments |
No | 10000 |
Maximum source documents that the collection may plan |
Data task source reads use a bounded internal stream and do not inherit the ordinary findLimit default. Planning reads at most maxDocuments + 1 source documents so concurrent growth fails closed instead of silently truncating or loading an unbounded result.
Field edits run in this order: filter -> projection -> rename -> set -> unset -> diff. They are deliberately limited to deterministic edits of a few release fields:
- Paths cannot touch
_id, identity fields, or overlap one another. __proto__,prototype, andconstructorpath segments are rejected.- Rename blocks when the destination already contains a different value.
- Dotted paths block when an intermediate value is not an object.
setaccepts BSON-serializable literals, not functions, random values, or update expressions.
Business-key matching:
identity: { mode: 'fields', fields: ['tenantId', 'code'] }Updates preserve the target _id; inserts let the target MongoDB generate _id. Use this when environments assign IDs independently.
Source-ID matching:
identity: { mode: 'source-id', conflictBy: ['tenantId', 'code'] }Inserts preserve source _id, and existing documents update only when _id matches. conflictBy detects the same business key under another _id and blocks preview.
| Path | Default | Meaning |
|---|---|---|
verify.mode |
sample |
sample or full; controls field-content comparison only |
verify.fields |
Planned write fields | Field paths compared after apply |
verify.sampleSize |
20 |
Stable sample count, from 1..1000 |
backup.format |
extended-jsonl |
BSON-preserving format; currently the only format |
backup.compression |
gzip |
gzip or none |
backup.retentionDays |
7 |
Retention metadata; files are not deleted automatically |
backup.maxBytes |
256 MiB | Maximum uncompressed rollback data size |
lock.ttlMs |
120000 |
Renewable target-database lease duration |
lock.waitTimeoutMs |
0 |
Maximum wait when another task holds the lease |
Index, identity, count, manifest, and actual-write checks are always full. verify.mode only changes user-selected field content verification.
const preview = await dataTasks.preview(job, {
sampleSize: 10,
approvalTtlMs: 15 * 60 * 1000
});
if (!preview.passed || !preview.approval) throw new Error(preview.errors.join('; '));
const result = await dataTasks.apply(job, { approval: preview.approval });Preview performs no writes. It returns index states, source/insert/update/unchanged/conflict, change samples, backup document count, and estimated bytes. Approval binds the normalized job, patched source image, target preimages, target indexes, and expiry.
Apply runs in this order:
- Acquire the optional target lease.
- Rebuild the full plan and reject approval on any hash drift.
- Write and read back the affected-scope backup.
- Read indexes again and create only reviewed missing indexes.
- Record each batch as pending before ordered data writes.
- Use target-preimage CAS filters to avoid overwriting concurrent changes.
- Read back the exact expected after-image and verify fields, identity, indexes, and manifest.
- Record final status and release the lease.
Indexes and cross-collection writes are not one MongoDB transaction. A mid-run error returns failed or partial, preserves the manifest, and never restores automatically.
const restorePreview = await dataTasks.previewRestore(result.backup);
if (!restorePreview.passed || !restorePreview.approval) throw new Error(restorePreview.errors.join('; '));
const restored = await dataTasks.restore(result.backup, {
approval: restorePreview.approval
});Restore preview verifies that the target still matches the applied after-image and resolves pending operations when a process stopped between a database write and manifest checkpoint. Ambiguous or later target changes block with RESTORE_DRIFT.
Restore creates a safety backup before writing. Document operations use exact current-image filters, and index recovery only touches indexes actually created by the task whose definitions remain unchanged. The safety backup can itself be previewed and restored to undo a restore.
The task file must directly export the DataTaskJob:
module.exports = {
name: 'release-settings',
source: { type: 'mongodb', databaseName: 'development', config: { uri: process.env.SOURCE_URI } },
target: { type: 'mongodb', databaseName: 'production', config: { uri: process.env.TARGET_URI } },
targetEnvironment: 'production',
collections: [{
name: 'settings',
data: { all: true, identity: { mode: 'source-id', conflictBy: ['code'] } }
}],
backup: { dir: './.monsqlize/data-tasks' }
};monsqlize data-task preview --task ./tasks/release.cjs --out preview.json --json
monsqlize data-task apply --task ./tasks/release.cjs --approval preview.json --out result.json --json
monsqlize data-task preview-restore --task ./tasks/release.cjs --backup ./path/manifest.json --out restore-preview.json --json
monsqlize data-task restore --task ./tasks/release.cjs --backup ./path/manifest.json --approval restore-preview.json --jsonInvalid configuration, stale approval, lock conflict, partial/failed status, checksum errors, and restore drift exit with code 1.
DataTaskJobError exposes code, phase, and optional collection. Stable codes are INVALID_JOB, IDENTITY_CONFLICT, INDEX_CONFLICT, APPROVAL_STALE, BACKUP_FAILED, LOCK_NOT_ACQUIRED, LOCK_LOST, APPLY_PARTIAL, RESTORE_DRIFT, and RESTORE_FAILED.
- A backup is an affected-scope rollback package, not a full backup or point-in-time restore.
- dataTasks does not synchronize Model schema, copy every source index, maintain migration directories, or run ongoing CDC.
- Use MongoDB native or managed backup tools for full moves and disaster recovery.
- Use Change Stream sync for ongoing changes after the initial backfill.