Create mirror
curl --request POST \
--url localhost:3000/api/v1/flows/cdc/create \
--header 'Authorization: Basic OnJsYWNrd3dhbmEyMw==' \
--header 'Content-Type: application/json' \
--data '
{
"connection_configs": {
"flow_job_name": "mirror_api_kick_off",
"source_name": "rds_peer",
"destination_name": "ch_peer",
"table_mappings": [
{
"source_table_identifier": "public.users",
"destination_table_identifier": "users_api"
},
{
"source_table_identifier": "public.payments",
"destination_table_identifier": "payments_api"
},
{
"source_table_identifier": "public.optional_ordering_key",
"destination_table_identifier": "optional_ordering_key",
"columns": [
{
"name": "id",
"ordering": 1
},
{
"name": "created_at",
"ordering": 2
}
]
},
],
"max_batch_size": 1000,
"idle_timeout_seconds": 300,
"publication_name": "",
"do_initial_snapshot": true,
"snapshot_num_rows_per_partition": 5000,
"snapshot_max_parallel_workers": 4,
"snapshot_num_tables_in_parallel": 2,
"resync": false,
"initial_snapshot_only": false,
"soft_delete_col_name": "_peerdb_is_deleted",
"synced_at_col_name": "_peerdb_synced_at"
}
}'
{
"workflowId": "mirror_kick_off-peerflow-2d269226-9757-4d9f-8cfd-6adb1ca29c0e"
}
POST api/v1/flows/cdc/create
This endpoint is used to create a CDC mirror.
Request Fields
Connection configuration
The name of the mirror to be created.
The name of the source peer (data store).
The name of the destination peer (data store).
Table mappings
The source table identifier.
The destination table identifier.
Columns from the source table to exclude, separated by commas.
For Clickhouse Only And Optional. Ordering key setting for Clickhouse target tables created by PeerDB.
Whether to do an initial snapshot.
The maximum number of rows to sync in a batch.
How often the mirror syncs, in seconds.
The name of the publication to use. PeerDB will try to create a publication if not provided here.
The number of rows per partition to sync during the initial snapshot. Default is 1 million rows.
The maximum number of parallel workers to use during the initial snapshot.
The number of tables to sync in parallel during the initial snapshot.
Whether to resync the mirror. The mirror must be dropped before resyncing.
Whether to do only the initial snapshot and not perform CDC.
The name of the column that indicates a soft delete.
The name of the column that indicates when the row was last synced.
Response Fields
The ID of the parent workflow created for this mirror.
curl --request POST \
--url localhost:3000/api/v1/flows/cdc/create \
--header 'Authorization: Basic OnJsYWNrd3dhbmEyMw==' \
--header 'Content-Type: application/json' \
--data '
{
"connection_configs": {
"flow_job_name": "mirror_api_kick_off",
"source_name": "rds_peer",
"destination_name": "ch_peer",
"table_mappings": [
{
"source_table_identifier": "public.users",
"destination_table_identifier": "users_api"
},
{
"source_table_identifier": "public.payments",
"destination_table_identifier": "payments_api"
},
{
"source_table_identifier": "public.optional_ordering_key",
"destination_table_identifier": "optional_ordering_key",
"columns": [
{
"name": "id",
"ordering": 1
},
{
"name": "created_at",
"ordering": 2
}
]
},
],
"max_batch_size": 1000,
"idle_timeout_seconds": 300,
"publication_name": "",
"do_initial_snapshot": true,
"snapshot_num_rows_per_partition": 5000,
"snapshot_max_parallel_workers": 4,
"snapshot_num_tables_in_parallel": 2,
"resync": false,
"initial_snapshot_only": false,
"soft_delete_col_name": "_peerdb_is_deleted",
"synced_at_col_name": "_peerdb_synced_at"
}
}'
{
"workflowId": "mirror_kick_off-peerflow-2d269226-9757-4d9f-8cfd-6adb1ca29c0e"
}
curl --request POST \
--url localhost:3000/api/v1/flows/cdc/create \
--header 'Authorization: Basic OnJsYWNrd3dhbmEyMw==' \
--header 'Content-Type: application/json' \
--data '
{
"connection_configs": {
"flow_job_name": "mirror_api_kick_off",
"source_name": "rds_peer",
"destination_name": "ch_peer",
"table_mappings": [
{
"source_table_identifier": "public.users",
"destination_table_identifier": "users_api"
},
{
"source_table_identifier": "public.payments",
"destination_table_identifier": "payments_api"
},
{
"source_table_identifier": "public.optional_ordering_key",
"destination_table_identifier": "optional_ordering_key",
"columns": [
{
"name": "id",
"ordering": 1
},
{
"name": "created_at",
"ordering": 2
}
]
},
],
"max_batch_size": 1000,
"idle_timeout_seconds": 300,
"publication_name": "",
"do_initial_snapshot": true,
"snapshot_num_rows_per_partition": 5000,
"snapshot_max_parallel_workers": 4,
"snapshot_num_tables_in_parallel": 2,
"resync": false,
"initial_snapshot_only": false,
"soft_delete_col_name": "_peerdb_is_deleted",
"synced_at_col_name": "_peerdb_synced_at"
}
}'
{
"workflowId": "mirror_kick_off-peerflow-2d269226-9757-4d9f-8cfd-6adb1ca29c0e"
}