Tool | Source | Target | Throughput | Lag/Latency |
---|---|---|---|---|
PeerDB | Postgres | Data Warehouses and Storage | 5K TPS | 30s-1min |
PeerDB | Postgres | Queues | 5K TPS | 1s-5s |
Other tools such as Fivetran, DMS etc. | Postgres | Data Warehouses, Queues and Storage | 5K TPS | At least 5mins. A few tools degrade over time and lag can grow to multiple hours |
REPLICA IDENTITY FULL
on your tables to reliably stream TOAST columns. This can affect the source Postgres database by increasing the resource (compute and IO) utilization. Postgres also recommends this approach as a last fallback option. With PeerDB, we implemented a caching mechanism which eliminates the need of setting REPLICA IDENTITY FULL
to replicate TOAST columns. This avoids additional load on the source, making it significantly safer than the former approach.