Kafka Setup Guide

Kafka as a destination (including the Confluent and Redpanda variants) is deprecated and no longer actively maintained. It remains fully functional and no code is currently being removed. For new mirrors, we recommend ClickHouse, ClickHouse Cloud, or Postgres as the destination.

If looking to try out without an already setup Kafka endpoint, see Redpanda quickstart, by default no authentication is necessary. To get the above redpanda container to work with PeerDB on OSS, you can first add it to the PeerDB containers’ network:

# container id of redpanda-0
docker network connect peerdb_network $(docker container ls -qf "name=redpanda-0")

# container id of redpanda-console
docker network connect peerdb_network $(docker container ls -qf "name=redpanda-console")

And then in UI you can create the Kafka peer like so:

PeerDB only supports plaintext and SASL authentication. If username isn’t supplied then PeerDB will attempt to connect without authentication. Multiple servers can be specified as a comma separated list. Partitioner can be usually be ignored; it maps to partitioning strategies in franz-go. To set partitioning keys you’ll need to specify Script when creating mirror, see Lua reference for details.

Elasticsearch Setup Guide Confluent Cloud Setup Guide

⌘I

Home

Quickstart

PeerDB Cloud

Setup

Queues Setup

Mirror

5-minute Tutorials

Features

Data types

Best practices

Use-cases

Benchmarks

Contributing

Metrics and Tuning

FAQs