CASSSIDECAR-419 : Add Docker compose setup for CDC by jyothsnakonisa · Pull Request #330 · apache/cassandra-sidecar

jyothsnakonisa · 2026-03-27T23:01:13Z

No description provided.

skoppu22 · 2026-03-30T09:12:44Z

+
+## Architecture
+
+```


Looks nice, easy to understand for new users

skoppu22

LGTM

jmckenzie-dev · 2026-03-30T13:20:41Z

+
+YAML="/etc/cassandra/cassandra.yaml"
+
+patch_yaml() {


Would it be worth adding a "didn't find it, couldn't patch" style warning or error here?

smiklosovic · 2026-04-08T16:37:51Z

+
+WORKDIR /build
+
+# Build descriptors first — keeps the dependency-download layer cached separately.


this is confusing a bit as the user has to do "gradle jar" at least least before this is executed. Without doing "jar", what you copy will not contain "sidecar.version" and then it all passes but it fails to start like this

sidecar-1 | INFO [main] 2026-04-08 16:14:45,333 SidecarSchemaModule.java:59 - Registering table schema: sidecar_internal.restore_range_v1 sidecar-1 | INFO [main] 2026-04-08 16:14:45,333 SidecarSchemaModule.java:59 - Registering table schema: sidecar_internal.restore_slice_v3 sidecar-1 | Exception in thread "main" com.google.inject.ProvisionException: Unable to provision, see the following errors: sidecar-1 | sidecar-1 | 1) [Guice/ErrorInCustomProvider]: IllegalStateException: Failed to retrieve Sidecar version from resource /sidecar.version sidecar-1 | at UtilitiesModule.sidecarVersionProvider(UtilitiesModule.java:88) sidecar-1 | at ConfigurationModule.instancesMetadata(ConfigurationModule.java:175) sidecar-1 | \_ for 4th parameter sidecar-1 | at ConfigurationModule.instancesMetadata(ConfigurationModule.java:175) sidecar-1 | at InstanceMetadataFetcher.<init>(InstanceMetadataFetcher.java:51)

smiklosovic · 2026-04-08T18:28:17Z

I suggest to also include this to make it more robust. I like to be explicit in the creation of a topic. If somebody starts the stack and goes to kafka-ui then there will not be any topic for them to see messages in which might be confusing.

7eb33fc

jyothsnakonisa · 2026-05-21T16:36:46Z

@smiklosovic Thank you, will add 7eb33fc as a separate PR

jmckenzie-dev

Still working on testing it; my immediate naive attempt to run things went like:

Try and build the dockerfile locally in the directory (failed; pathing needs to be REPO_ROOT)
Look for scripts in REPO_ROOT/scripts for running the docker test (nothing)
Then actually read the documentation in the docker directory to see what I'm actually supposed to be doing 😀
Run into an error with docker compose on the Dockerfile; try start.sh with --help and get no help there

So there's some devx "intuitive explorability" rough edges that I think we could do some work to smooth over. Will update more when I figure out why I'm getting these errors on building the docker file and determine if it's a local env problem or with the patch.

jmckenzie-dev · 2026-05-21T19:54:04Z

    implementation "com.esotericsoftware:kryo-shaded:${kryoVersion}"

+    // Confluent Avro serializer — used when value.serializer=KafkaAvroSerializer (confluent mode)
+    implementation 'io.confluent:kafka-avro-serializer:7.6.0'


Is this only needed for this demo? Or do we need this for execution in the sidecar broadly for the avro serializer support in CDC? If the former, I'm a little on the fence on including this non-trivial dependency in the broader project vs. having a separate gradle subproject for the demo so it doesn't pollute the primary project namespaces w/stuff we only need for the demo.

KafkaAvroSerializer is a popular schemastore integrated serializer. If one wants to use it, without this dependency, they will run into class not found issues.

Stefan was trying this serializer and ran into class not found error hence I added this dependency as it is needed for both Demo and other users who want to use this serializer

jmckenzie-dev · 2026-05-21T19:54:51Z

 export CASSANDRA_USE_JDK11=true
+# Trunk (5.1-SNAPSHOT) compiles 2700+ source files; without an explicit heap
+# limit ant hits the JVM default (~2GB) and gets OOM-killed on large executors.
+export ANT_OPTS="${ANT_OPTS:-} -Xmx4g"


Is this related to this CDC dockerfile demo setup out of curiosity? I'm fine either way, just wondered how they connected.

jmckenzie-dev · 2026-05-21T19:55:26Z

+| `cassandra` | `cassandra:5.0` | CDC-enabled Cassandra node |
+| `cassandra-init` | `cassandra:5.0` | One-shot: seeds sidecar schema + configs |
+| `sidecar` | `cassandra-sidecar:dev` | Reads commit logs, publishes to Kafka |
+| `kafka-ui` | `ghcr.io/kafbat/kafka-ui:v1.6.1` | Browse topics + decoded Avro messages |


The file has 1.5.0 and we have 1.6.1 here - update this or that?

I tried 1.6.1 version earlier and it worked, after a while that version was no longer available. I then changed it to 1.5.0 but missed updating it in the readme. will update it!

jmckenzie-dev · 2026-05-21T19:56:34Z

+        'docker compose logs -f sidecar 2>&1 | grep -m 1 "CDC iterators started successfully"' \
+        > /dev/null || echo "Warning: timed out waiting for CDC iterators — check: docker compose logs sidecar"
+else
+    docker compose logs -f sidecar 2>&1 | grep -m 1 "CDC iterators started successfully" > /dev/null || true


Do we have any other options on macOS instead of just "follow forever"? :)

jmckenzie-dev · 2026-05-21T19:57:08Z

+# ── Wait for sidecar ─────────────────────────────────────────────────────────
+echo ""
+echo "Waiting for sidecar to be ready (follow progress: docker compose logs -f cassandra-init sidecar)..."
+until curl -sf http://localhost:9043/api/v1/__health > /dev/null 2>&1; do


This can infinite loop in the error scenario; a timeout would be friendly. 5 minutes or something - whatever makes sense.

jmckenzie-dev · 2026-05-21T20:16:32Z

+SKIP_BUILD=false
+SERIALIZER_MODE=confluent
+
+for arg in "$@"; do


No --help - could we add that?

jyothsnakonisa force-pushed the cdc-docker-compose branch from e4440eb to 65c1f70 Compare March 27, 2026 23:06

skoppu22 reviewed Mar 30, 2026

View reviewed changes

skoppu22 approved these changes Mar 30, 2026

View reviewed changes

jmckenzie-dev reviewed Mar 30, 2026

View reviewed changes

jyothsnakonisa force-pushed the cdc-docker-compose branch 3 times, most recently from 06c2451 to 1cd0b8d Compare April 6, 2026 20:00

jyothsnakonisa force-pushed the cdc-docker-compose branch from 1cd0b8d to 11d4fbc Compare April 8, 2026 14:30

smiklosovic reviewed Apr 8, 2026

View reviewed changes

Comment thread docker/cdc-demo/docker-compose.yml Outdated

smiklosovic reviewed Apr 8, 2026

View reviewed changes

Comment thread docker/cdc-demo/scripts/start.sh

jyothsnakonisa force-pushed the cdc-docker-compose branch from 11d4fbc to 336a853 Compare May 12, 2026 20:46

jyothsnakonisa force-pushed the cdc-docker-compose branch from 3b6da17 to bf6f8c2 Compare May 20, 2026 23:20

smiklosovic self-requested a review May 21, 2026 06:17

smiklosovic approved these changes May 21, 2026

View reviewed changes

jkonisa added 2 commits May 21, 2026 08:41

CASSSIDECAR-419 : Add Docker compose setup for CDC

010949c

fixing build

07fc335

jyothsnakonisa force-pushed the cdc-docker-compose branch from bf6f8c2 to 07fc335 Compare May 21, 2026 15:41

jmckenzie-dev requested changes May 21, 2026

View reviewed changes


		WORKDIR /build

		# Build descriptors first — keeps the dependency-download layer cached separately.


		## Architecture

		```


		YAML="/etc/cassandra/cassandra.yaml"

		patch_yaml() {

Conversation

jyothsnakonisa commented Mar 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

skoppu22 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

smiklosovic Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

smiklosovic commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jyothsnakonisa commented May 21, 2026

Uh oh!

jmckenzie-dev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smiklosovic Apr 8, 2026 •

edited

Loading

smiklosovic commented Apr 8, 2026 •

edited

Loading