Python SDK: omitting `poll_interval` in `consumer_group()` polls with no delay — ~30% of a core on an idle topic, and nothing in Python documents it

[bartoszkobylinski]

TL;DR — Leaving poll_interval unset on consumer_group() makes the consumer re-poll as fast as the broker can answer. On a 1-vCPU host with an essentially idle topic that measured ~30% of the core (iggy-server 17.3% + consumer 12.6%) and ~34,700 context switches/sec; passing poll_interval=250ms took both processes below 0.3% with no functional change. This is my omission rather than an SDK bug — I'm filing it because two of my codebases made the identical omission independently, and because the consequence isn't discoverable from Python. Please re-triage as docs/enhancement if that fits better.

Bug description

My worker created its consumer without passing poll_interval. consume_messages(callback, shutdown_event) takes no interval either, so consumer_group() is the only place it can be set. With it unset the SDK leaves it None, and the consumer polls continuously.

The cost, on a topic receiving a handful of messages per day:

	unset	poll_interval=250ms
iggy-server	17.3 %	< 0.3 %
consumer process	12.6 %	< 0.3 %
context switches/sec	~34,700	~2,300–3,900
PSI cpu some avg10	70.2	1.6

Stopping just the broker and the consumer took the whole machine to 100% idle, so this loop was effectively everything the host was doing. The server's own CPU tracks the client's poll rate — it drops below 0.3% while a consumer is still attached and polling at 250ms.

Three things made it hard to notice:

1. In core/sdk/src/clients/consumer_builder.rs, polling_retry_interval is given a default of IggyDuration::ONE_SECOND, while polling_interval is a bare Option<IggyDuration> with no default — seeing the first handled for you makes the second look handled too.
2. The Python stub types it poll_interval: Optional[datetime.timedelta] = None, and the complete docstring for consumer_group is "Creates a new consumer group consumer. Returns the consumer or a PyRuntimeError on failure." — a Python user cannot learn what omitting it does without reading the Rust source.
3. The only poll_interval example in the docs is .poll_interval(IggyDuration::from_str("1ms")?) — 1000 polls/sec — presented with no note that it's a throughput setting for dedicated hardware.

That two separate projects of mine, written months apart, both omitted it is what makes me think it's the shape of the API rather than one careless call site.

Suggested direction

Either a small non-zero default for polling_interval, or (cheaper) one sentence in the Python consumer_group docstring plus a non-1ms example in the docs.

One question, not a complaint

Does Iggy support or plan server-side long-polling — the broker holding a poll open until a message arrives or a timeout expires (Kafka's fetch.max.wait.ms, SQS long polling)? That makes an idle consumer nearly free regardless of what the client sets, and would remove this category at the source rather than documenting around it.

Affected area / component

Python SDK (foreign/python), consumer configuration; core/sdk/src/clients/consumer_builder.rs defaults; docs (high-level SDK page).

Deployment

Single iggy-server binary under systemd, TCP transport on loopback, one stream / one topic / one partition, one consumer group.

Versions

Server iggy-server 0.8.0, Python SDK apache-iggy 0.8.0 (PyPI).

Hardware / environment

Proxmox LXC container, 1 vCPU, 4 GiB RAM, no swap, ~20 other services co-resident. Local disk.

Sample code

Before (the omission):

consumer = await client.consumer_group(
    group, stream, topic,
    polling_strategy=PollingStrategy.Next(),
    auto_commit=AutoCommit.After(AutoCommitAfter.ConsumingEachMessage()),
)
await consumer.consume_messages(on_message, shutdown)

After:

consumer = await client.consumer_group(
    group, stream, topic,
    polling_strategy=PollingStrategy.Next(),
    auto_commit=AutoCommit.After(AutoCommitAfter.ConsumingEachMessage()),
    poll_interval=timedelta(milliseconds=250),
)

Reproduction

1. Run iggy-server and one Python consumer built as above, on a box where you can see per-process CPU clearly (a single core makes it obvious).
2. Publish nothing — leave the topic idle.
3. Observe iggy-server and the consumer process in top, and the context-switch rate in vmstat 2.
4. Add poll_interval=timedelta(milliseconds=250) and repeat.

On Linux, cat /proc/pressure/cpu is the clearest single signal — it went from some avg10=70.2 to some avg10=1.6 across that change.

Note on framing

I want to be straightforward that this may be working as intended: Iggy is a throughput broker, and "set the interval, that's what it's for" is a fair answer. If that's the call, I'd suggest keeping only the documentation part — the Python docstring never mentions the consequence, and the sole published example is 1ms.