-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Open
Description
Hello,
When using the IBM ACE integration, we are seeing intermittent MQRC_TRUNCATED_MSG_FAILED errors in the Datadog Agent logs. Based on the IBM MQ documentation, this error occurs when the client specifies a buffer that is too small for a message being received.
Full exception in Datadog agent logs:
2026-02-18 13:44:16 GMT | CORE | ERROR | (pkg/collector/python/datadog_agent.go:144 in LogMessage) | ibm_ace:e879804a74649c26 | (subscription.py:86) | Subscription error for topic string: $SYS/Broker/+/Statistics/JSON/SnapShot/#/applications/#
Traceback (most recent call last):
File "/opt/datadog-agent/embedded/lib/python3.13/site-packages/datadog_checks/ibm_ace/subscription.py", line 69, in get_latest_messages
payload = self.sub.get(None, pymqi.md(), self._get_options())
File "/opt/datadog-agent/embedded/lib/python3.13/site-packages/pymqi/__init__.py", line 2517, in get
return self.sub_queue.get(max_length, *opts)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/opt/datadog-agent/embedded/lib/python3.13/site-packages/pymqi/__init__.py", line 2159, in get
raise MQMIError(rv[-2], rv[-1])
pymqi.MQMIError: MQI Error. Comp: 1, Reason 2080: WARNING: MQRC_TRUNCATED_MSG_FAILED
Impact
Metrics datapoints from the failed messages will not be sent to Datadog.
Steps to reproduce:
- Configure the agent's IBM ACE integration to monitor an IBM ACE server as described here.
- Load test with a large amount of integration servers/message queues. With sufficient volume, statistics messages will occasionally be too large and the above error will appear in the logs.
Investigation:
- Looking at the integration code here, there is currently no maximum buffer size being specified in the integration (first parameter in the
get()function call, currently set toNone.) - Looking at the behavior in the
pymqilibrary here, if a maximum buffer length is not specified, it falls back to 4096 bytes.
Suggested solutions:
- Use the
inquire()method (here) to query for the maximum message length allowed by the queue, and set the maximum buffer size dynamically. - Add a
try/exceptcase for the integration to re-request with a larger buffer size if this exception is thrown. - Make the buffer size configurable so operators can set it to match the maximum message length configured on the queue.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels