Add try,catch statement for query - #65
Open
JaeGeunBang wants to merge 1 commit into
Open
Conversation
y-ken
added a commit
that referenced
this pull request
Jun 17, 2026
out_mysql_bulk's write/1 opened a fresh MySQL connection on every flush and closed it only on the success path. That had two problems: * A failing INSERT (e.g. a NOT NULL violation on bad input) skipped the close and leaked the connection on every Fluentd retry. * Even on success, reconnecting for each flush is wasteful. Now the connection is kept open and reused across successful flushes, and reconnected only when there is no usable handler or when the target database changed (the INSERT uses an unqualified table name, so it must run on a connection bound to the right database). On error the connection is dropped so the next retry reconnects (this also recovers from a server-side "gone away"), and the held connection is released on plugin shutdown via #close. The error is deliberately NOT rescued-and-swallowed. PR #65 proposed `rescue Exception` + log.warn to stop endless retries, but swallowing the error makes Fluentd treat the flush as successful and purge the chunk, silently dropping every buffered record in it (a whole bulk batch, not just the bad row). Letting it propagate keeps Fluentd's retry / secondary handling intact; operators who want to cap retries on permanently-bad data should use retry_max_times / <secondary>, which this change leaves working. Note: as before, write/1 uses a single @handler instance variable and is not safe for flush_thread_count > 1; that pre-existing limitation is unchanged. Add regression tests: - unit (no live MySQL): a stubbed handler proves write/1 reuses one connection across successful flushes and closes it on shutdown, and that a failing INSERT re-raises while closing and dropping the connection. - integration: a NOT NULL violation (the exact error from PR #65) must raise out of write/1 and insert nothing; a second successful flush reuses the same connection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add try, catch statements.
I am using fluent-plugin-mysql plygun to save log to MySQL periodically every minute.
However, after Mysql2 error, the data received from source can't save to MySQL. Keep trying retry.
Even though Mysql2 error occurs after adding try, catch statements, data received from source can save to MySQL.
Before
After