add http sse cosyvocie tts api and omni createItem function#205
add http sse cosyvocie tts api and omni createItem function#205songguocola wants to merge 2 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces HTTP-based Text-to-Speech (TTS) synthesis support and enhances the Omni and Qwen realtime modules with connection timeouts and improved WebSocket error handling. Key additions include the HttpSpeechSynthesizer class, associated parameter/result models, and a createItem method for tool call results. Review feedback identifies a bug in the synchronous TTS call where errors are not correctly propagated, potentially leading to silent failures, and suggests replacing generic RuntimeException usage with structured ApiException objects for better error handling consistency.
| public ByteBuffer callAndReturnAudio(HttpSpeechSynthesisParam param) | ||
| throws ApiException, NoApiKeyException, InputRequiredException { | ||
|
|
||
| ByteArrayOutputStream audioBuffer = new ByteArrayOutputStream(); | ||
| CountDownLatch latch = new CountDownLatch(1); | ||
|
|
||
| streamCall( | ||
| param, | ||
| new ResultCallback<HttpSpeechSynthesisResult>() { | ||
| @Override | ||
| public void onEvent(HttpSpeechSynthesisResult result) { | ||
| if (result.getAudioData() != null) { | ||
| try { | ||
| audioBuffer.write(result.getAudioData()); | ||
| } catch (Exception e) { | ||
| throw new ApiException(e); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void onComplete() { | ||
| if (latch.getCount() > 0) { | ||
| latch.countDown(); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void onError(Exception e) { | ||
| if (latch.getCount() > 0) { | ||
| latch.countDown(); | ||
| } | ||
| throw new ApiException(e); | ||
| } | ||
| }); | ||
|
|
||
| try { | ||
| if (latch.getCount() > 0) { | ||
| boolean completed = latch.await(60 * 10, TimeUnit.SECONDS); | ||
| if (!completed) { | ||
| Status timeoutStatus = Status.builder() | ||
| .statusCode(408) // 或 504 Gateway Timeout | ||
| .code("RequestTimeOut") | ||
| .message("Timeout waiting for audio data from server.") | ||
| .build(); | ||
| throw new ApiException(timeoutStatus); | ||
| } | ||
| } | ||
| return ByteBuffer.wrap(audioBuffer.toByteArray()); | ||
| } catch (Exception e) { | ||
| log.error("Failed to get audio data", e); | ||
| throw new ApiException(e); | ||
| } | ||
| } |
There was a problem hiding this comment.
The callAndReturnAudio method has a significant issue with error propagation. When an error occurs in the asynchronous streamCall (reported via onError), the latch is released, but the method proceeds to return the current (and likely incomplete) audioBuffer without notifying the caller of the failure. This results in a silent failure where the user receives a partial or empty audio file as if the request succeeded.
Additionally, if an exception is thrown inside the onEvent callback (e.g., at line 178), it occurs on the callback thread and is not caught by the try-catch block in the main thread (line 212). If such an exception prevents latch.countDown() from being called, the method will hang until the 10-minute timeout is reached.
To fix this, you should use an AtomicReference<Exception> to capture any exception from the callback and rethrow it (wrapped in an ApiException) in the main thread after the latch is released.
1 participant
No description provided.