Error handling for Grobid when not responding by Sanakhamassi · Pull Request #35 · ScienciaLAB/document-qa

Sanakhamassi · 2026-04-23T18:45:20Z

Related to issue #11

Copilot

Pull request overview

Adds explicit error handling for Grobid failures so the Streamlit UI can surface a clear “please try later” message instead of failing ambiguously (issue #11).

Changes:

Introduce GrobidServiceError and raise it when Grobid errors or returns non-200.
Catch GrobidServiceError in the Streamlit upload/embedding flow and display an error message.
Add a (currently redundant) guard in DocumentQAEngine for missing Grobid output.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
`streamlit_app.py`	Catches Grobid failures during embedding creation and shows a user-facing error.
`document_qa/grobid_processors.py`	Defines `GrobidServiceError` and raises it from Grobid processing on failure/non-200.
`document_qa/document_qa_engine.py`	Imports/raises `GrobidServiceError` when Grobid structure is missing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

lfoppiano

There is a missing space, the rest looks fine. I did not test it, so please make sure you tested it before merge/squash.

…treamlit app

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

lfoppiano · 2026-05-06T11:13:28Z

+                tmp_file = NamedTemporaryFile()
+                tmp_file.write(bytearray(binary))
+                st.session_state['binary'] = binary
+
+                st.session_state['doc_id'] = hash = st.session_state['rqa'][model].create_memory_embeddings(
+                    tmp_file.name,
+                    chunk_size=chunk_size,
+                    perc_overlap=0.1
+                )


Yes, @Sanakhamassi here you need to either add tempFile in the with () or handle that somehow

+            st.session_state['doc_id'] = None
+            st.session_state['loaded_embeddings'] = False
+            st.session_state['uploaded'] = False
+            st.error(f"{message} Please try later.")


lfoppiano · 2026-05-06T11:09:07Z

        if grobid_url:
-            self.grobid_processor = GrobidProcessor(grobid_url)
+            self.grobid_processor = GrobidProcessor(grobid_url, ping_server=False)



@Sanakhamassi why was this removed?

+        try:
+            pdf_file, status, text = self.grobid_client.process_pdf("processFulltextDocument",
+                                                                    input_path,
+                                                                    consolidate_header=True,
+                                                                    consolidate_citations=False,
+                                                                    segment_sentences=False,
+                                                                    tei_coordinates=coordinates,
+                                                                    include_raw_citations=False,
+                                                                    include_raw_affiliations=False,
+                                                                    generateIDs=True)
+        except Exception as exc:
+            raise GrobidServiceError("Grobid service did not respond.") from exc

        if status != 200:
-            return
+            raise GrobidServiceError(
+                f"Grobid service returned status {status}.",
+                status_code=status
+            )


lfoppiano

Better than before, however there are few changes that needs to be done.

lfoppiano · 2026-05-06T11:09:07Z

        if grobid_url:
-            self.grobid_processor = GrobidProcessor(grobid_url)
+            self.grobid_processor = GrobidProcessor(grobid_url, ping_server=False)



@Sanakhamassi why was this removed?

lfoppiano · 2026-05-06T11:13:28Z

+                tmp_file = NamedTemporaryFile()
+                tmp_file.write(bytearray(binary))
+                st.session_state['binary'] = binary
+
+                st.session_state['doc_id'] = hash = st.session_state['rqa'][model].create_memory_embeddings(
+                    tmp_file.name,
+                    chunk_size=chunk_size,
+                    perc_overlap=0.1
+                )


Yes, @Sanakhamassi here you need to either add tempFile in the with () or handle that somehow

… type hints

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

document_qa/document_qa_engine.py:266

The type annotation says Tuple[List[Document], list], but this method actually returns a list of strings (context_as_text) plus coordinates. Update the annotation to match the returned value (e.g., Tuple[List[str], list]) or return List[Document] if that’s what callers should get.

    def query_storage(self, query: str, doc_id, context_size=4) -> Tuple[List[Document], list]:
        """
        Returns the context related to a given query
        """
        documents, coordinates = self._get_context(doc_id, query, context_size)

+                    st.session_state['binary'] = binary
+
+                    st.session_state['doc_id'] = hash = st.session_state['rqa'][model].create_memory_embeddings(
+                        tmp_path,
+                        chunk_size=chunk_size,
+                        perc_overlap=0.1
+                    )


            output_parser=None,
            context_size=4,
            extraction_schema=None,
            verbose=False
-    ) -> (Any, str):
+    ) -> Tuple[Any, str]:


        return parsed_output

-    def _run_query(self, doc_id, query, context_size=4) -> (List[Document], list):
+    def _run_query(self, doc_id, query, context_size=4) -> Tuple[List[Document], list]:


+                                                                    include_raw_affiliations=False,
+                                                                    generateIDs=True)
+        except Exception as exc:
+            raise GrobidServiceError("Grobid service did not respond.") from exc



                 grobid_url=None,
-                 memory=None
+                 memory=None,
+                 ping_grobid_server: bool = False


Finalizes PR #35 (issue #11). Resolves the outstanding Copilot and reviewer remarks and adds handling for Grobid responses that are empty or truncated (HTTP 200 with no usable body), which were previously undetected. grobid_processors.process_structure (single validation chokepoint): - Catch only requests.exceptions.RequestException instead of bare Exception, so local/usage errors (bad path, parser bugs) keep their real traceback instead of being mislabelled "Grobid did not respond". - Raise GrobidServiceError for empty body, malformed/truncated XML, and well-formed XML with no extractable text. document_qa_engine: - ping_grobid_server now defaults to True (fail-fast for library users); Streamlit passes ping_grobid_server=False to degrade gracefully. - Remove the now-dead "if not structure" guard. - Fix return-type hints (query_document now consistently returns a 3-tuple; query_storage -> tuple[List[str], list]; _run_query -> tuple[Any, list]); drop unused Tuple import; fix verbose hash print. streamlit_app: - Drop the `hash` alias that shadowed the builtin. - Clean the error message (no duplicated status, correct punctuation). tests: - Use requests exceptions; assert local errors are not masked; add empty / malformed / no-extractable-text cases. 19 passed.

@Sanakhamassi

This PR extended the work from @Sanakhamassi in #35

lfoppiano · 2026-06-07T18:35:11Z

Thanks @Sanakhamassi this PR has been merged through #43. There were some changes to be made and I simply done that in a local branch.

Add error handling for Grobid service when not responding

6a83c20

lfoppiano requested review from Copilot and lfoppiano April 24, 2026 08:47

Copilot started reviewing on behalf of lfoppiano April 24, 2026 20:50 View session

Copilot AI reviewed Apr 24, 2026

View reviewed changes

Comment thread streamlit_app.py Outdated

Comment thread streamlit_app.py Outdated

Comment thread document_qa/grobid_processors.py

Comment thread document_qa/document_qa_engine.py

Comment thread document_qa/grobid_processors.py

lfoppiano approved these changes Apr 26, 2026

View reviewed changes

Comment thread streamlit_app.py Outdated

Sanakhamassi added 2 commits April 30, 2026 10:52

Update GrobidProcessor initialization and enhance error handling in S…

5670b4b

…treamlit app

Fix missing space in error message

cf29781

lfoppiano requested a review from Copilot May 6, 2026 10:55

Copilot started reviewing on behalf of lfoppiano May 6, 2026 10:56 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

lfoppiano requested changes May 6, 2026

View reviewed changes

Sanakhamassi added 3 commits May 7, 2026 11:23

use temporary file with suffix and ensure cleanup

a7f3546

Enhance DocumentQAEngine with ping_grobid_server parameter and update…

76673e2

… type hints

add tests for GrobidServiceError handling

24c0f84

Sanakhamassi requested a review from lfoppiano May 8, 2026 19:06

lfoppiano requested a review from Copilot May 8, 2026 20:57

Copilot started reviewing on behalf of lfoppiano May 8, 2026 20:58 View session

Copilot AI reviewed May 8, 2026

View reviewed changes

lfoppiano mentioned this pull request Jun 7, 2026

Handle grobid connection failures #43

Merged

lfoppiano mentioned this pull request Jun 7, 2026

Doc/add documentation on the document qa engine library #36

Merged

lfoppiano added a commit that referenced this pull request Jun 7, 2026

Handle grobid connection failures (#43)

0d0843f

This PR extended the work from @Sanakhamassi in #35

lfoppiano closed this Jun 7, 2026

Conversation

Sanakhamassi commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lfoppiano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

lfoppiano May 6, 2026

Choose a reason for hiding this comment

Uh oh!

lfoppiano May 6, 2026

Choose a reason for hiding this comment

Uh oh!

lfoppiano left a comment

Choose a reason for hiding this comment

Uh oh!

lfoppiano May 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lfoppiano May 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

lfoppiano commented Jun 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sanakhamassi commented Apr 23, 2026 •

edited

Loading