LLM simplifies citations for filenames with special characters, breaking document links

[pamelafox]

Description

When uploading documents via the user upload feature (with USE_USER_UPLOAD=true and AZURE_ENFORCE_ACCESS_CONTROL=true), the sourcefile field in the Azure AI Search index gets a "- " prefix that shouldn't be there.

Steps to Reproduce

1. Enable user upload and access control:

   azd env set USE_USER_UPLOAD true
   azd env set AZURE_ENFORCE_ACCESS_CONTROL true
   azd provision && azd deploy
   

2. Log in and upload a document (e.g., PyCon US 2025.pdf)
3. Query the search index to inspect the indexed document

Expected Behavior

The sourcefile field should match the original filename:

sourcefile: "PyCon US 2025.pdf"

Actual Behavior

The sourcefile field has a "- " prefix:

sourcefile: "- PyCon US 2025.pdf"

Impact

This causes the /content/<path> route to return 403 Forbidden when trying to view the document, because check_path_auth() filters by sourcefile eq 'PyCon US 2025.pdf' but the indexed value is '- PyCon US 2025.pdf'.

Environment

• User upload enabled with ADLS Gen2
• Access control enforcement enabled
• Local development (may also affect deployed environments)

[pamelafox]

Additional data point: filenames with parentheses also appear to be affected.

Example filename:

(Poster) Online visualization of recursive Python functions (1).pdf

The "- " prefix may be related to how special characters in filenames are being processed during ingestion.

[pamelafox]

Investigation Findings

After investigation, the issue is more nuanced than initially thought:

1. The sourcefile is stored correctly in the index with the "- " prefix - this is the actual filename as it exists in blob storage
2. The /list_uploaded endpoint returns the correct filename
3. The real problem is LLM citation simplification - when the LLM generates citations, it simplifies filenames with special characters

Root Cause

The LLM does not like generating citations for filenames with "funny" characters like a leading hyphen. When given sources like "- PyCon US 2025.pdf#page=1", the LLM simplifies the citation to [PyCon US 2025.pdf#page=1], removing the leading "- ".

This causes a mismatch when the frontend tries to fetch the document content via /content/PyCon US 2025.pdf - the check_path_auth() function filters by sourcefile eq 'PyCon US 2025.pdf' but the indexed value is '- PyCon US 2025.pdf'.

Example Response JSON

{
    "context": {
        "data_points": {
            "citations": [
                "- PyCon US 2025.pdf#page=2",
                "- PyCon US 2025.pdf#page=1"
            ],
            "text": [
                "- PyCon US 2025.pdf#page=2: Review the PyCon US Code of Conduct...",
                "- PyCon US 2025.pdf#page=1: This invoice has been issued..."
            ]
        },
        "thoughts": [
            {
                "description": [...],
                "title": "Search results"
            }
        ]
    },
    "message": {
        "content": "PyCon US 2025 requires all staff... [PyCon US 2025.pdf#page=2].\n\nAn example invoice... [PyCon US 2025.pdf#page=1].",
        "role": "assistant"
    }
}

Notice how:

• citations array has "- PyCon US 2025.pdf#page=2" (correct)
• message.content has [PyCon US 2025.pdf#page=1] (simplified by LLM - missing the "- " prefix)

Potential Solutions

1. Auto-heal citations - Post-process LLM response to match simplified citations to valid citations from search results (e.g., if [PyCon US 2025.pdf#page=1] does not match any citation, check if any valid citation ends with that string)

2. Wait for LLM improvements - Future models may handle special characters in citations better

3. Sanitize filenames on upload - Strip or replace leading special characters when storing files, though this could cause other issues with filename matching