Streamline interview handoff review and continuation

docs/developer-notes/issue-166-handoff-ux.md

@@ -0,0 +1,51 @@
+# Streamlining the RFA interview-to-filing flow
+
+Reviewed on October 22, 2026 for [issue 166](https://github.com/SuffolkLITLab/LITEfile/issues/166). The RFA changes build on the still-open [sender PR 426](https://github.com/LSVermont/docassemble-RFApackage/pull/426), branch `feature/litefile-handoff`. The shared sender now lives in `SuffolkLITLab/docassemble-LITEfile`, pinned by RFA to v0.1.0.
+
+## Changes in this pass
+
+- Put the next action above the imported information. Group missing fields into one task per destination screen.
+- Keep coordinated case and filing choices editable in an optional details section; reserve correction confirmation for actual correction requirements.
+- Approve the transferred PDF copies on the handoff page using the existing document fingerprint check. Previously final review redirected the user back through the case-information form solely because those PDFs were not marked reviewed.
+- Match ordinary document readiness: honor an explicitly confirmed empty confidentiality choice, and leave filing-component validation to the existing final payload checks. Do not supply new confidentiality defaults. Clerk-requested corrections still block continuation.
+- Skip repeated account claiming for an already-owned draft. Use “Your interview” instead of an internal source identifier.
+- Add a direct link to the submission section while keeping court terms, the attestation, and the submit action together.
+- In RFA, put electronic filing first when enabled; replace download-only introductory and completion text only in that mode. Keep the manual filing instructions when disabled. Prefill service-question city and state from the previously supplied defendant address; leave both editable.
+
+## Independent reviews
+
+Requested read-only reviews from `agy --model gemini-4-argon-high --mode plan` and Claude CLI in plan mode. Both completed. Argon traced the original flow; Claude also inspected the in-progress edits. These were source reviews, not independent live staging sessions.
+
+Both identified the duplicate case-confirmation loop, buried actions, outdated shared Vermont text, and optional document-setting inconsistencies. Claude also caught that returning to the interview updates PDFs without synchronizing the saved filing metadata. The action now says “Fix a PDF in my interview” and explains that changed names, addresses, or court details must also be edited in LITEfile. Model findings were checked against the implementation; proposed blanket PDF approval and unverified confidentiality defaults were not adopted.
+
+## Proposed next improvements
+
+1. **Synchronize interview corrections:** design a revision protocol for changed names, addresses, county, and added/removed forms. The current replacement endpoint deliberately updates only known PDFs. Do this before promising an unrestricted “Edit answers” round trip.
+2. **Verify confidential contact handling:** RFA asks `share_contact`, while its sender exports contact fields to filer/party records. Check which fields the target EFM exposes and represent the interview's privacy choice explicitly. Do not treat PDF confidentiality as proof that party metadata is confidential.
+3. **Shorten the shared sender transition:** consider continuing directly after a successful transfer, while retaining the retry/recovery screen on failure. This belongs in `docassemble-LITEfile`, with a package release and RFA dependency update.
+4. **Reuse account email carefully:** a missing interview email currently opens a form that defaults to the account email. Consider using that known value earlier while preserving an obvious way to choose a safe contact address.
+5. **Unify filing instructions:** review the RFA NEXT STEPS document alongside the new online path, so users understand when they have already submitted and what remains to do. Do not change legal or emergency-filing instructions without partner review.
+6. **Resolve only verified choices:** add maintained semantic aliases or single required-component handling where live metadata supports them. Leave ambiguous matches and confidentiality decisions visible; do not guess that ordinary RFA forms should be public.
+
+## Validation and limits
+
+- Handoff, final-review, and PDF-preview tests: 89 passed before the optional-settings refinement; final handoff suite: 43 passed. Ruff and Ty pass.
+- Parsed the RFA YAML and rendered its introduction/download Mako templates with synthetic helpers in both enabled and disabled modes. Confirmed the enabled mode omits the contradictory manual-only instructions and puts the filing action before downloads.
+- Chromium at 390 × 844: ready/missing handoff primary actions are above the fold; the saved-details disclosure opens and closes with Enter. [Synthetic screenshots](https://gist.github.com/nonprofittechy/841e3dd9efb519a3cc91e481aa1f1a40).
+- These checks do not establish a successful Vermont test filing. Full Docassemble execution, test-EFM submission, court receipt/status, and Vermont partner acceptance remain outstanding under issue 166.
+
+## Independent code review and follow-up
+
+Claude and agy/Argon reviewed the full implementation through `0a10c55` independently. Their raw reports are retained in the unlisted evidence gist; some report line numbers and symbol names were inaccurate, so findings were checked against the source and regression tests.
+
+Confirmed issues addressed in the follow-up:
+
+- Restrict the focused replacement lookup to new cases. Existing-case drafts keep the court-record lookup and case-specific permitted filing types; drafts with an undecided case stage no longer show unusable replacement links.
+- Resume matching of dependent interview hints after a successful replacement. Keep metadata matching out of PDF approval actions so metadata-only timestamp changes cannot invalidate the displayed fingerprint.
+- Approve the displayed PDF copies when continuing to missing details, avoiding duplicate review at payment; stale fingerprints still block approval.
+- Reset blank confidentiality confirmation when its filing-type scope changes, including clerk corrections. Carry the live amount-in-controversy requirement into document metadata.
+- Preserve all saved codes, refresh conflict results from current records, and recover when a document disappears during live validation. Write one provenance event per selected replacement.
+- Seed lookup from one relevant hint rather than combining alternative names into an AND query; preserve the lead document's top-level hint fallback. Surface unmatched filing and party-role hints outside the accordion.
+- Display unavailable results with their explanation and no selection control, with a link to later result pages.
+
+Validation uses synthetic court responses. It does not establish a live Vermont EFM acceptance test. The focused replacement route does not replace the existing party-role or confidentiality editors.

efile_app/efile/services/filing_code_search.py

@@ -426,6 +426,8 @@ def get(self, suffix="", **params):
             if not code:
                 raise ValueError("Court code API returned an empty code")
             result[code] = {"code": code, "name": str(item["name"])}
+            if "amountincontroversy" in item:
+                result[code]["amountincontroversy"] = item["amountincontroversy"]
         return list(result.values())
 
     def courts(self):
@@ -920,6 +922,7 @@ def search_paths(
     courts=(),
     category=None,
     case_type=None,
+    filing_type=None,
     include_total=True,
     include_details=True,
 ):
@@ -935,6 +938,8 @@ def search_paths(
         paths = paths.filter(case_category__code=category)
     if case_type:
         paths = paths.filter(case_type__code=case_type)
+    if filing_type:
+        paths = paths.filter(filing_type__code=filing_type)
     if contexts is not None:
         paths = paths.filter(context_filter(contexts))
     total = paths.count() if include_total else None

efile_app/efile/services/handoff.py

@@ -144,6 +144,33 @@ def effective_hints(payload, *, court_name="", document=None):
     return result
 
 
+def interview_filing_hints(draft, payload, documents):
+    """Each document's interview filing-type hints by pk; the lead falls back to the case's."""
+    sources = {}
+    for event in draft.metadata_events.filter(
+        kind="source_suggestion", path__in=[f"documents.{document.pk}" for document in documents]
+    ):
+        sources.setdefault(event.path, event.value)
+    case_hints = effective_hints(payload, court_name=draft.court_name).get("filing_type_name_hints", [])
+    return {
+        document.pk: effective_hints(
+            payload, court_name=draft.court_name, document=sources.get(f"documents.{document.pk}", {})
+        ).get("filing_type_name_hints", [])
+        or (case_hints if document.role == "lead" else [])
+        for document in documents
+    }
+
+
+def apply_filing_type_flags(document, option):
+    """Copy the court's per-filing-type flags onto a newly chosen filing type; return fields to save."""
+    document.filing_requires_amount_in_controversy = str(option.get("amountincontroversy", "")).lower() == "required"
+    update_fields = ["filing_requires_amount_in_controversy"]
+    if not document.document_type_code:
+        document.document_type_confirmed = False
+        update_fields.append("document_type_confirmed")
+    return update_fields
+
+
 def validate_payload(payload, source_config, files, *, require_lead=True):
     _object(payload, "payload")
     if type(payload.get("schema_version")) is not int or payload["schema_version"] != 1:
@@ -380,7 +407,10 @@ def choose(obj, field, options, hints):
             name_field = field.replace("_code", "_name") if field != "party_type" else "party_type_name"
             setattr(obj, name_field, str(option.get("name", "")))
             obj._metadata_kind = "live_resolution"
-            obj.save(update_fields=[field, name_field, "updated_at"])
+            update_fields = [field, name_field, "updated_at"]
+            if field == "filing_type_code" and isinstance(obj, FilingDocument):
+                update_fields += apply_filing_type_flags(obj, option)
+            obj.save(update_fields=update_fields)
             del obj._metadata_kind
             record(
                 draft,
@@ -507,17 +537,27 @@ def need(path, value, message, view):
     for doc in draft.documents.all():
         for field, label in (
             ("filing_type_code", "filing type"),
-            ("document_type_code", "document type"),
-            ("filing_component_code", "filing component"),
+            ("document_type_code", "confidentiality setting"),
+            ("filing_component_code", "document settings"),
         ):
             if (
                 current_metadata
                 and field != "filing_type_code"
                 and f"documents.{doc.pk}.{field}" not in draft.correction_fields
             ):
                 continue
+            path = f"documents.{doc.pk}.{field}"
+            if path not in draft.correction_fields:
+                # Match the ordinary workflow's document readiness rules. The
+                # document editor confirms an empty confidentiality choice when
+                # the court offers none; components are validated by the final
+                # payload checks, not a mandatory question for every filer.
+                if field == "filing_component_code":
+                    continue
+                if field == "document_type_code" and doc.filing_type_code and doc.document_type_confirmed:
+                    continue
             need(
-                f"documents.{doc.pk}.{field}",
+                path,
                 getattr(doc, field),
                 f"Choose the {label} for {doc.name}.",
                 "organize_documents",
@@ -665,6 +705,10 @@ def create_correction(draft, detail, fields=()):
                 obj.filing_component_code = ""
                 obj.filing_component_name = ""
                 obj.requested_optional_services = []
+            if parts[0] == "documents" and parts[2] in ("filing_type_code", "document_type_code"):
+                obj.document_type_confirmed = False
+                if parts[2] == "filing_type_code":
+                    obj.filing_requires_amount_in_controversy = False
             obj.save()
         elif path != "documents":
             setattr(revision, path, "")
@@ -683,6 +727,8 @@ def create_correction(draft, detail, fields=()):
                 setattr(document, f"{field}_code", "")
                 setattr(document, f"{field}_name", "")
             document.requested_optional_services = []
+            document.document_type_confirmed = False
+            document.filing_requires_amount_in_controversy = False
             document.save()
         for party in revision.parties.all():
             party._metadata_kind = "clerk_correction"

efile_app/efile/services/handoff_replacements.py

@@ -0,0 +1,68 @@
+"""Explore indexed filing paths while preserving every saved classification."""
+
+from django.db import transaction
+
+from efile.models import FilingDocument, FilingDraft
+from efile.services.filing_code_search import FACETS, validate_path
+from efile.services.handoff import HandoffError, apply_filing_type_flags, record
+from efile.workflow import ExistingCase
+
+
+def locked_choices(draft, document):
+    return {facet: getattr(document if facet == "filing_type" else draft, f"{facet}_code") for facet in FACETS}
+
+
+def lookup_offered(draft, document):
+    """New cases only, and only while some court choice is still missing."""
+    return draft.existing_case == ExistingCase.NEW and not all(locked_choices(draft, document).values())
+
+
+def compatible(draft, document, path):
+    return path["initial"] == (draft.existing_case == ExistingCase.NEW) and all(
+        not code or str(path[facet]["code"]) == code for facet, code in locked_choices(draft, document).items()
+    )
+
+
+def apply_replacement(draft, document, path):
+    """Validate live first, then recheck locked values before filling only blanks."""
+    if draft.existing_case != ExistingCase.NEW:
+        raise HandoffError("Use the confirmed case’s filing choices for an existing case.", status=409)
+    validated = validate_path(path)
+    with transaction.atomic():
+        try:
+            current = FilingDraft.objects.select_for_update().get(pk=draft.pk, user=draft.user)
+            target = current.documents.select_for_update().get(pk=document.pk)
+        except (FilingDraft.DoesNotExist, FilingDocument.DoesNotExist) as exc:
+            raise HandoffError(
+                "Your documents changed. Return to your saved filing and try again.", status=409
+            ) from exc
+        if current.status != FilingDraft.Status.DRAFT or current.deletion_pending:
+            raise HandoffError("This draft is no longer editable.", status=409)
+        if (
+            current.existing_case != ExistingCase.NEW
+            or locked_choices(current, target) != locked_choices(draft, document)
+            or not compatible(current, target, validated)
+        ):
+            raise HandoffError("Your saved choices changed. Search again; nothing was replaced.", status=409)
+        corrections = list(current.correction_fields)
+        for facet in FACETS:
+            obj = target if facet == "filing_type" else current
+            field = f"{facet}_code"
+            if getattr(obj, field):
+                continue
+            setattr(obj, field, str(validated[facet]["code"]))
+            setattr(obj, f"{facet}_name", validated[facet]["name"])
+            update_fields = [field, f"{facet}_name", "updated_at"]
+            if obj is target:
+                update_fields += apply_filing_type_flags(target, validated[facet])
+            obj._metadata_kind = "code_search_replacement"
+            try:
+                obj.save(update_fields=update_fields)
+            finally:
+                del obj._metadata_kind
+            event_path = f"documents.{target.pk}.{field}" if obj is target else field
+            record(current, event_path, "user_confirmation", {**validated[facet], "source": "code search"})
+            if event_path in corrections:
+                corrections.remove(event_path)
+        if corrections != current.correction_fields:
+            FilingDraft.objects.filter(pk=current.pk).update(correction_fields=corrections)

efile_app/efile/static/css/handoff-review.css

@@ -0,0 +1,33 @@
+.handoff-details {
+    width: 100%;
+    table-layout: fixed;
+    margin-bottom: 1.5rem;
+}
+
+.handoff-details th,
+.handoff-details td {
+    padding: 0.75rem;
+    vertical-align: top;
+    overflow-wrap: anywhere;
+}
+
+.handoff-details th {
+    width: 34%;
+    padding-left: 0;
+}
+
+.handoff-details td {
+    padding-right: 0;
+}
+
+.handoff-details p {
+    margin: 0;
+}
+
+.handoff-details p+p {
+    margin-top: 0.5rem;
+}
+
+.handoff-details .handoff-hint {
+    color: #545454;
+}

efile_app/efile/templates/efile/handoff_claim.html

@@ -9,7 +9,7 @@ <h1>This draft belongs to another account</h1>
         <p>Sign in with the account you used to claim the draft.</p>
     {% else %}
         <h1>Continue your filing</h1>
-        <p>{{ source }} sent your answers and documents. Add them to your account to continue filing.</p>
+        <p>Your interview sent your answers and documents. Add them to your account to continue filing.</p>
         <form method="post">{% csrf_token %}<button class="btn btn-primary" type="submit">Add this draft to my account</button></form>
     {% endif %}
 {% endblock public_content %}