fix: strip erroneous '#kv#' and '#nv#' prefixes from AHB XML attributes during reading (#255)

* chore: bump test data submodule to 2026-03-27 data

including fun like
```xml
  <AWF
    Pruefidentifikator="#kv# 55038"
    Beschreibung="#kv# Info-Meldung zur Aufhebung einer zuk. Zuordnung"
    Kommunikation_von="NB an LF"
  >
```

* fix: strip erroneous '#kv# ' prefix from AHB XML attributes during reading

The XML authors introduced random "#kv# " prefixes to several attribute
values in the test data submodule. This commit strips these prefixes at
read time in the AhbReader so they never leak into the data model.

Affected XML files and lines (50 total occurrences across 3 files):

UTILMD_AHB_Strom_2_1_Fehlerkorrektur_20260327.xml (FV2604, 24 occurrences):
  - Pruefidentifikator: lines 7791, 8189, 8532
  - Beschreibung: lines 7792, 8190, 8533, 9803, 10117, 10993, 11856,
    12770, 13678, 16282, 16572, 19584, 20069, 20593, 21154, 21760
  - Bedingung text: lines 41302, 41369, 41413, 41705, 41724

UTILMD_AHB_Strom_2_1_Fehlerkorrektur_20260327.xml (FV2510, 24 occurrences):
  - Pruefidentifikator: lines 7791, 8189, 8532
  - Beschreibung: lines 7792, 8190, 8533, 9803, 10117, 10993, 11856,
    12770, 13678, 16282, 16572, 19584, 20069, 20593, 21154, 21760
  - Bedingung text: lines 41302, 41369, 41413, 41705, 41724

UTILMD_AHB_Strom_2_1_Fehlerkorrektur_20250623.xml (FV2504, 2 occurrences):
  - Bedingung text: lines 131688, 131694

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: apply black formatting and extend kv prefix stripping to ub_bedingungen

Apply remove_kv_prefix defensively to _to_ub_bedingung as well, and
extend the integration test to also check ub_bedingungen for leaking
prefixes. Fix black formatting in test file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: generalize hashtag prefix stripping to handle #nv# in addition to #kv#

Rename remove_kv_prefix to remove_hashtag_prefix and use a regex
(^#\w+# ) to strip any leading '#xx# ' prefix, not just '#kv# '.

Additional '#nv# ' occurrences found in Beschreibung attributes:

UTILMD_AHB_Gas_1_0a_außerordentliche_20240726.xml (FV2504, 2 occurrences):
  - Beschreibung: line 22762: `Beschreibung="#nv# Anfrage an MSB mit Abhängig-keiten "`
  - Beschreibung: line 43668: `Beschreibung="#nv# Nicht bila.rel. Änderung vom LF "`

UTILMD_AHB_Gas_1_0a_außerordentliche_20240726.xml (FV2410, 2 occurrences):
  - Beschreibung: line 22762: `Beschreibung="#nv# Anfrage an MSB mit Abhängig-keiten "`
  - Beschreibung: line 43668: `Beschreibung="#nv# Nicht bila.rel. Änderung vom LF "`

UTILMD_AHB_Gas_1_0a_außerordentliche_20240726.xml (FV2510, 2 occurrences):
  - Beschreibung: line 22762: `Beschreibung="#nv# Anfrage an MSB mit Abhängig-keiten "`
  - Beschreibung: line 43668: `Beschreibung="#nv# Nicht bila.rel. Änderung vom LF "`

UTILMD_AHB_Gas_1_1_Fehlerkorrektur_20260327.xml (FV2604, 2 occurrences):
  - Beschreibung: line 22854: `Beschreibung="#nv# Anfrage an MSB mit Abhängig-keiten "`
  - Beschreibung: line 43760: `Beschreibung="#nv# Nicht bila.rel. Änderung vom LF "`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: clarify that remove_hashtag_prefix does not strip ##alt## markers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add cases ensuring ##alt## and ##veraltet## markers are not stripped

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Konstantin <konstantin.klein+github@hochfrequenz.de>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: 3 people

src/fundamend/reader/ahbreader.py

@@ -31,7 +31,13 @@
     _is_segment_group,
     _is_uebertragungsdatei,
 )
-from fundamend.utils import lstrip, remove_linebreaks_and_hyphens, remove_unnecessary_hyphens, strip
+from fundamend.utils import (
+    lstrip,
+    remove_hashtag_prefix,
+    remove_linebreaks_and_hyphens,
+    remove_unnecessary_hyphens,
+    strip,
+)
 
 # pylint:disable=duplicate-code
 # yes, it's very similar to the MigReader
@@ -53,14 +59,14 @@ def _to_code(element: ET.Element) -> Code:
 def _to_bedingung(element: ET.Element) -> Bedingung:
     return Bedingung(
         nummer=strip("[", element.attrib["Nummer"], "]"),
-        text=(element.text or "").strip(),
+        text=remove_hashtag_prefix((element.text or "").strip()),
     )
 
 
 def _to_ub_bedingung(element: ET.Element) -> UbBedingung:
     return UbBedingung(
         nummer=strip("[", element.attrib["Nummer"], "]"),
-        text=(element.text or "").strip(),
+        text=remove_hashtag_prefix((element.text or "").strip()),
     )
 
 
@@ -208,10 +214,10 @@ def get_anwendungsfall(self, pruefidentifikator: str) -> Anwendungsfall | None:
         for element in self._element_tree.getroot():
             if element.tag != "AWF":
                 continue
-            if element.attrib["Pruefidentifikator"] != pruefidentifikator:
+            raw_pruefi = remove_hashtag_prefix(element.attrib["Pruefidentifikator"]).strip()
+            if raw_pruefi != pruefidentifikator:
                 continue
-            if element.tag == "AWF" and element.attrib["Pruefidentifikator"] == pruefidentifikator:
-                return self._read_anwendungsfall(element)
+            return self._read_anwendungsfall(element)
         return None
 
     def get_anwendungsfaelle(self) -> list[Anwendungsfall]:
@@ -251,9 +257,9 @@ def _read_anwendungsfall(self, original_element: ET.Element) -> Anwendungsfall:
         if not format_element.tag.startswith("M_"):
             format_element = next((child for child in original_element[0] if child.tag.startswith("M_")))
         return Anwendungsfall(
-            pruefidentifikator=original_element.attrib["Pruefidentifikator"],
+            pruefidentifikator=remove_hashtag_prefix(original_element.attrib["Pruefidentifikator"]).strip(),
             beschreibung=remove_unnecessary_hyphens(
-                remove_linebreaks_and_hyphens(original_element.attrib["Beschreibung"])
+                remove_linebreaks_and_hyphens(remove_hashtag_prefix(original_element.attrib["Beschreibung"]))
             ),
             kommunikation_von=original_element.attrib["Kommunikation_von"].strip(),
             format=EdifactFormat(lstrip("M_", format_element.tag)),

src/fundamend/utils.py

@@ -162,11 +162,25 @@ def remove_unnecessary_hyphens(candidate: Optional[str]) -> Optional[str]:
     return _unnecessary_hyphen_pattern.sub("", candidate)
 
 
+_HASHTAG_PREFIX_PATTERN = re.compile(r"^#\w+# ")
+"""matches leading prefixes like '#kv# ', '#nv# ' etc. that XML authors erroneously add to attribute values"""
+
+
+def remove_hashtag_prefix(text: str) -> str:
+    """
+    Removes leading '#xx# ' prefixes (e.g. '#kv# ', '#nv# ') that some XML authors erroneously add to attribute values.
+    The pattern (^#\\w+# ) intentionally does NOT match double-hash markers like '##alt##' or '##veraltet##'
+    because those are used by Anwendungsfall.is_outdated to flag outdated entries.
+    """
+    return _HASHTAG_PREFIX_PATTERN.sub("", text)
+
+
 __all__ = [
     "lstrip",
     "rstrip",
     "strip",
     "parse_kommunikation_von",
+    "remove_hashtag_prefix",
     "remove_linebreaks_and_hyphens",
     "remove_unnecessary_hyphens",
 ]

unittests/__snapshots__/test_ahbreader.ambr

@@ -11,8 +11,6 @@
 # name: test_sanitizing_all_awf_beschreibungen
   list([
     '',
-    '#nv# Anfrage an MSB mit Abhängigkeiten',
-    '#nv# Nicht bila.rel. Änderung vom LF',
     '(Ankündigung) Zuordnung des LF zur erz. MaLo/ Tranche',
     'Ab-/Bestellung BK-SZR auf Aggregationsebene RZ',
     'Abbestellung von Werten',
@@ -274,6 +272,9 @@
     'Gerätestatus',
     'Grundlage POG-Ermittlung',
     'Info Entsperrauftrag',
+    'Info-Meldung zur Aufhebung einer zuk. Zuordnung',
+    'Info-Meldung zur Beendigung der Zuordnung',
+    'Info-Meldung über existierende Zuordnung',
     'Informationsmeldung',
     'Informationsmeldung zur Aufhebung einer zuk. Zuordnung',
     'Informationsmeldung zur Beendigung der Zuordnung',

unittests/test_utils.py

@@ -1,3 +1,4 @@
+import re
 from pathlib import Path
 from typing import Generator
 
@@ -6,7 +7,7 @@
 from fundamend import AhbReader
 from fundamend.models.anwendungshandbuch import Anwendungsfall
 from fundamend.models.kommunikationsrichtung import Kommunikationsrichtung
-from fundamend.utils import parse_kommunikation_von, remove_linebreaks_and_hyphens
+from fundamend.utils import parse_kommunikation_von, remove_hashtag_prefix, remove_linebreaks_and_hyphens
 
 from .conftest import is_private_submodule_checked_out
 
@@ -128,6 +129,56 @@ def test_parsing_all_kommunikation_von_there_is() -> None:
         _ = parse_kommunikation_von(kommunikation_von)  # must not crash
 
 
+_hashtag_prefix_pattern = re.compile(r"^#\w+# ")
+
+
+@pytest.mark.parametrize(
+    "original, expected",
+    [
+        pytest.param("hello", "hello", id="no prefix"),
+        pytest.param("#kv# 55038", "55038", id="pruefidentifikator with #kv# prefix"),
+        pytest.param(
+            "#kv# Info-Meldung zur Aufhebung", "Info-Meldung zur Aufhebung", id="beschreibung with #kv# prefix"
+        ),
+        pytest.param("#kv# Wenn die Messlokation", "Wenn die Messlokation", id="bedingung text with #kv# prefix"),
+        pytest.param("#nv# Anfrage an MSB mit Abhängigkeiten", "Anfrage an MSB mit Abhängigkeiten", id="#nv# prefix"),
+        pytest.param("#nv# Nicht bila.rel. Änderung vom LF", "Nicht bila.rel. Änderung vom LF", id="#nv# prefix 2"),
+        pytest.param("#kv#no space", "#kv#no space", id="hashtag prefix without trailing space is not stripped"),
+        pytest.param("##alt##", "##alt##", id="##alt## marker must not be stripped (used by is_outdated)"),
+        pytest.param("##alt## 55001", "##alt## 55001", id="##alt## with pruefidentifikator must not be stripped"),
+        pytest.param("##veraltet##", "##veraltet##", id="##veraltet## marker must not be stripped"),
+        pytest.param("", "", id="empty string"),
+    ],
+)
+def test_remove_hashtag_prefix(original: str, expected: str) -> None:
+    actual = remove_hashtag_prefix(original)
+    assert actual == expected
+
+
+def test_no_hashtag_prefix_leaks_through_ahb_reader() -> None:
+    """Ensures that '#xx# ' prefixes introduced by XML authors are stripped during reading."""
+    if not is_private_submodule_checked_out():
+        pytest.skip("Skipping test because of missing private submodule")
+    private_submodule_root = Path(__file__).parent.parent / "xml-migs-and-ahbs"
+    for ahb_file_path in private_submodule_root.rglob("**/*AHB*.xml"):
+        ahb = AhbReader(ahb_file_path).read()
+        for awf in ahb.anwendungsfaelle:
+            assert not _hashtag_prefix_pattern.match(
+                awf.pruefidentifikator
+            ), f"pruefidentifikator '{awf.pruefidentifikator}' in {ahb_file_path} still has a hashtag prefix"
+            assert not _hashtag_prefix_pattern.match(
+                awf.beschreibung
+            ), f"beschreibung '{awf.beschreibung}' in {ahb_file_path} still has a hashtag prefix"
+        for bedingung in ahb.bedingungen:
+            assert not _hashtag_prefix_pattern.match(
+                bedingung.text
+            ), f"bedingung '{bedingung.nummer}' text in {ahb_file_path} still has a hashtag prefix"
+        for ub_bedingung in ahb.ub_bedingungen:
+            assert not _hashtag_prefix_pattern.match(
+                ub_bedingung.text
+            ), f"ub_bedingung '{ub_bedingung.nummer}' text in {ahb_file_path} still has a hashtag prefix"
+
+
 @pytest.mark.parametrize(
     "original, expected",
     [