Implement German compound word tokenizer and fix head word capitalization

[nciric]

This PR implements the German compound word tokenizer (DeTokenizer) and integrates it into the inflection engine. It also resolves capitalization lookup issues for lowercased head-word segments resulting from compound splitting.

Key Changes

1. German Tokenizer Infrastructure & Configuration (DeTokenizer)

• Implemented DeTokenizer and DeDictionaryTokenizerConfig registered in TokenizerFactory.cpp.
• Configured German to use the new tokenizer and compiled binary dictionary (/de/tokenizer.tokd) in config_de.properties.
• Configured German Fugenelemente (s, es, e, n, en, er, ens) in DeDictionaryTokenizerConfig.

2. German Head-Word Capitalization Lookup

• Lowercased compound head-word segments (e.g. frau from Ehefrau) are checked against capitalized noun entries (Frau) in dictionary.getCombinedBinaryType to resolve noun POS and feminine gender morphology, restoring lower-case formatting on inflectedHeadWord after inflection synthesis.

3. German Tokenizer Dictionary

• Added tokenizer.dictionary for German containing word frequencies and flags (isNoCompound, isNoAtomic, isSegment).

Verification

• Added unit tests in TokenizerTest.cpp covering German compound splitting (Hauptstraße, Waldweg, Schlossgasse, Marktplatz, Kaiserring, Sonnenallee, Arbeitszimmer, Kindergarten).
• Executed make check - all 264 test cases passed (290,502 assertions).

[nciric]

Technical Investigation: Case Sensitivity in DictionaryMetaData Lookup

Following review discussion regarding case sensitivity during dictionary lookup, we investigated how DictionaryMetaData::getCombinedBinaryType and underlying Trie lookups behave in the engine:

1. Trie Byte Matching

Dictionary entries in the binary dictionary (e.g. Frau, Weg, Strasse) are stored in a marisa::Trie (MarisaTrie.hpp), which performs exact byte-sequence lookups.

2. One-Way Lowercasing Fallback in getCombinedBinaryType

In DictionaryMetaData::getCombinedBinaryType (DictionaryMetaData.cpp):

int64_t* DictionaryMetaData::getCombinedBinaryType(int64_t* result, std::u16string_view word) const
{
    *npc(result) = 0;
    auto combinedType = dictionary->getWordType(word);
    if (!combinedType) {
        ::std::u16string normalized;
        transform(&normalized, word, dictionary->getLocale()); // Lowercases word
        if (normalized != word) {
            combinedType = dictionary->getWordType(normalized);
        }
    }
    if (!combinedType) {
        return nullptr;
    }
    *npc(result) = *combinedType;
    return result;
}

• Uppercase/Capitalized inputs (Frau / FRAU): First checks exact match Frau, then falls back to lowercasing frau.
• Lower-case inputs (frau): Searches frau (fails, as entry is stored as Frau), then transform lowercases frau -> frau (normalized == word), returning nullptr.

Conclusion for German Compound Tokenization

When a compound noun like Ehefrau is split into Ehe + frau, the head segment frau is lowercased. Because getCombinedBinaryType does not perform capitalization fallback for lowercased inputs, capitalizing frau -> Frau before getCombinedBinaryType is necessary for frau to resolve noun properties (dictionaryNoun | dictionaryFeminine) from capitalized dictionary entries.