[datlechin]

Summary

• Safe Mode and read-only connections on Databend tiered a pasted DROP as a SELECT when it sat behind an optimizer hint holding a quote, a stage name, $$ or another hint, or, on servers before 1.2.884, behind a comment such as /** x **/. The server read the hint's body as tokens and ran the DROP.
• The same gates tiered SETTINGS (...) DROP TABLE t and CREATE OR REPLACE TABLE中 (...) as plain writes. Databend runs the statement under SETTINGS, and from 1.2.888 ends a keyword where a non-ASCII name starts.
• DatabendTokenizer in TableProSQLGrammar reads text the way Databend's tokenizer does, with each rule a release since 1.2.473 changed. The parameter binder and the statement gate both use it.
• The gate keeps its old reading and adds the server's: the tokens of each statement the app sends, and of the whole text, under every release's rules, less a statement the server refuses to tokenize. The worst tier wins.
• StarRocks parameter binding: a form feed in a value is written as itself, since StarRocks reads \f as f. A -- comment ends at a carriage return. A $$ inside a name or a number opens nothing, and nothing after a $$ the statement leaves open is bound.

Test plan

• apple/scripts/probes/check-databend-lexing.sh passes against Databend 1.2.880 and 1.2.958 in Docker, now with each text above that dropped or replaced a table.
• apple/scripts/probes/check-starrocks-literals.sh passes against StarRocks 4.1.7 in Docker.
• New: DatabendTokenizerTests, DatabendStatementReadingsTests (TableProSQLGrammarTests), QueryClassifierLexicalTests.databendSkippedHintDoesNotHideADrop, StarRocksLiteralTests (form feed, round trip through a port of StarRocks' reader, carriage return, $$ in names and attachments).
• Changed on purpose: SQLLexicalFeatureMappingTests, since Databend's grammar gains a fact only the app's table sets.
• DatabendLiteralTests pass unchanged on the shared tokenizer.