Skip to content

fix(consensus): use Locale.ROOT for case-insensitive#127

Open
halibobo1205 wants to merge 3 commits intodevelopfrom
fix/locale-sensitive-case-folding
Open

fix(consensus): use Locale.ROOT for case-insensitive#127
halibobo1205 wants to merge 3 commits intodevelopfrom
fix/locale-sensitive-case-folding

Conversation

@halibobo1205
Copy link
Copy Markdown
Owner

@halibobo1205 halibobo1205 commented Apr 12, 2026

User description

String.toLowerCase()/toUpperCase() without an explicit Locale uses Locale.getDefault(), which on Turkish (tr) or Azerbaijani (az) systems folds 'I' to dotless-ı (U+0131) instead of 'i' (U+0069). This caused AccountIdIndexStore to generate different index keys on tr/az nodes, enabling a consensus split where tr/az nodes accept duplicate accountId transactions that other nodes reject.

Fix AccountIdIndexStore.getLowerCaseAccountId to use Locale.ROOT, and add a legacy fallback in get()/has() so that tr/az nodes can still read old data stored with the locale-sensitive key.

Also fix all other toLowerCase()/toUpperCase() calls across the codebase and enable the built-in ErrorProne StringCaseLocaleUsage checker at ERROR level to prevent future regressions at compile time.

What does this PR do?

Why are these changes required?

This PR has been tested by:

  • Unit Tests
  • Manual Testing

Follow up

Extra details


CodeAnt-AI Description

Use locale-independent case handling across nodes and migrate old Turkish keys

What Changed

  • Case-insensitive matching now uses the same rules on every node, avoiding different results on Turkish and Azerbaijani systems
  • Old account ID index entries written with Turkish casing are converted to the standard form at startup so existing accounts still resolve after upgrade
  • Disabled API checks, wallet paths, command handling, account type parsing, and several hex/text displays now normalize case consistently
  • Added checks to catch future locale-sensitive case usage, plus tests covering Turkish key behavior and the migration path

Impact

✅ Fewer consensus splits on Turkish and Azerbaijani nodes
✅ Existing account IDs remain readable after upgrade
✅ Consistent API blocking and command handling across locales

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 12, 2026

CodeAnt AI is reviewing your PR.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai bot added the size:L This PR changes 100-499 lines, ignoring generated files label Apr 12, 2026
@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch from 1f83a8f to 05bfab2 Compare April 12, 2026 08:00
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f83a8fd7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 12, 2026

CodeAnt AI finished reviewing your PR.

@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch 8 times, most recently from 2179fbb to bedec50 Compare April 12, 2026 08:50
@halibobo1205
Copy link
Copy Markdown
Owner Author

@CodeAnt-AI: review

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 12, 2026

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@halibobo1205
Copy link
Copy Markdown
Owner Author

@codex: review

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codeant-ai codeant-ai bot added size:XL This PR changes 500-999 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Apr 12, 2026
@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 12, 2026

Sequence Diagram

This PR changes account ID index key generation to use locale-independent lowercase and adds a Turkish-like locale fallback so nodes both avoid future consensus splits and can still read and reject duplicates for legacy data.

sequenceDiagram
    participant Node
    participant IndexStore
    participant Database

    Node->>IndexStore: Store account by accountId
    IndexStore->>IndexStore: Compute newKey = accountId lowercased with ROOT locale
    IndexStore->>Database: Save newKey mapped to account address

    Node->>IndexStore: Lookup or check duplicate by accountId
    IndexStore->>IndexStore: Compute newKey = accountId lowercased with ROOT locale
    IndexStore->>Database: Read by newKey
    alt No value and system locale is Turkish like
        IndexStore->>IndexStore: Compute legacyKey using Turkish locale rules
        IndexStore->>Database: Read by legacyKey
    end
    IndexStore-->>Node: Return address if found so Node can read or reject duplicate
Loading

Generated by CodeAnt AI

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 12, 2026

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch from bedec50 to 3dbe19e Compare April 12, 2026 09:04
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch from 3dbe19e to bf2c75c Compare April 12, 2026 09:12
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@halibobo1205
Copy link
Copy Markdown
Owner Author

@codex: review

@halibobo1205
Copy link
Copy Markdown
Owner Author

@codex: review

@halibobo1205
Copy link
Copy Markdown
Owner Author

@CodeAnt-AI: review

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai bot added size:XXL This PR changes 1000+ lines, ignoring generated files and removed size:XL This PR changes 500-999 lines, ignoring generated files labels Apr 13, 2026
@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

Sequence Diagram

This PR makes account ID handling locale-independent by migrating legacy Turkish keys at startup and adding a runtime lookup that first uses Locale.ROOT, then falls back to Turkish-encoded keys so all nodes agree on duplicate account IDs.

sequenceDiagram
    participant Node
    participant Manager
    participant DynamicProps as DynamicPropertiesStore
    participant Migrator as MigrateTurkishKeyHelper
    participant Client
    participant AccountIndex as AccountIdIndexStore
    participant DB as RevokingDB

    Node->>Manager: init node
    Manager->>DynamicProps: getTurkishKeyMigrationDone
    alt Migration not done
        Manager->>Migrator: migrate Turkish legacy keys
        Migrator->>DB: scan and normalize Turkish keys to ROOT form
        Migrator->>DynamicProps: saveTurkishKeyMigrationDone(1)
    end

    Client->>AccountIndex: resolve accountId
    AccountIndex->>DB: lookup Locale.ROOT lowercase key
    alt ROOT key miss
        AccountIndex->>DB: lookup Turkish legacy keys (direct and normalized)
    end
    DB-->>AccountIndex: stored account address if found
    AccountIndex-->>Client: return address or not found
Loading

Generated by CodeAnt AI

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e00d67ddf4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

String.toLowerCase()/toUpperCase() without an explicit Locale uses
Locale.getDefault(), which on Turkish (tr) or Azerbaijani (az) systems
folds 'I' to dotless-ı (U+0131) instead of 'i' (U+0069). This caused
different index keys on tr/az nodes → potential consensus split.

Changes:
- Fix all toLowerCase()/toUpperCase() calls to use Locale.ROOT
- Enable the ErrorProne StringCaseLocaleUsage checker at ERROR level
  to prevent future regressions at compile time
- Add dual Turkish legacy fallback in AccountIdIndexStore for backward
  compatibility with keys written under tr/az locale:
  1. Direct probe: toLowerCase(TURKISH) for same-case queries
  2. Normalized probe: replace 'i' with 'ı' for cross-case queries
- Add one-time data migration (MigrateTurkishKeyHelper) to normalize
  all Turkish legacy keys (ı → i) at startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: codeant-ai[bot] <151821869+codeant-ai[bot]@users.noreply.github.com>
Co-Authored-By: codex <codex@openai.com>
@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch from e00d67d to 979beae Compare April 13, 2026 07:52
@halibobo1205
Copy link
Copy Markdown
Owner Author

@codex: review

@halibobo1205
Copy link
Copy Markdown
Owner Author

@CodeAnt-AI: review

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai bot added size:XL This PR changes 500-999 lines, ignoring generated files and removed size:XXL This PR changes 1000+ lines, ignoring generated files labels Apr 13, 2026
@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

Sequence Diagram

This PR adds a startup migration that normalizes legacy Turkish account ID index keys to a locale‑independent form and ensures all future account ID lookups use Locale.ROOT, preventing consensus splits across nodes with different default locales.

sequenceDiagram
    participant Node
    participant Manager
    participant DynamicPropertiesStore
    participant MigrateTurkishKeyHelper
    participant AccountIdIndexStore

    Node->>Manager: Start node (init)
    Manager->>DynamicPropertiesStore: getTurkishKeyMigrationDone
    alt migration not done
        Manager->>MigrateTurkishKeyHelper: doWork
        MigrateTurkishKeyHelper->>AccountIdIndexStore: Scan keys for Turkish dotless i
        MigrateTurkishKeyHelper->>AccountIdIndexStore: Rewrite keys with standard i and delete old keys
        MigrateTurkishKeyHelper->>DynamicPropertiesStore: saveTurkishKeyMigrationDone(1)
    end
    Node->>AccountIdIndexStore: Put or query account by accountId
    AccountIdIndexStore->>AccountIdIndexStore: Normalize accountId with Locale.ROOT and read/write index entry
Loading

Generated by CodeAnt AI

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 979beae895

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

halibobo1205 and others added 2 commits April 13, 2026 16:07
The dual Turkish fallback (direct + normalized probes) in
AccountIdIndexStore is no longer needed because:
1. MigrateTurkishKeyHelper normalizes all legacy keys at startup
   before any queries are served
2. New writes always use Locale.ROOT, so no new Turkish keys are created
3. If migration is interrupted, the DynamicPropertiesStore flag ensures
   it reruns on next startup

This simplifies get()/has() to single-lookup operations and removes
~100 lines of fallback logic (DOTLESS_I, TURKISH locale, direct/
normalized key methods, lookupWithFallback).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace remaining @SuppressWarnings("StringCaseLocaleUsage") with
explicit Locale.ROOT in 6 test files. The suppress annotations were
hiding real locale-sensitivity bugs:

- BytecodeCompiler: toUpperCase() on opcode names like "jumpi" would
  produce "JUMPİ" on Turkish locale, causing opcode lookup failure
- ShieldedReceiveTest: toUpperCase() on ztron address containing 'i'
  would produce 'İ' instead of 'I', breaking address decoding
- CronExpressionTest, OperationsTest, DataWord, SM2KeyTest: hex
  formatting with toUpperCase() (low risk but inconsistent)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@halibobo1205 halibobo1205 force-pushed the fix/locale-sensitive-case-folding branch from 979beae to aa5489a Compare April 13, 2026 08:09
@halibobo1205
Copy link
Copy Markdown
Owner Author

@codex: review

@halibobo1205
Copy link
Copy Markdown
Owner Author

@CodeAnt-AI: review

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI is running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai bot added size:XL This PR changes 500-999 lines, ignoring generated files and removed size:XL This PR changes 500-999 lines, ignoring generated files labels Apr 13, 2026
@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

Sequence Diagram

This PR adds a one-time migration that normalizes Turkish legacy account ID index keys and updates all case handling to use Locale.ROOT so that account ID lookups behave consistently across nodes and prevent consensus splits.

sequenceDiagram
    participant Node
    participant Manager
    participant DynamicPropertiesStore
    participant TurkishKeyMigration
    participant AccountIdIndexStore
    participant Client

    Node->>Manager: Startup init
    Manager->>DynamicPropertiesStore: Check Turkish key migration flag

    alt Migration not done
        Manager->>TurkishKeyMigration: Run Turkish key migration
        TurkishKeyMigration->>AccountIdIndexStore: Scan and rewrite Turkish keys to ROOT form
        TurkishKeyMigration->>DynamicPropertiesStore: Mark migration done
    end

    Client->>AccountIdIndexStore: Put or query accountId
    AccountIdIndexStore->>AccountIdIndexStore: Normalize accountId with Locale.ROOT
    AccountIdIndexStore-->>Client: Store or lookup result using normalized key
Loading

Generated by CodeAnt AI

@codeant-ai
Copy link
Copy Markdown

codeant-ai bot commented Apr 13, 2026

CodeAnt AI finished running the review.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa5489a55b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI:reviewed AI review passed size:XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant