feat: Improve perf and add conveniences#8
Merged
Conversation
Benchmark resultsNo performance regression detected (within 25% tolerance). Aggregate report vs. baseCompared against base |
There was a problem hiding this comment.
Pull request overview
This PR expands the JSON repair library with advanced APIs (structured repair results, multi-document repair, streaming repair), adds configurable duplicate-key handling, and refactors the internal parser for better state/output management and improved numeric/keyword normalization.
Changes:
- Added advanced/public APIs:
repairWithDetails(),repairAll(),StreamingJsonRepairer, scalar-capabledecode(), andRepairResult. - Introduced duplicate-key handling via
DuplicateKeyPolicyand supporting parser changes. - Refactored internals with
ParserStateandOutputTracking, plus input handling improvements and benchmark automation.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Unit/ValuesAndStructuresTest.php | New focused tests for scalar values, escaping, unicode, and structures. |
| tests/Unit/StreamingJsonRepairerTest.php | Adds incremental streaming repair coverage. |
| tests/Unit/OptionsTest.php | Adds option-focused tests for omit behaviors. |
| tests/Unit/LoggingTest.php | Adds logging/assertions for repair diagnostics and logger integration. |
| tests/Unit/JsonRepairsTest.php | Splits core repair behavior tests from the former monolithic suite. |
| tests/Unit/JsonRepairerTest.php | Removes the old monolithic test file (suite split across new files). |
| tests/Unit/EdgeCasesTest.php | Adds coverage for markdown extraction, comments, and complex edge cases. |
| tests/Unit/ApiUsageTest.php | Covers new APIs (repairWithDetails, repairAll, duplicate-key policy, scalar decode). |
| tests/Datasets/JsonRepair.php | Extends datasets for markdown/text extraction and invalid/non-finite number cases. |
| src/StreamingJsonRepairer.php | Implements streaming “feed/current/reset” repair wrapper with logger passthrough. |
| src/RepairResult.php | Adds structured result object for repairWithDetails(). |
| src/ParserState.php | Centralizes parser state constants for traits/state machine. |
| src/JsonRepairer.php | Core implementation changes: new APIs, duplicate key policy hooks, output finalization, output tracking. |
| src/functions.php | Extends helper functions with duplicateKeyPolicy and updates decode return type to mixed. |
| src/DuplicateKeyPolicy.php | Adds enum for keep-first/keep-last duplicate key strategies. |
| src/Concerns/StringHeuristics.php | Updates state references to new ParserState constants. |
| src/Concerns/StateMachine.php | Refactors parser state machine, keyword/number handling, and duplicate-key hooks. |
| src/Concerns/RepairLogging.php | Adds fix-collection support for structured reporting. |
| src/Concerns/OutputTracking.php | Adds efficient “last non-whitespace char” tracking for output manipulation. |
| src/Concerns/InputSanitization.php | Improves markdown extraction fast-path and adds top-level segment extraction for repairAll(). |
| phpbench.json | Tunes phpbench config and storage path. |
| docs/json-repair/configuration.mdx | Documents duplicate key policy and advanced APIs. |
| composer.json | Adds benchmark scripts for baseline/CI comparison. |
| .gitignore | Ignores phpbench artifacts directory. |
| .github/workflows/benchmark.yml | Adds PR benchmark regression workflow and reporting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/Concerns/StateMachine.php:54
- The PHPDoc block starting at
/** Handle the starting state of parsing... */is not attached to any symbol (it’s followed by another docblock and thentryOpenObjectOrArray()), so it becomes an orphaned PHPDoc comment. Consider converting it to a regular block comment (/* ... */) or moving it directly above the method it documents to avoid tooling confusion.
/**
* Handle the starting state of parsing.
*
* Processes the first character of the JSON, expecting either an object
* opening brace or an array opening bracket.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several enhancements and refactorings to the JSON repair library, focusing on improved duplicate key handling, advanced API features, and internal code structure. The most notable changes include the addition of a configurable duplicate key policy, new advanced APIs for detailed repair reports and streaming, and significant internal refactoring for better state management and output tracking.
Feature Enhancements:
duplicateKeyPolicyoption to the JSON repairer, allowing users to specify whether to keep the first or last value for duplicate object keys, or leave them as-is.repairWithDetails()returns a structured report including the repaired JSON, validity status, and a list of fixes.repairAll()supports repairing multiple top-level JSON values (e.g., NDJSON).StreamingJsonRepairerenables streaming repair with best-effort output on partial data.decode()now supports direct decoding of top-level scalar values.Internal Refactoring and New Utilities:
OutputTrackingtrait to efficiently track the last non-whitespace character and manage output string state, improving handling of edge cases like missing values.ParserStateclass/enum instead of raw constants, making the parser's state transitions clearer and less error-prone. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]extractAllTopLevelJson()utility to extract all top-level objects/arrays from input, supporting more robust handling of multi-document or NDJSON-style input.Logging and Diagnostics:
RepairLoggingtrait to support collecting and returning a list of all fixes applied during repair, enabling detailed diagnostics and structured reporting.Input Handling Improvements:
These changes collectively improve the flexibility, robustness, and observability of the JSON repair process, making the library more suitable for advanced and streaming scenarios.