Skip to content

Commit 5b44956

Browse files
gnata: initial release
Pure-Go implementation of JSONata 2.x, built for high-throughput streaming evaluation. Features: - Full JSONata 2.x language support (1,273 test cases from the official jsonata-js suite) - Two-tier evaluation: GJSON zero-copy fast path for simple expressions, full AST walk for complex ones - Lock-free StreamEvaluator for batched concurrent evaluation - Schema-aware GroupPlan caching with bounded FIFO ring buffer - WASM build target for browser use - Single external dependency (tidwall/gjson) Copyright (c) 2026 RecoLabs Inc. Licensed under the MIT License. Signed-off-by: NirBarak-RecoLabs <nirb@recolabs.ai>
0 parents  commit 5b44956

1,397 files changed

Lines changed: 37929 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
name: CI
2+
on:
3+
push:
4+
branches: [main]
5+
pull_request:
6+
branches: [main]
7+
jobs:
8+
test:
9+
runs-on: ubuntu-latest
10+
timeout-minutes: 15
11+
steps:
12+
- uses: actions/checkout@v4
13+
- uses: actions/setup-go@v5
14+
with:
15+
go-version-file: go.mod
16+
- run: go test -race -count=1 ./...
17+
lint:
18+
runs-on: ubuntu-latest
19+
steps:
20+
- uses: actions/checkout@v4
21+
- uses: actions/setup-go@v5
22+
with:
23+
go-version-file: go.mod
24+
- uses: golangci/golangci-lint-action@v9
25+
with:
26+
version: latest

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*.wasm
2+
.secret
3+
.DS_Store

.golangci.yaml

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
version: "2"
2+
linters:
3+
enable:
4+
- asasalint
5+
- asciicheck
6+
- bidichk
7+
- copyloopvar
8+
- decorder
9+
- dogsled
10+
- dupl
11+
- dupword
12+
- durationcheck
13+
- errchkjson
14+
- errname
15+
- errorlint
16+
- exhaustive
17+
- exptostd
18+
- fatcontext
19+
- funlen
20+
- goconst
21+
- gocritic
22+
- gocyclo
23+
- goprintffuncname
24+
- gosec
25+
- gosmopolitan
26+
- grouper
27+
- iface
28+
- intrange
29+
- iotamixing
30+
- lll
31+
- makezero
32+
- misspell
33+
- modernize
34+
- nakedret
35+
- nilerr
36+
- nilnesserr
37+
- nolintlint
38+
- nosprintfhostport
39+
- prealloc
40+
- predeclared
41+
- reassign
42+
- recvcheck
43+
- revive
44+
- sloglint
45+
- staticcheck
46+
- testableexamples
47+
- testifylint
48+
- unconvert
49+
- unparam
50+
- usestdlibvars
51+
- usetesting
52+
- wastedassign
53+
- whitespace
54+
settings:
55+
funlen:
56+
lines: 100
57+
statements: 50
58+
gocritic:
59+
enabled-tags:
60+
- diagnostic
61+
- experimental
62+
- opinionated
63+
- performance
64+
- style
65+
lll:
66+
line-length: 140
67+
revive:
68+
rules:
69+
- name: var-naming
70+
disabled: true
71+
exclusions:
72+
presets:
73+
- comments
74+
- common-false-positives
75+
- legacy
76+
- std-error-handling
77+
rules:
78+
- path: (.+)\.go$
79+
text: G101
80+
- path: (.+)\.go$
81+
text: G404
82+
- path: (.+)\.go$
83+
text: G115
84+
- path: (.+)\.go$
85+
text: Error return value of `.*log.*ger.*\.Log` is not checked
86+
paths:
87+
- third_party$
88+
- builtin$
89+
- examples$
90+
issues:
91+
fix: true
92+
formatters:
93+
enable:
94+
- gci
95+
- gofmt
96+
- gofumpt
97+
- goimports
98+
exclusions:
99+
paths:
100+
- third_party$
101+
- builtin$
102+
- examples$

AGENTS.md

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# AGENTS.md
2+
3+
This file provides guidance to AI agents (Claude Code, GitHub Copilot, Cursor, etc.) when working with code in this repository.
4+
5+
## What Is Gnata
6+
7+
Gnata is a full JSONata 2.x implementation in Go, built for production streaming workloads. JSONata is a query and transformation language for JSON data. Gnata provides two-tier evaluation: fast-path (GJSON zero-copy) for simple expressions and full AST evaluation for complex ones, plus a lock-free `StreamEvaluator` for high-throughput batched evaluation.
8+
9+
## Development Commands
10+
11+
```sh
12+
# Lint (from package root)
13+
golangci-lint run
14+
15+
# Run all tests (1,273 JSON test cases from official jsonata-js suite + unit tests)
16+
go test ./...
17+
18+
# Run a single test
19+
go test -run TestName
20+
21+
# Run benchmarks
22+
go test -bench=. -benchmem
23+
```
24+
25+
## Architecture
26+
27+
### Compilation Pipeline
28+
29+
Lexer → Parser → AST Processing → Fast-Path Analysis → Expression
30+
31+
1. **Lexer** (`internal/lexer/`) — Tokenizes JSONata expression strings
32+
2. **Parser** (`internal/parser/`) — Pratt (top-down operator precedence) parser producing AST nodes
33+
3. **AST Processing** (`parser.ProcessAST`) — Normalizes and optimizes the AST
34+
4. **Fast-Path Analysis** (`parser.AnalyzeFastPath`) — Classifies expressions into:
35+
- Pure-path fast path (e.g., `Account.Name`) — uses GJSON zero-copy
36+
- Comparison fast path (e.g., `a.b = "x"`) — zero allocations
37+
- Full AST evaluation required
38+
39+
### Two-Tier Evaluation
40+
41+
- `Eval(ctx, any)` — Evaluate against pre-parsed Go values via full AST walk
42+
- `EvalBytes(ctx, json.RawMessage)` — Fast-path expressions use GJSON directly on raw JSON bytes; full-path falls back to unmarshal + Eval
43+
44+
### StreamEvaluator (stream.go)
45+
46+
Batch-evaluates multiple expressions against events. Schema-keyed `GroupPlan` caching deduplicates field extraction across expressions. Lock-free reads via `atomic.Pointer` snapshot; writes serialized by `sync.Mutex`. Single JSON scan per event via `gjson.GetManyBytes`.
47+
48+
### Evaluator Dispatch (internal/evaluator/)
49+
50+
`evaluator.Eval(node, input, env)` dispatches by `node.Type`. Each eval category is in its own file:
51+
- `eval_binary.go` — Binary operators, subscripts, filtering
52+
- `eval_function.go` — Function calls, lambdas, partial application
53+
- `eval_chain.go` — Path chaining, pipes, blocks, conditions
54+
- `eval_sort.go` — Sorting with generic `SortItemsErr[T any]` helper
55+
- `eval_transform.go` — JSONata transform operator (`|obj|updates|deletes|` syntax)
56+
- `eval_group.go` — Group-by reduction (`{key: val}` syntax)
57+
- `eval_range.go` — Range operator (`[start..end]`)
58+
- `eval_regex.go` — Regex compilation and matching
59+
- `eval_unary.go` — Unary operators (negation, array constructor)
60+
61+
### Standard Library (functions/)
62+
63+
55+ built-in JSONata functions across categorized files: `string_funcs.go`, `string_match_replace.go`, `string_format_number.go`, `string_format_integer.go`, `string_encoding.go`, `numeric_funcs.go`, `array_funcs.go`, `object_funcs.go`, `hof_funcs.go`, `boolean_funcs.go`, `datetime_funcs.go`, `datetime_format.go`, `datetime_parse.go`. All registered via `functions.RegisterAll`.
64+
65+
### Fast-Path Byte Evaluation (func_fast.go)
66+
67+
Dispatch-map of `funcFastHandlers` maps each `FuncFastKind` to a standalone handler function (e.g., `evalFuncContains`, `evalFuncString`). Each handler operates directly on `gjson.Result` for zero-copy evaluation. `FuncFastRound` is intentionally absent — it requires banker's rounding handled by the full evaluator.
68+
69+
### Key Types
70+
71+
- **Expression** (`gnata.go`) — Compiled, goroutine-safe JSONata expression with fast-path metadata
72+
- **StreamEvaluator** (`stream.go`) — Copy-on-write expression slice + `BoundedCache` for schema plans
73+
- **BoundedCache** (`bounded_cache.go`) — Lock-free FIFO ring-buffer cache (atomic pointer reads)
74+
- **OrderedMap** (`internal/evaluator/ordered_map.go`) — Insertion-ordered map preserving JSON field order
75+
- **Environment** (`internal/evaluator/env.go`) — Lexical scope chain for variable bindings
76+
77+
## Dependencies
78+
79+
Only one direct dependency (pure Go, no CGo):
80+
- `tidwall/gjson` — Zero-copy JSON field extraction for fast-path byte-level evaluation
81+
82+
Regex uses Go's standard `regexp` package (no external regex library).
83+
84+
## Testing
85+
86+
Tests use separate `_test` packages (`gnata_test`, `lexer_test`, `parser_test`). The primary test suite (`suite_test.go`) loads 1,200+ JSON test cases from `testdata/groups/` (100+ subdirectories) — each `.json` file is a case with `expr`, `dataset`, `bindings`, and `result` fields. Datasets live in `testdata/datasets/`. Additional unit tests in `evaluator_test.go` cover regression tests using table-driven test (TDT) style.
87+
88+
## Custom Functions
89+
90+
Register custom functions via `StreamEvaluator` options or standalone `CustomEnv`:
91+
```go
92+
customFuncs := map[string]gnata.CustomFunc{
93+
"md5": func(args []any, focus any) (any, error) { ... },
94+
}
95+
se := gnata.NewStreamEvaluator(nil, gnata.WithCustomFunctions(customFuncs))
96+
```
97+
98+
## WASM
99+
100+
`wasm/main.go` exports `gnataEval`, `gnataCompile`, `gnataEvalHandle` for browser use. Build with `GOOS=js GOARCH=wasm go build -ldflags="-s -w" -trimpath -o gnata.wasm ./wasm/`.

CODE_OF_CONDUCT.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
6+
7+
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
8+
9+
## Our Standards
10+
11+
Examples of behavior that contributes to a positive environment for our community include:
12+
13+
* Demonstrating empathy and kindness toward other people
14+
* Being respectful of differing opinions, viewpoints, and experiences
15+
* Giving and gracefully accepting constructive feedback
16+
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
17+
* Focusing on what is best not just for us as individuals, but for the overall community
18+
19+
Examples of unacceptable behavior include:
20+
21+
* The use of sexualized language or imagery, and sexual attention or advances of any kind
22+
* Trolling, insulting or derogatory comments, and personal or political attacks
23+
* Public or private harassment
24+
* Publishing others' private information, such as a physical or email address, without their explicit permission
25+
* Other conduct which could reasonably be considered inappropriate in a professional setting
26+
27+
## Enforcement Responsibilities
28+
29+
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
30+
31+
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
32+
33+
## Scope
34+
35+
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
36+
37+
## Enforcement
38+
39+
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **security@reco.ai**. All complaints will be reviewed and investigated promptly and fairly.
40+
41+
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
42+
43+
## Enforcement Guidelines
44+
45+
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
46+
47+
### 1. Correction
48+
49+
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
50+
51+
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
52+
53+
### 2. Warning
54+
55+
**Community Impact**: A violation through a single incident or series of actions.
56+
57+
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
58+
59+
### 3. Temporary Ban
60+
61+
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
62+
63+
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
64+
65+
### 4. Permanent Ban
66+
67+
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
68+
69+
**Consequence**: A permanent ban from any sort of public interaction within the community.
70+
71+
## Attribution
72+
73+
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
74+
75+
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
76+
77+
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].
78+
79+
[homepage]: https://www.contributor-covenant.org
80+
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
81+
[Mozilla CoC]: https://github.com/mozilla/diversity
82+
[FAQ]: https://www.contributor-covenant.org/faq
83+
[translations]: https://www.contributor-covenant.org/translations

0 commit comments

Comments
 (0)