Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c5e3292
build: bootstrap the module, root files and CI (#89)
hmdsefi Oct 20, 2026
11f7b14
build: add the pre-commit gate script (#92)
hmdsefi Oct 20, 2026
bade9a8
test(archtest): enforce import, license and apply-path rules (#93)
hmdsefi Oct 20, 2026
0cde88f
docs: link the roadmap issue from the README (#90)
hmdsefi Oct 20, 2026
5c579e6
feat(chain): add chain values, filter, bloom and the Source interface…
hmdsefi Oct 20, 2026
afa71ff
feat(chainlog): add log entries and the record codec (#94)
hmdsefi Oct 20, 2026
dedc12c
test(fakechain): add the seeded fake chain and source (#96)
hmdsefi Oct 20, 2026
38a5aa6
feat(chainlog): add the validator state (#97)
hmdsefi Oct 20, 2026
9d2392d
feat(wal): add the meta and checkpoint files (#99)
hmdsefi Oct 20, 2026
ee9c45e
feat(chainlog): add the log interface, conformance suite and in-memor…
hmdsefi Oct 20, 2026
2dd35a0
feat(wal): add the segmented file log (#102)
hmdsefi Oct 20, 2026
232a219
feat(consumer): add the consumer contract and Run (#100)
hmdsefi Oct 20, 2026
18ce502
feat(tracker): follow new heads, detect reorgs and finalize (#101)
hmdsefi Oct 20, 2026
b6d0b3b
feat(consumer): add the snapshot-and-replay memory consumer (#105)
hmdsefi Oct 20, 2026
4a30d77
feat(tracker): cap the unfinalized window (#103)
hmdsefi Oct 20, 2026
e92d3e6
test(invariant): add the reorg invariant property test (#107)
hmdsefi Oct 20, 2026
3f2edd2
feat(wal): recover a torn tail on open (#104)
hmdsefi Oct 20, 2026
f333405
feat(wal): compact whole segments below the bound (#106)
hmdsefi Oct 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Contributing to evmsync

This page covers how to build and test evmsync, where code goes, the rules that keep a consumer's
state equal to the final canonical chain, and how to send a change.

For a bug fix or a small change, send a pull request. For a new feature, a new package or a change
in behavior, open an issue first, so we can agree on the design before you write the code.

## Development setup

You need Go 1.26 or newer and golangci-lint v2.14.0. golangci-lint v2.7.2 and other releases built
with Go 1.25 refuse to load a `go 1.26.0` module.

```shell
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.14.0
```

Before every commit, run the gate from the repository root:

```shell
scripts/gate.sh
```

It runs the same checks as CI on your working tree: gofmt, `go mod tidy -diff`, `go vet`, the
build and tests with `CGO_ENABLED=0`, the tests with the race detector, the nested modules and
golangci-lint. It prints one `PASS`, `FAIL` or `SKIP` line per step and ends with `RESULT GREEN` (exit 0)
or `RESULT RED` (exit 1). Three environment variables change what it runs:

| Variable | Default | Meaning |
|---|---|---|
| `GATE_TCS` | `local` | Space-separated `GOTOOLCHAIN` values to run the steps with, for example `local go1.26.0` |
| `GATE_LOGS` | `tmp/gate` under the repository root | Directory for each step's log, `<go version>-<step>.log` |
| `GATE_LINT` | `golangci-lint` | The golangci-lint binary; the gate checks that it is v2.14.0 |

CI runs golangci-lint, a build with `CGO_ENABLED=0` on the latest Go release, and the tests with the
race detector on the Go version in `go.mod`. It fails when total test coverage drops below 95%, the
threshold in `.testcoverage.yml`.

## Repository layout

Each directory is one package with one job; `pgstore`, `e2e` and `examples` are nested modules
with their own `go.mod`. The version is the release a package first ships in.

| Package | Holds | Version |
|---|---|---|
| `chain` | Block references, headers, logs, filters, blooms and the `Source` interface | v0.1 |
| `chainlog` | The canonical log: entries, offsets, the `Log` interface, the validator `State`, the record codec and an in-memory log | v0.1 |
| `chainlog/wal` | A `chainlog.Log` stored in segment files | v0.1 |
| `chainlog/logtest` | The conformance suite for `chainlog.Log` implementations | v0.1 |
| `tracker` | Reorg detection and finality | v0.1 |
| `consumer` | The `Consumer` contract, `Run` and `Memory` | v0.1 |
| `internal/archtest` | Tests for the import rules, license headers and apply-path rules | v0.1 |
| `internal/fakechain` | A seeded fake chain and a fake `Source` for tests | v0.1 |
| `internal/invariant` | The reorg invariant property test | v0.1 |
| `rpc` | The JSON-RPC client that implements `chain.Source` | v0.2 |
| `ingest` | Backfill, realtime sync and validation | v0.2 |
| `evmsync` (root) | `App`, typed handler dispatch and the `Store` interface | v0.3 |
| `memstore` | An in-memory store with an undo journal | v0.3 |
| `cmd/evmsync` | The `evmsync gen` command | v0.3 |
| `internal/gen` | ABI to Go source generation | v0.3 |
| `pgstore` | The PostgreSQL store, a nested module | v0.4 |
| `e2e` | End-to-end tests against Anvil, a nested module | v0.2 |
| `examples` | Runnable examples, a nested module | v0.4 |

Which package may import which is fixed, and `internal/archtest` fails the tests when an import
breaks the rules. A new package needs a design change first: open an issue that proposes it. The
change that adds it also adds it to this table and to the inventory in `internal/archtest`.

## Rules for apply-path code

The apply path is the code that turns log entries into state: `chain`, `chainlog`, `tracker`,
`consumer` and `memstore`. Replaying the same log must always build the same state, so this code
follows four rules, and `internal/archtest` checks them. The dispatch code of the root `evmsync`
package and `pgstore` follows the same rules; review checks those by hand.

- **No wall clock.** Read time from the block (`chain.BlockRef.Time`), never from `time.Now`,
`time.Since`, `time.Until`, `time.Sleep`, timers or tickers.
- **No global randomness.** No functions of `math/rand` or `math/rand/v2`.
- **No goroutines.** No `go` statement, except inside `consumer.Run`.
- **No map order.** Range over sorted keys, never over a map directly. Wrap `maps.Keys` and
`maps.Values` in `slices.Sorted`; instead of `maps.All`, range over the sorted keys.

When the order of a map range never reaches state, an entry, an error or a log line, annotate the
loop with a reason:

```go
for _, v := range balances { //evmsync:maporder addition does not depend on order
total += v
}
```

The directive applies to its own line when code precedes it, otherwise to the next line. It needs a
reason, it must suppress a finding, and it has no space after `//`.

## License header

Every `.go` file, generated files included, starts with these two lines and a blank line. Files
under `testdata/` are the exception.

```go
// Copyright 2026 Hamed Yousefi
// SPDX-License-Identifier: Apache-2.0
```

## Commits

Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/): a type, the
package as the scope, and a short subject, for example `feat(chainlog): add the record codec` or
`fix(tracker): stop at the finalized block`. Changes to `go.mod` and the lint config use the
`build` type, changes to workflows the `ci` type.
80 changes: 80 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: build
on: [ push, pull_request ]
permissions:
contents: read
jobs:
golangci:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: v2.14.0

build:
name: Go build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version: stable # the test job covers the minimum version in go.mod
- name: Build
env:
CGO_ENABLED: '0'
run: go build -v ./...

test:
name: Go test
runs-on: ubuntu-latest
outputs:
coverage: ${{ steps.coverage.outputs.total-coverage }}
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
- name: Test
run: go test -race -covermode=atomic -coverprofile=cover.out ./...

- name: Check coverage
id: coverage
uses: vladopajic/go-test-coverage@f94bcf0d6b9fa5fb8b783830b22648f6c17475e2 # v2.19.0
with:
config: ./.testcoverage.yml

- name: Coverage summary
if: ${{ !cancelled() && steps.coverage.outputs.report != '' }}
env:
REPORT: ${{ fromJSON(steps.coverage.outputs.report) }}
run: |
{
echo '```'
echo "$REPORT"
echo '```'
} >> "$GITHUB_STEP_SUMMARY"

coverage-badge:
name: Coverage badge
needs: test
if: github.event_name == 'push' && github.ref_name == 'main'
runs-on: ubuntu-latest
permissions:
issues: write # the README coverage badge shows the title of issue #95
steps:
- name: Update coverage badge
env:
GH_TOKEN: ${{ github.token }}
COVERAGE: ${{ needs.test.outputs.coverage }}
run: |
test -n "$COVERAGE"
title="$COVERAGE%"
current=$(gh issue view 95 --repo "$GITHUB_REPOSITORY" --json title --jq .title)
if [ "$current" != "$title" ]; then
gh issue edit 95 --repo "$GITHUB_REPOSITORY" --title "$title"
fi
29 changes: 29 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: nightly

on:
schedule:
- cron: '0 3 * * *'
workflow_dispatch:

permissions:
contents: read

env:
GOTOOLCHAIN: local

jobs:
invariant:
name: reorg invariant (100,000 seeds)
runs-on: ubuntu-latest
timeout-minutes: 150
env:
GOWORK: 'off'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.27.x'
- name: reorg invariant
run: |
EVMSYNC_INVARIANT_SEEDS=100000 EVMSYNC_INVARIANT_BASE=$(date -u +%Y%m%d)000000 \
go test ./internal/invariant -run '^TestReorgInvariant$' -count=1 -timeout 120m
5 changes: 0 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,6 @@ go.work.sum
*.swo
*~

# AI coding tools (local settings)
.cursor/
.claude/settings.local.json
.claude/worktrees/

# OS files
.DS_Store
Thumbs.db
Expand Down
50 changes: 50 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
version: "2"
linters:
enable:
- bodyclose
- dogsled
- gochecknoinits
- gocritic
- gosec
- nakedret
- staticcheck
- unconvert
- unparam
- whitespace
disable:
- dupl
- gochecknoglobals
- lll
settings:
errcheck:
check-type-assertions: true
goconst:
min-len: 2
min-occurrences: 2
gocritic:
disabled-checks:
- ifElseChain
gocyclo:
min-complexity: 25
nakedret:
max-func-lines: 15
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
formatters:
enable:
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
7 changes: 7 additions & 0 deletions .testcoverage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Settings for go-test-coverage, used by CI (.github/workflows/build.yml).
# See https://github.com/vladopajic/go-test-coverage for all options.
profile: cover.out

threshold:
# Minimum coverage of the whole module, in percent.
total: 95
Loading
Loading