Skip to content

feat(compat): adapt OpenEvolve evaluators for AlphaEvolve candidates - #526

Merged
codelion merged 2 commits into
algorithmicsuperintelligence:mainfrom
Yushu-Li:feat/alphaevolve-reverse-evaluator
Oct 21, 2026
Merged

codelion merged 2 commits into
algorithmicsuperintelligence:mainfrom
Yushu-Li:feat/alphaevolve-reverse-evaluator

Conversation

@Yushu-Li

Copy link
Copy Markdown
Contributor

Existing OpenEvolve evaluators accept a file path, while AlphaEvolve supplies a candidate object. Add an opt-in adapt_openevolve_evaluator(evaluate, objective_metric=...) helper that writes a single Python candidate to a temporary file and returns the nested score dictionary accepted by the upstream SDK's AlphaEvolveExperiment.

This continues the evaluator-contract work in #468 in the reverse direction. It is an independent slice; #468 remains open.

  • Support metrics dictionaries and EvaluationResult, preserving numeric metric names and values.
  • Ignore candidate path metadata and clean up temporary files on success, exceptions and interrupts.
  • Convert evaluator exceptions, explicit error/timeout markers and non-finite scores to a configurable finite failure score. Generic feedback does not copy exception text or artifact contents.
  • Reject malformed, multi-file, non-Python and asynchronous evaluator contracts. Omit artifacts with a warning and document these limits, including the absence of timeout enforcement or sandboxing.

The adapter adds no Google dependency or client/authentication behavior. Export bundles, reverse configuration mapping and managed-service execution remain separate work.

Validation:

  • 18 new offline tests, including concurrent calls, callable serialization and round-trip use with the existing inbound adapter.
  • Full unit suite: 751 tests passed with a placeholder API key; Black, isort and whitespace checks passed.
  • Offline integration against the unmodified Google SDK at 674dd5e558e917320c251c085e47fae33e1bbcff: both unchanged OpenEvolve circle_packing and function_minimization evaluators/seeds passed through the real SDK Experiment and EvaluationWorker, and the outgoing payload passed its submission model validation. A local recording transport replaced the cloud client. An intentionally invalid candidate also produced valid failure scores. No network calls, credential lookups or managed-service run were made.

Developed with AI coding assistance; the implementation and validation results were reviewed before submission.

The 200-draw Monte Carlo assertion can fail for a valid sampler when the set-backed population order changes. Inspect the distribution passed to random.choices and compare its expected score against uniform sampling instead. This retains the high-fitness preference check without random CI failures.
@codelion
codelion merged commit 162c3fe into algorithmicsuperintelligence:main Oct 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants