Skip to content

Commit 737ccee

Browse files
shuds13shuds13
authored
Merge pull request #1709 from Libensemble/skills/add_generate_scripts
Add generate_scripts skill
2 parents 5dfb75b + ea4bbf0 commit 737ccee

5 files changed

Lines changed: 447 additions & 0 deletions

File tree

‎.claude/skills/generate-scripts/SKILL.md‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
---
2+
name: generate-scripts
3+
description: Generate libEnsemble calling scripts based on user requirements
4+
---
5+
6+
You are generating libEnsemble scripts. libEnsemble coordinates parallel simulations
7+
with generator-directed optimization or sampling. You will produce a calling script
8+
and, when an external application is involved, a sim function file.
9+
10+
libEnsemble repository: https://github.com/Libensemble/libensemble
11+
If not running inside the libEnsemble repo, find examples and source code there.
12+
13+
## Workflow
14+
15+
1. If converting an existing Xopt or Optimas workflow to libEnsemble, use the
16+
existing generator and VOCS settings exactly as-is — even if it is a sampling
17+
or exploration generator. Do not switch to a classic generator unless the user
18+
specifically asks.
19+
Otherwise, if there is not a clear generator to use, read `references/generators.md`
20+
to determine which generator and
21+
style to use. If a specific generator is identified (e.g., APOSMM), read its
22+
dedicated guide (e.g., `references/aposmm.md`).
23+
24+
2. Find a relevant example in `libensemble/tests/regression_tests/` and read it as a
25+
reference. Some examples:
26+
- Xopt Bayesian optimization (VOCS): `test_xopt_EI_initial_sample.py` — best Xopt
27+
example as it demonstrates the initial sampling approach Xopt generators need
28+
- Optimas Ax optimization (VOCS): `test_optimas_ax_sf.py`
29+
- APOSMM with NLopt (classic): `test_persistent_aposmm_nlopt.py`
30+
- Random uniform sampling (classic): `test_1d_sampling.py`
31+
Use glob and grep to find others matching the generator or pattern needed.
32+
The regression tests have clear descriptions in the docstring.
33+
34+
3. Write the calling script adapting the example to the user's requirements.
35+
Do not copy test boilerplate from examples
36+
(e.g., "Execute via one of the following commands..." headers). Set nworkers
37+
directly in the script (in LibeSpecs) — do not use parse_args or command-line
38+
arguments unless the user asks for that. If parse_args is not used and no
39+
options are taken, then do not ever suggest running with "-n/nworkers" or comms.
40+
Those optins are used only with parse_args (used in tests).
41+
42+
4. If the user has an external application (executable), also write a sim function file
43+
that uses the executor to run it.
44+
45+
5. If the user provides an input file, check whether it has Jinja2 template markers
46+
(`{{ varname }}`). If not, create a templated copy: replace parameter values with
47+
`{{ name }}` markers matching `input_names` in sim_specs (case-sensitive). The sim
48+
function uses `jinja2.Template` to render the file before each simulation. Never
49+
modify the user's original file.
50+
51+
6. Verify the scripts:
52+
- Bounds and dimension match the user's request
53+
- Executable path is correct
54+
- For VOCS: variable names are consistent between VOCS definition and sim function
55+
- For APOSMM: gen_specs outputs include all required fields
56+
- Input file template markers match input_names (case-sensitive)
57+
- The app_name in submit() matches register_app()
58+
59+
7. Present a concise summary highlighting: generator choice, bounds, parameters,
60+
sim_max, and objective field. Do NOT suggest `mpirun` or other MPI
61+
runner (srun, mpiexec, etc.) to launch libEnsemble unless the user explicitly
62+
asks for MPI-based comms.
63+
64+
8. Ask the user if they want to run the scripts.
65+
66+
9. If running: execute with `python script.py`. Do not use `mpirun` or other MPI
67+
runner (srun, mpiexec, etc.) to launch libEnsemble unless the user explicitly
68+
asks for MPI-based comms for distributing workers. This is unrelated to
69+
MPIExecutor, which workers use to launch simulation applications across nodes
70+
— libEnsemble manages node allocation.
71+
If scripts fail, retry if you can see a fix, otherwise stop. After a successful
72+
run, read `references/results_metadata.md` and
73+
`references/finding_objectives.md` to interpret the output.
74+
75+
## Generator style
76+
77+
VOCS (gest-api) is the default style. It uses a VOCS object to define variables and
78+
objectives, and a generator object from Xopt or Optimas. Use VOCS unless the user
79+
explicitly asks for the classic style or the generator only exists in classic form
80+
(e.g., APOSMM, persistent_sampling).
81+
82+
## Defaults
83+
84+
- nworkers defaults to 4 unless the user specifies otherwise (or 1 for sequential
85+
generators like Nelder-Mead)
86+
- All nworkers are available for simulations
87+
- No alloc_specs needed — all allocator options are available as GenSpecs parameters
88+
- Use `async_return=True` in GenSpecs unless there is a reason to use batch returns
89+
90+
## VOCS generators (Xopt / Optimas)
91+
92+
Key patterns:
93+
- Variables named individually in VOCS: `{"x0": [lb, ub], "x1": [lb, ub]}`
94+
- Objectives named in VOCS: `{"f": "MINIMIZE"}`
95+
- GenSpecs uses `generator=`, `vocs=`, `batch_size=`
96+
- SimSpecs uses `vocs=` or `simulator=` for gest-api style sim functions
97+
- No `add_random_streams()` needed
98+
- Xopt generators need `initial_sample_method="uniform"` and `initial_batch_size=`
99+
for initial evaluated data. Optimas handles its own sampling.
100+
101+
See `references/generators.md` for the full generator selection guide.
102+
103+
## Classic generators
104+
105+
Used only when the generator has no VOCS version or the user explicitly requests it.
106+
- One worker is consumed by the persistent generator
107+
- Requires `add_random_streams()`
108+
- APOSMM: see `references/aposmm.md` for full configuration details
109+
110+
## Sim function patterns
111+
112+
**Inline sim function** (no external app): Takes `(H, persis_info, sim_specs, libE_info)`
113+
and returns `(H_o, persis_info)`. Or for VOCS gest-api style, takes `input_dict: dict`
114+
and returns a dict. See `libensemble/sim_funcs/` for built-in examples.
115+
116+
**Executor-based sim function** (external app): Uses MPIExecutor to run an application.
117+
Pattern:
118+
1. Register app in calling script: `exctr.register_app(full_path=..., app_name=...)`
119+
2. In sim function: get executor from `libE_info["executor"]`, submit with
120+
`exctr.submit(app_name=...)`, wait with `task.wait()`
121+
3. Read output file to get objective value
122+
4. Set `sim_dirs_make=True` in LibeSpecs
123+
5. If using input file templating, set `sim_dir_copy_files=[input_file]`
124+
125+
## Results interpretation
126+
127+
After a successful run:
128+
- Load the .npy output file with `np.load()`
129+
- Always filter by `sim_ended == True` before analyzing — rows where sim_ended is False
130+
contain uninitialized values (often zeros) that are NOT real results
131+
- For APOSMM: check rows where `local_min == True` to find identified minima
132+
- Report the count, location, and objective value of minima or best points found
133+
- If the best objective value is exactly 0.0, verify those rows have sim_ended == True
134+
- See `references/results_metadata.md` for full details
135+
136+
## Reference docs (read as needed)
137+
138+
All paths relative to this skill's directory:
139+
140+
- `references/generators.md` — Generator selection guide, VOCS vs classic
141+
- `references/aposmm.md` — APOSMM configuration, optimizer options, tuning
142+
- `references/finding_objectives.md` — Identifying objective fields in results
143+
- `references/results_metadata.md` — Interpreting history array, filtering results
144+
145+
## User request
146+
147+
$ARGUMENTS

‎.claude/skills/generate-scripts/references/aposmm.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# APOSMM — Asynchronously Parallel Optimization Solver for Multiple Minima
2+
3+
APOSMM coordinates concurrent local optimization runs to find multiple local minima on parallel hardware. Use when the user wants to find minima, optimize, or explore an optimization landscape.
4+
5+
Module: `persistent_aposmm`
6+
Function: `aposmm`
7+
Allocator: `persistent_aposmm_alloc` (NOT the default `start_only_persistent`)
8+
Requirements: mpmath, SciPy (plus optional packages for specific local optimizers)
9+
10+
## APOSMM gen_specs in generated scripts
11+
12+
When the MCP tool generates APOSMM scripts, run_libe.py gets this gen_specs structure:
13+
14+
```python
15+
gen_specs = GenSpecs(
16+
gen_f=gen_f,
17+
inputs=[],
18+
persis_in=["sim_id", "x", "x_on_cube", "f"],
19+
outputs=[("x", float, n), ("x_on_cube", float, n), ("sim_id", int),
20+
("local_min", bool), ("local_pt", bool)],
21+
user={
22+
"initial_sample_size": num_workers,
23+
"localopt_method": "scipy_Nelder-Mead",
24+
"opt_return_codes": [0],
25+
"nu": 1e-8,
26+
"mu": 1e-8,
27+
"dist_to_bound_multiple": 0.01,
28+
"max_active_runs": 6,
29+
"lb": np.array([...]), # MUST match user's requested bounds
30+
"ub": np.array([...]), # MUST match user's requested bounds
31+
}
32+
)
33+
```
34+
35+
With allocator:
36+
```python
37+
from libensemble.alloc_funcs.persistent_aposmm_alloc import persistent_aposmm_alloc as alloc_f
38+
```
39+
40+
## Required gen_specs["user"] Parameters
41+
42+
| Parameter | Type | Description |
43+
|-----------|------|-------------|
44+
| `lb` | n floats | Lower bounds on search domain |
45+
| `ub` | n floats | Upper bounds on search domain |
46+
| `localopt_method` | str | Local optimizer (see table below) |
47+
| `initial_sample_size` | int | Uniform samples before starting local runs |
48+
49+
When using a SciPy method, must also supply `opt_return_codes` — e.g. [0] for Nelder-Mead/BFGS, [1] for COBYLA.
50+
51+
## Optional gen_specs["user"] Parameters
52+
53+
| Parameter | Type | Description |
54+
|-----------|------|-------------|
55+
| `max_active_runs` | int | Max concurrent local optimization runs. Must not exceed nworkers. |
56+
| `dist_to_bound_multiple` | float (0,1] | Fraction of distance to boundary for initial step size |
57+
| `mu` | float | Min distance from boundary for starting points |
58+
| `nu` | float | Min distance from identified minima for starting points |
59+
| `stop_after_k_minima` | int | Stop after this many local minima found |
60+
| `stop_after_k_runs` | int | Stop after this many runs ended |
61+
| `sample_points` | numpy array | Specific points to sample (original domain) |
62+
| `lhs_divisions` | int | Latin hypercube partitions (0 or 1 = uniform) |
63+
| `rk_const` | float | Multiplier for r_k value |
64+
65+
## Worker Configuration
66+
67+
With `gen_on_manager=True`, the persistent generator runs on the manager process and all `nworkers` are available for simulations.
68+
69+
## Local Optimizer Methods
70+
71+
### SciPy (no extra install)
72+
73+
| Method | Gradient? | `opt_return_codes` |
74+
|--------|-----------|-------------------|
75+
| `scipy_Nelder-Mead` | No | [0] |
76+
| `scipy_COBYLA` | No | [1] |
77+
| `scipy_BFGS` | Yes | [0] |
78+
79+
### NLopt (requires nlopt package)
80+
81+
| Method | Gradient? | Description |
82+
|--------|-----------|-------------|
83+
| `LN_SBPLX` | No | Subplex. Good for noisy/nonsmooth |
84+
| `LN_BOBYQA` | No | Quadratic model. Good for smooth problems |
85+
| `LN_COBYLA` | No | Constrained optimization |
86+
| `LN_NEWUOA` | No | Unconstrained quadratic model |
87+
| `LN_NELDERMEAD` | No | Classic simplex |
88+
| `LD_MMA` | Yes | Method of Moving Asymptotes |
89+
90+
NLopt methods require convergence tolerances. If the user does not specify tolerances, use these defaults:
91+
92+
```python
93+
"xtol_abs": 1e-6,
94+
"ftol_abs": 1e-6,
95+
```
96+
97+
When using an NLopt method, always include `rk_const` scaled to the problem dimension:
98+
99+
```python
100+
from math import gamma, pi, sqrt
101+
n = <number of dimensions>
102+
rk_const = 0.5 * ((gamma(1 + (n / 2)) * 5) ** (1 / n)) / sqrt(pi)
103+
```
104+
105+
Use this formula directly in the generated script — do not precompute the value.
106+
107+
### PETSc/TAO (requires petsc4py package)
108+
109+
| Method | Needs | Description |
110+
|--------|-------|-------------|
111+
| `pounders` | fvec | Least-squares trust-region |
112+
| `blmvm` | grad | Bounded limited-memory variable metric |
113+
| `nm` | f only | Nelder-Mead variant |
114+
115+
### DFO-LS (requires dfols package)
116+
117+
| Method | Needs | Description |
118+
|--------|-------|-------------|
119+
| `dfols` | fvec | Derivative-free least-squares |
120+
121+
## Choosing a Local Optimizer
122+
123+
- **Default / simple**: `scipy_Nelder-Mead` — no extra packages
124+
- **Smooth, bounded**: `LN_BOBYQA` (NLopt)
125+
- **Noisy objectives**: `LN_SBPLX` (NLopt) or `scipy_Nelder-Mead`
126+
- **Gradient available**: `scipy_BFGS` or `LD_MMA`
127+
- **Least-squares (vector output)**: `pounders` (PETSc) or `dfols`
128+
- **Constrained**: `scipy_COBYLA` or `LN_COBYLA`
129+
130+
## Interpreting Results
131+
132+
After a run, report the number of minima found. Load the results `.npy` file,
133+
filter by `sim_ended == True`, then check `local_min == True` rows.
134+
Report the count, objective value, and location of each minimum.
135+
136+
## Tuning
137+
138+
If APOSMM is not finding minima, try increasing the multiplier in `rk_const` (e.g., from 0.5 to a larger value) to make it more aggressive about starting new local optimization runs in different regions.
139+
140+
Use this formula directly in the generated script — do not precompute the value.
141+
Also consider increasing `dist_to_bound_multiple` (e.g., 0.5) for a larger initial
142+
step size.
143+
144+
## Important
145+
146+
Always use the bounds, sim_max, and paths from the user's request. Never substitute values from examples or known problem domains.

‎.claude/skills/generate-scripts/references/finding_objectives.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Finding Objective Fields
2+
How to find objective field names in results files.
3+
4+
## VOCS scripts
5+
6+
The objective field name is defined in the VOCS object:
7+
```python
8+
vocs = VOCS(
9+
variables={"x0": [-2, 2], "x1": [-1, 1]},
10+
objectives={"f": "MINIMIZE"},
11+
)
12+
```
13+
14+
The key in `objectives` (e.g. `"f"`) is the objective field name in the results.
15+
16+
## Classic scripts
17+
18+
The objective field name is defined in `sim_specs` outputs:
19+
```python
20+
sim_specs = SimSpecs(
21+
...
22+
outputs=[("f", float)], # "f" is the objective field name
23+
)
24+
```
25+
26+
The field name in `outputs` (e.g. `"f"`) matches the field name in the `.npy` results file.
27+
28+
## Common patterns
29+
- Single objective: `{"f": "MINIMIZE"}` (VOCS) or `outputs=[("f", float)]` (classic)
30+
- Multiple outputs: `"f"` is typically the objective — the scalar float used by the generator
31+
- The objective field name in the VOCS definition or sim_specs outputs matches the field in the results

0 commit comments

Comments
 (0)