Skip to content

fix: validate model exists in settings.toml#36

Open
NishchayMahor wants to merge 1 commit into
amazon-science:mainfrom
NishchayMahor:fix/validate-model-section
Open

fix: validate model exists in settings.toml#36
NishchayMahor wants to merge 1 commit into
amazon-science:mainfrom
NishchayMahor:fix/validate-model-section

Conversation

@NishchayMahor
Copy link
Copy Markdown

Summary

Closes #27.

Calling wraval with a --model value that has no matching [<model>] section in config/settings.toml blows up later in get_settings() at settings.endpoint_type with AttributeError: 'Settings' object has no attribute 'ENDPOINT_TYPE', because Dynaconf silently loads nothing for the missing env and the default values never get attached.

This adds a small preflight check that validates model against the section names actually present in settings.toml before Dynaconf is instantiated, and raises a ValueError listing the available models so the user can see what they typed wrong.

What changed

  • src/wraval/main.py: extracted the settings file path to a module constant, added _known_models() (regex over settings.toml for section headers, ignoring [default]), and added a case-insensitive validation step inside get_settings() that matches the Dynaconf case_sensitive=False semantics introduced in Add case_insensitive=False to Dynaconf instantiation #28.
  • No new dependencies. Stdlib re only.

Behavior

Before:

$ wraval inference --model totally-bogus
... (later)
AttributeError: 'Settings' object has no attribute 'ENDPOINT_TYPE'

After:

$ wraval inference --model totally-bogus
ValueError: Unknown model 'totally-bogus'. Add a [totally-bogus] section to config/settings.toml,
or pick one of: haiku-3, haiku-3-5, haiku-3-5-cross, haiku-4-5, nova-lite, phi-3-5-4B,
phi-3-ollama, phi-4-4B, qwen-2-5-1-5B, qwen-3-0-6B, qwen-3-1-7B, qwen-3-1-7B-async,
qwen-3-4B, qwen-3-4B-async, sonnet-3, sonnet-3-5.

Test plan

Repo currently has no test suite. Verified manually that:

  • known model names load through (e.g. haiku-3)
  • default is accepted explicitly
  • case-insensitive matches go through (e.g. Haiku-3, HAIKU-3)
  • unknown names raise ValueError with the full list of available sections
  • empty string raises ValueError

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

handle exceptions in settings.toml

1 participant