Add LLM-oriented API reference#1
Conversation
There was a problem hiding this comment.
Overall i'm not opposed to having an llm doc for the repo. There are a couple things i'd like to see addressed before i merge it. I fixed some issues as well, please rebase the pr if you need the LLM which generated it to look over the definitions again.
| - Normally this is sent once before move/eval requests. | ||
| - If `resettable_state` is true, setup may be sent again later. | ||
| - If `free_setup` is false, the board must contain exactly one cross at `(0, 0)` and no circles. | ||
| - The websocket board schema does not include `to_move`; turn context is inferred from requests. |
There was a problem hiding this comment.
to_move should be inferred from the first move_request or eval_request packet.
| ### `evaluation_time_limit` | ||
|
|
||
| The bot supports and respects evaluation time limits. The base capability spec mentions this | ||
| flag, but the websocket packet schema in this repo does not define an evaluation time-limit field. |
There was a problem hiding this comment.
Will update the spec in a bit, rebase the pr, remove this comment after.
There was a problem hiding this comment.
Updated the spec to correctly include evaluation_time_limit.
|
|
||
| The base capability spec says the client may send an interrupt packet if this is true. | ||
| However, no interrupt packet schema is defined in `definitions/basic_websocket/bws-v1-alpha.yaml`. | ||
| Treat this as an incomplete part of the spec unless a separate contract exists. |
There was a problem hiding this comment.
Correct, until someone needs this capability, there is no spec for interruption packets, so by definition this capability can not be ever supported as of right now
| An implementation or integration should account for these issues: | ||
|
|
||
| - No authentication scheme is defined. | ||
| - No standard error response format is defined. |
There was a problem hiding this comment.
Standard error response on basic_websocket api is to terminate the socket.
| - No standard error response format is defined. | ||
| - No explicit websocket error packet is defined. | ||
| - `interruptable` is advertised, but no interrupt packet schema is included. | ||
| - `evaluation_time_limit` is advertised, but no eval time-limit request field is defined. |
There was a problem hiding this comment.
As mentioned above, rebase, remove once the changes are pushed.
| - `interruptable` is advertised, but no interrupt packet schema is included. | ||
| - `evaluation_time_limit` is advertised, but no eval time-limit request field is defined. | ||
| - The stateless move schema intends a two-piece move but does not enforce array length. | ||
| - Some descriptions in the websocket YAML contain small wording issues, but the intended behavior |
There was a problem hiding this comment.
This line is unnecessary and may cause confusion for LLMs using the doc.
| "v1-alpha": { | ||
| "api_root": "bws/v1-alpha", | ||
| "move_time_limit": true, | ||
| "evaluation_time_limit": false, | ||
| "config": { | ||
| "dynamic": false | ||
| }, | ||
| "free_setup": false, | ||
| "move_skips": false, | ||
| "dual_sided": false, | ||
| "free_move_order": false, | ||
| "evaluation": true, | ||
| "resettable_state": false, | ||
| "interruptable": false | ||
| } |
There was a problem hiding this comment.
It is better to leave out unsupported capability fields, rather than setting them to false.
2 participants
No description provided.