Skip to content

Commit 8bb62a0

Browse files
robotoerrobotoer
authored
Update README.md
1 parent 3cdb80f commit 8bb62a0

1 file changed

Lines changed: 187 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 187 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,190 @@
11
# script-runner
22

33
Enables the deployment of a script as an API protected by pluggable authentication
4-
(only auth0 currently supported).
4+
(only `auth0` or `none` currently supported). Script runner relies on a configuration
5+
file (`config.json`) to specify the command that should be executed when a task is
6+
started. Script runner exposes an API with two main endpoints:
7+
8+
- `POST /task` - Start new task
9+
- `GET /task/{id}` - Check task status/get results
10+
11+
Arguments to the script command can be passed via the request path, query, or JSON
12+
body.
13+
14+
Script runner relies on two running containers:
15+
16+
- Server - Responsible for responding to API requests and validating auth headers.
17+
- Worker - Responsible for running the script whenever a new task is started.
18+
19+
Script runner is built for running scripts that take a long period of time and uses
20+
worker nodes from above to launch jobs. Communication between the server and worker
21+
nodes happens through a Redis instance allowing multiple script invocations to be
22+
running simultaneously
23+
24+
25+
## Configuration
26+
27+
Script runner can be configured through a special file called `config.json`:
28+
29+
```
30+
{
31+
"command": [ ... ],
32+
"inputs": {
33+
"path_args": [ ... ],
34+
"query_args": [ ... ],
35+
"body_args": [ ... ]
36+
},
37+
"outputs": {
38+
"results": {
39+
"kind": "file",
40+
"format": "csv",
41+
"input_path": ...,
42+
"remapped_columns": [ ... ]
43+
},
44+
"attachments": [ ... ]
45+
}
46+
}
47+
```
48+
49+
### Command
50+
51+
The command used to launch run the desired script is specified through the `"command": [ ... ]`
52+
field. Specify each argument as a separate string. Parameters from the request path, query, or
53+
body can be used in the command by prefixing the parameter name with a `$` sign:
54+
55+
```
56+
"command": [ "/scripts/my_script.sh", "regular-arg-1", "$THIS_ARGUMENT_COMES_FROM_REQUEST" ]
57+
```
58+
59+
### Inputs
60+
61+
Parameters used in the script command are specified in the `"inputs": { ... }` field. The name of the
62+
variable for use in the command definition above is set by the `"env_var_name": "..."` field. First,
63+
select whether or not you would like to have the parameter specified in the request path, query, or body.
64+
65+
![image](https://user-images.githubusercontent.com/1199079/114761896-f417b300-9d15-11eb-9aa0-c498ae727eaa.png)
66+
67+
Path parameters are defined using regular expressions (https://www.w3schools.com/python/python_regex.asp)
68+
69+
```
70+
...
71+
72+
"path_args": [
73+
{
74+
"env_var_name": "BASESPACE_ID",
75+
"path_regex": "([^/]+)",
76+
"path_regex_match_group": 1
77+
}
78+
],
79+
80+
...
81+
```
82+
83+
Query parameters are defined by specifying the query parameter name:
84+
85+
```
86+
...
87+
88+
"query_args": [
89+
{
90+
"env_var_name": "SEASON",
91+
"query_var_name": "season"
92+
}
93+
],
94+
95+
...
96+
```
97+
98+
Body parameters are defined by specifying a JSONPath query (https://jsonpath.com/):
99+
100+
```
101+
...
102+
103+
"body_args": [
104+
{
105+
"env_var_name": "CITY",
106+
"body_jsonpath": "$.address.city"
107+
}
108+
],
109+
110+
...
111+
```
112+
113+
### Outputs
114+
115+
The JSON data returned by the script-runner API can be configured in the `"outputs"` section.
116+
117+
Script-runner reads a main file that can be either in a JSON or CSV file format to parse and
118+
send back as JSON under the `"results"` field. Script-runner also supports an `"attachments"`
119+
field that base64-encodes requested files into the JSON object response.
120+
121+
For example, the following config:
122+
123+
```
124+
"outputs": {
125+
"results": {
126+
"kind": "file",
127+
"format": "csv",
128+
"input_path": "LIMS_results.csv",
129+
"remapped_columns": [
130+
{
131+
"output_name": "plateIndex",
132+
"input_name": "Plate_384_Number",
133+
"skip_list": ["NA"]
134+
}
135+
]
136+
},
137+
"attachments": [
138+
"*.pdf",
139+
"LIMS_results.csv"
140+
]
141+
}
142+
```
143+
144+
Could produce the following output:
145+
146+
```
147+
{
148+
"id": "168af779-a72e-4dec-a2e3-36240cad5973",
149+
"status": "ready",
150+
"attachments": {
151+
"results-n1a54tmd.pdf": "JVBERi0xLjUKJdDUxdgKNiAwI...",
152+
...
153+
},
154+
"results": [
155+
{
156+
"plateIndex": "1",
157+
"plateCell": "A01",
158+
"marker1": "CATCTGTATC",
159+
"marker2": "ATGAGACTTG",
160+
"classification": "failed: low S2 & RPP30"
161+
},
162+
...
163+
]
164+
}
165+
```
166+
167+
168+
## Usage
169+
170+
To get a script-runner API setup and running:
171+
172+
1. Build a docker container with your script and script-runner installed.
173+
174+
```
175+
FROM python
176+
177+
pip install script-runner-api
178+
179+
# Install your script here
180+
COPY ./my_script.sh /app/my_script.sh
181+
182+
# ...
183+
```
184+
185+
2. Setup your `config.json` file. See `Configuration` section for more information.
186+
3. Deploy a worker and a server to your desired environment. We have provided some started
187+
[terraform](https://terraform.io/) modules
188+
([AWS](https://github.com/lab-grid/terraform-aws-ecs-script-runner),
189+
[Azure](https://github.com/lab-grid/terraform-azurerm-container-instances-script-runner))
190+
+ [examples](https://github.com/lab-grid/script-runner/tree/main/terraform) to help get started.

0 commit comments

Comments
 (0)