Skip to content

Commit d0acbbb

Browse files
thomasyu888claude
andcommitted
[SYNPY-1480]: Improve documentation around configuration file
- Expand .synapseConfig example with improved comments for all sections - Document previously undocumented `use_boto_sts` transfer option - Improve `max_threads` comment with default, min, and max (128) values - Rewrite docs/tutorials/configuration.md with full table-driven coverage of every section including sftp, S3, cache, debug, endpoints, transfer - Remove duplication between configuration.md and authentication.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 693d01b commit d0acbbb

2 files changed

Lines changed: 128 additions & 33 deletions

File tree

docs/tutorials/configuration.md

Lines changed: 81 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,30 +4,102 @@ The Synapse Python client can be configured either programmatically or by using
44

55
**The default configuration file does not need to be modified for most use-cases**.
66

7+
When installing the Synapse Python client, the `.synapseConfig` file is added to your home directory. This file stores configuration options including your Synapse auth token, cache location, multi-threading settings, and storage credentials.
78

8-
When installing the Synapse Python client, the `.synapseConfig` is added to your home directory. This configuration file is used to store a number of configuration options, including your Synapse authtoken, cache, and multi-threading settings.
9-
10-
A full example `.synapseConfig` can be found in the [github repository](https://github.com/Sage-Bionetworks/synapsePythonClient/blob/develop/synapseclient/.synapseConfig).
9+
A full annotated example `.synapseConfig` can be found in the [GitHub repository](https://github.com/Sage-Bionetworks/synapsePythonClient/blob/develop/synapseclient/.synapseConfig).
1110

1211
## `.synapseConfig` sections
1312

14-
### `[authentication]`
13+
### `[default]` and `[profile <name>]`
14+
15+
Holds Synapse login credentials. `[default]` is used when no profile is specified; named profiles use `[profile <name>]` syntax. See the [authentication](./authentication.md) document for full details including how to create tokens, select profiles, and use environment variables.
16+
17+
### `[sftp://hostname]`
18+
19+
Credentials for files stored on SFTP servers. Use one section per server; the section name is the full SFTP URL.
20+
21+
| Key | Description |
22+
| --- | --- |
23+
| `username` | Username for the SFTP server. |
24+
| `password` | Password for the SFTP server. |
25+
26+
```ini
27+
[sftp://some.sftp.url.com]
28+
username = sftpuser
29+
password = sftppassword
30+
```
31+
32+
### `[https://s3.amazonaws.com/bucket_name]`
33+
34+
Credentials for files stored in AWS S3 or S3-compatible storage that Synapse does not manage access for. Use one section per bucket; the section name is the full endpoint URL including the bucket name.
35+
36+
| Key | Description |
37+
| --- | --- |
38+
| `profile_name` | Name of an AWS CLI profile from `~/.aws/credentials`. If omitted, the `default` AWS profile is used. |
39+
40+
```ini
41+
[https://s3.amazonaws.com/bucket_name]
42+
profile_name = local_credential_profile_name
43+
```
1544

16-
See details on this section in the [authentication](./authentication.md) document.
45+
For more information on AWS credentials files, see the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html).
1746

1847
### `[cache]`
1948

20-
Your downloaded files are cached to avoid repeat downloads of the same file. Change 'location' to use a different folder on your computer as the cache location
49+
Downloaded files are cached to avoid repeat downloads of the same file.
50+
51+
| Key | Description |
52+
| --- | --- |
53+
| `location` | Path to the cache directory. Supports `~` and environment variables. Default: `~/.synapseCache`. |
54+
55+
```ini
56+
[cache]
57+
location = ~/.synapseCache
58+
```
59+
60+
### `[debug]`
61+
62+
When this section is present (no keys required), the client prints debug-level log output. Equivalent to passing `debug=True` to the `Synapse()` constructor.
63+
64+
```ini
65+
[debug]
66+
```
2167

2268
### `[endpoints]`
2369

24-
Configuring these will cause the Python client to use these as Synapse service endpoints instead of the default prod endpoints.
70+
Override the default Synapse production service endpoints. Useful for testing against staging or development environments.
71+
72+
| Key | Description |
73+
| --- | --- |
74+
| `repoEndpoint` | Synapse repository REST API endpoint. |
75+
| `authEndpoint` | Synapse authentication service endpoint. |
76+
| `fileHandleEndpoint` | Synapse file service endpoint. |
77+
| `portalEndpoint` | Synapse web portal URL. |
78+
79+
```ini
80+
[endpoints]
81+
repoEndpoint = https://repo-prod.prod.sagebase.org/repo/v1
82+
authEndpoint = https://auth-prod.prod.sagebase.org/auth/v1
83+
fileHandleEndpoint = https://file-prod.prod.sagebase.org/file/v1
84+
portalEndpoint = https://www.synapse.org/
85+
```
2586

2687
### `[transfer]`
2788

28-
Settings to configure how Synapse uploads/downloads data.
89+
Settings to configure how Synapse uploads and downloads data.
90+
91+
| Key | Description |
92+
| --- | --- |
93+
| `max_threads` | Number of concurrent threads/connections for file transfers. Applies to AWS S3 transfers (uploads and downloads). Default: `cpu_count + 4`. Maximum: `128`. Minimum: `1`. |
94+
| `use_boto_sts` | If `true`, use AWS STS (Security Token Service) to obtain temporary credentials for S3 transfers instead of using stored AWS credentials directly. Valid values: `true` or `false` (case-insensitive). Default: `false`. |
95+
96+
```ini
97+
[transfer]
98+
max_threads = 16
99+
use_boto_sts = false
100+
```
29101

30-
You may also set the `max_threads` programmatically via:
102+
You may also set `max_threads` programmatically:
31103

32104
```python
33105
import synapseclient

synapseclient/.synapseConfig

Lines changed: 47 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,16 @@
55
## Used for logging in to Synapse. See https://python-docs.synapse.org/tutorials/authentication/
66
## for information on retrieving an auth token.
77

8+
## The [default] section is used when no profile is specified.
9+
## username is optional; if both username and authtoken are provided, they must match.
10+
811
#[default]
912
#username = default_user
1013
#authtoken = default_auth_token
1114

15+
## Named profiles can be selected via syn.login(profile="user1"),
16+
## the SYNAPSE_PROFILE environment variable, or the --profile CLI flag.
17+
1218
#[profile user1]
1319
#username = user1
1420
#authtoken = user1_auth_token
@@ -17,30 +23,37 @@
1723
#username = user2
1824
#authtoken = user2_auth_token
1925

20-
## If you have projects with file stored on SFTP servers, you can specify your credentials here
21-
## You can specify multiple sftp credentials
26+
## If you have projects with files stored on SFTP servers, you can specify your credentials here.
27+
## You can specify multiple SFTP credentials — use one section per server.
28+
2229
#[sftp://some.sftp.url.com]
23-
#username= <sftpuser>
24-
#password= <sftppwd>
30+
#username = <sftpuser>
31+
#password = <sftppwd>
2532
#[sftp://a.different.sftp.url.com]
26-
#username= <sftpuser>
27-
#password= <sftppwd>
33+
#username = <sftpuser>
34+
#password = <sftppwd>
2835

2936

30-
## If you have projects that need to be stored in an S3-like (e.g. AWS S3, Openstack) storage but cannot allow Synapse
31-
## to manage access your storage you may put your credentials here.
32-
## To avoid duplicating credentials with that used by the AWS Command Line Client,
33-
## simply put the profile name form your ~/.aws/credentials file
34-
## more information about aws credentials can be found here http://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html
35-
#[https://s3.amazonaws.com/bucket_name] # this is the bucket's endpoint
36-
#profile_name=local_credential_profile_name
37+
## If you have projects that need to be stored in an S3-like (e.g. AWS S3, OpenStack) storage
38+
## but cannot allow Synapse to manage access to your storage, you may put your credentials here.
39+
## To avoid duplicating credentials already used by the AWS CLI, specify the profile name from
40+
## your ~/.aws/credentials file. If profile_name is omitted, the "default" AWS profile is used.
41+
## More information about AWS credentials: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
42+
## Use one section per bucket; the section name is the full endpoint URL including the bucket name.
43+
44+
#[https://s3.amazonaws.com/bucket_name]
45+
#profile_name = local_credential_profile_name
3746

3847

3948
###########################
4049
# Caching #
4150
###########################
4251

43-
## your downloaded files are cached to avoid repeat downloads of the same file. change 'location' to use a different folder on your computer as the cache location
52+
## Downloaded files are cached to avoid repeat downloads of the same file.
53+
## Change 'location' to use a different folder on your computer as the cache location.
54+
## Supports ~ for the home directory and environment variable expansion.
55+
## Default: ~/.synapseCache
56+
4457
#[cache]
4558
#location = ~/.synapseCache
4659

@@ -49,20 +62,30 @@
4962
# Advanced Configurations #
5063
###########################
5164

52-
## If this section is specified, then the synapseclient will print out debug information
65+
## If this section is present, the synapseclient will print debug-level log output.
5366
#[debug]
5467

5568

56-
## Configuring these will cause the Python client to use these as Synapse service endpoints instead of the default prod endpoints.
69+
## Configuring these will cause the Python client to use these as Synapse service endpoints
70+
## instead of the default production endpoints. Useful for testing against staging environments.
5771
#[endpoints]
58-
#repoEndpoint=<repoEndpoint>
59-
#authEndpoint=<authEndpoint>
60-
#fileHandleEndpoint=<fileHandleEndpoint>
61-
#portalEndpoint=<portalEndpoint>
72+
#repoEndpoint = <repoEndpoint>
73+
#authEndpoint = <authEndpoint>
74+
#fileHandleEndpoint = <fileHandleEndpoint>
75+
#portalEndpoint = <portalEndpoint>
76+
6277

63-
## Settings to configure how Synapse uploads/downloads data
78+
## Settings to configure how Synapse uploads/downloads data.
6479
#[transfer]
6580

66-
# use this to configure the default for how many threads/connections Synapse will use to perform file transfers.
67-
# Currently this applies only to files whose underlying storage is AWS S3.
68-
# max_threads=16
81+
## max_threads: number of concurrent threads/connections used for file transfers.
82+
## Applies to AWS S3 transfers (uploads and downloads).
83+
## Default: cpu_count + 4. Maximum: 128. Minimum: 1.
84+
## Can also be set programmatically: syn.max_threads = 16
85+
#max_threads = 16
86+
87+
## use_boto_sts: if true, use AWS STS (Security Token Service) to obtain temporary
88+
## credentials for S3 transfers instead of using stored AWS credentials directly.
89+
## Useful when your storage location is configured with STS-based access.
90+
## Valid values: true or false (case-insensitive). Default: false.
91+
#use_boto_sts = false

0 commit comments

Comments
 (0)