Skip to content

Commit 532ceb3

Browse files
thomasyu888claudeCopilot
authored
[SYNPY-1480]: Improve documentation around configuration file (#1350)
* [SYNPY-1480]: Improve documentation around configuration file - Expand .synapseConfig example with improved comments for all sections - Document previously undocumented `use_boto_sts` transfer option - Improve `max_threads` comment with default, min, and max (128) values - Rewrite docs/tutorials/configuration.md with full table-driven coverage of every section including sftp, S3, cache, debug, endpoints, transfer - Remove duplication between configuration.md and authentication.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Address PR comments * Add note block --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 875c8f5 commit 532ceb3

2 files changed

Lines changed: 132 additions & 34 deletions

File tree

‎docs/tutorials/configuration.md‎

Lines changed: 85 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,32 +2,107 @@
22

33
The Synapse Python client can be configured either programmatically or by using a configuration file.
44

5-
**The default configuration file does not need to be modified for most use-cases**.
5+
!!! note "Default Configuration"
6+
The default configuration file does not need to be modified for most use-cases
67

8+
When installing the Synapse Python client, the `.synapseConfig` file is added to your home directory if it doesn't exist already. This file stores configuration options including your Synapse auth token, cache location, multi-threading settings, and storage credentials.
79

8-
When installing the Synapse Python client, the `.synapseConfig` is added to your home directory. This configuration file is used to store a number of configuration options, including your Synapse authtoken, cache, and multi-threading settings.
9-
10-
A full example `.synapseConfig` can be found in the [github repository](https://github.com/Sage-Bionetworks/synapsePythonClient/blob/develop/synapseclient/.synapseConfig).
10+
A full annotated example `.synapseConfig` can be found in the [GitHub repository](https://github.com/Sage-Bionetworks/synapsePythonClient/blob/develop/synapseclient/.synapseConfig).
1111

1212
## `.synapseConfig` sections
1313

14-
### `[authentication]`
14+
### `[default]` and `[profile <name>]`
15+
16+
Holds Synapse login credentials. `[default]` is used when no profile is specified; named profiles use `[profile <name>]` syntax. See the [authentication](./authentication.md) document for full details including how to create tokens, select profiles, and use environment variables.
17+
18+
### `[sftp://hostname]`
19+
20+
Credentials for files stored on SFTP servers. Use one section per server; the section name is the full SFTP URL.
21+
22+
| Key | Description |
23+
| --- | --- |
24+
| `username` | Username for the SFTP server. |
25+
| `password` | Password for the SFTP server. |
26+
27+
```ini
28+
[sftp://some.sftp.url.com]
29+
username = sftpuser
30+
password = sftppassword
31+
```
1532

16-
See details on this section in the [authentication](./authentication.md) document.
33+
### `[https://s3.amazonaws.com/bucket_name]`
34+
35+
Credentials for files stored in AWS S3 or S3-compatible storage that Synapse does not manage access for. Use one section per bucket; the section name is the full endpoint URL including the bucket name.
36+
37+
| Key | Description |
38+
| --- | --- |
39+
| `profile_name` | Name of an AWS CLI profile from `~/.aws/credentials`. If omitted, the `default` AWS profile is used. |
40+
41+
```ini
42+
[https://s3.amazonaws.com/bucket_name]
43+
profile_name = local_credential_profile_name
44+
```
45+
46+
For more information on AWS credentials files, see the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html).
1747

1848
### `[cache]`
1949

20-
Your downloaded files are cached to avoid repeat downloads of the same file. Change 'location' to use a different folder on your computer as the cache location
50+
Downloaded files are cached to avoid repeat downloads of the same file.
51+
52+
| Key | Description |
53+
| --- | --- |
54+
| `location` | Path to the cache directory. Supports `~` and environment variables. Default: `~/.synapseCache`. |
55+
56+
```ini
57+
[cache]
58+
location = ~/.synapseCache
59+
```
60+
61+
### `[debug]`
62+
63+
When this section is present (no keys required), the client prints debug-level log output. Equivalent to passing `debug=True` to the `Synapse()` constructor.
64+
65+
```ini
66+
[debug]
67+
```
2168

2269
### `[endpoints]`
2370

24-
Configuring these will cause the Python client to use these as Synapse service endpoints instead of the default prod endpoints.
71+
Override the default Synapse production service endpoints. Useful for testing against staging or development environments.
72+
73+
| Key | Description |
74+
| --- | --- |
75+
| `repoEndpoint` | Synapse repository REST API endpoint. |
76+
| `authEndpoint` | Synapse authentication service endpoint. |
77+
| `fileHandleEndpoint` | Synapse file service endpoint. |
78+
| `portalEndpoint` | Synapse web portal URL. |
79+
80+
Note: The following are the default endpoints.
81+
82+
```ini
83+
[endpoints]
84+
repoEndpoint = https://repo-prod.prod.sagebase.org/repo/v1
85+
authEndpoint = https://auth-prod.prod.sagebase.org/auth/v1
86+
fileHandleEndpoint = https://file-prod.prod.sagebase.org/file/v1
87+
portalEndpoint = https://www.synapse.org/
88+
```
2589

2690
### `[transfer]`
2791

28-
Settings to configure how Synapse uploads/downloads data.
92+
Settings to configure how Synapse uploads and downloads data.
93+
94+
| Key | Description |
95+
| --- | --- |
96+
| `max_threads` | Number of concurrent threads/connections for file transfers. Applies to AWS S3 transfers (uploads and downloads). Default: `min(cpu_count + 4, 128)`. Maximum: `128`. Minimum: `1`. |
97+
| `use_boto_sts` | If `true`, use AWS STS (Security Token Service) to obtain temporary credentials for S3 transfers instead of using stored AWS credentials directly. Valid values: `true` or `false` (case-insensitive). Default: `false`. |
98+
99+
```ini
100+
[transfer]
101+
max_threads = 16
102+
use_boto_sts = false
103+
```
29104

30-
You may also set the `max_threads` programmatically via:
105+
You may also set `max_threads` programmatically:
31106

32107
```python
33108
import synapseclient

‎synapseclient/.synapseConfig‎

Lines changed: 47 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,16 @@
55
## Used for logging in to Synapse. See https://python-docs.synapse.org/tutorials/authentication/
66
## for information on retrieving an auth token.
77

8+
## The [default] section is used when no profile is specified.
9+
## username is optional; if both username and authtoken are provided, they must match.
10+
811
#[default]
912
#username = default_user
1013
#authtoken = default_auth_token
1114

15+
## Named profiles can be selected via syn.login(profile="user1"),
16+
## the SYNAPSE_PROFILE environment variable, or the --profile CLI flag.
17+
1218
#[profile user1]
1319
#username = user1
1420
#authtoken = user1_auth_token
@@ -17,30 +23,37 @@
1723
#username = user2
1824
#authtoken = user2_auth_token
1925

20-
## If you have projects with file stored on SFTP servers, you can specify your credentials here
21-
## You can specify multiple sftp credentials
26+
## If you have projects with files stored on SFTP servers, you can specify your credentials here.
27+
## You can specify multiple SFTP credentials — use one section per server.
28+
2229
#[sftp://some.sftp.url.com]
23-
#username= <sftpuser>
24-
#password= <sftppwd>
30+
#username = <sftpuser>
31+
#password = <sftppwd>
2532
#[sftp://a.different.sftp.url.com]
26-
#username= <sftpuser>
27-
#password= <sftppwd>
33+
#username = <sftpuser>
34+
#password = <sftppwd>
2835

2936

30-
## If you have projects that need to be stored in an S3-like (e.g. AWS S3, Openstack) storage but cannot allow Synapse
31-
## to manage access your storage you may put your credentials here.
32-
## To avoid duplicating credentials with that used by the AWS Command Line Client,
33-
## simply put the profile name form your ~/.aws/credentials file
34-
## more information about aws credentials can be found here http://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html
35-
#[https://s3.amazonaws.com/bucket_name] # this is the bucket's endpoint
36-
#profile_name=local_credential_profile_name
37+
## If you have projects that need to be stored in an S3-like (e.g. AWS S3, OpenStack) storage
38+
## but cannot allow Synapse to manage access to your storage, you may put your credentials here.
39+
## To avoid duplicating credentials already used by the AWS CLI, specify the profile name from
40+
## your ~/.aws/credentials file. If profile_name is omitted, the "default" AWS profile is used.
41+
## More information about AWS credentials: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
42+
## Use one section per bucket; the section name is the full endpoint URL including the bucket name.
43+
44+
#[https://s3.amazonaws.com/bucket_name]
45+
#profile_name = local_credential_profile_name
3746

3847

3948
###########################
4049
# Caching #
4150
###########################
4251

43-
## your downloaded files are cached to avoid repeat downloads of the same file. change 'location' to use a different folder on your computer as the cache location
52+
## Downloaded files are cached to avoid repeat downloads of the same file.
53+
## Change 'location' to use a different folder on your computer as the cache location.
54+
## Supports ~ for the home directory and environment variable expansion.
55+
## Default: ~/.synapseCache
56+
4457
#[cache]
4558
#location = ~/.synapseCache
4659

@@ -49,20 +62,30 @@
4962
# Advanced Configurations #
5063
###########################
5164

52-
## If this section is specified, then the synapseclient will print out debug information
65+
## If this section is present, the synapseclient will print debug-level log output.
5366
#[debug]
5467

5568

56-
## Configuring these will cause the Python client to use these as Synapse service endpoints instead of the default prod endpoints.
69+
## Configuring these will cause the Python client to use these as Synapse service endpoints
70+
## instead of the default production endpoints. Useful for testing against staging environments.
5771
#[endpoints]
58-
#repoEndpoint=<repoEndpoint>
59-
#authEndpoint=<authEndpoint>
60-
#fileHandleEndpoint=<fileHandleEndpoint>
61-
#portalEndpoint=<portalEndpoint>
72+
#repoEndpoint = <repoEndpoint>
73+
#authEndpoint = <authEndpoint>
74+
#fileHandleEndpoint = <fileHandleEndpoint>
75+
#portalEndpoint = <portalEndpoint>
76+
6277

63-
## Settings to configure how Synapse uploads/downloads data
78+
## Settings to configure how Synapse uploads/downloads data.
6479
#[transfer]
6580

66-
# use this to configure the default for how many threads/connections Synapse will use to perform file transfers.
67-
# Currently this applies only to files whose underlying storage is AWS S3.
68-
# max_threads=16
81+
## max_threads: number of concurrent threads/connections used for file transfers.
82+
## Applies to AWS S3 transfers (uploads and downloads).
83+
## Default: min(cpu_count + 4, 128). Maximum: 128. Minimum: 1.
84+
## Can also be set programmatically: syn.max_threads = 16
85+
#max_threads = 16
86+
87+
## use_boto_sts: if true, use AWS STS (Security Token Service) to obtain temporary
88+
## credentials for S3 transfers instead of using stored AWS credentials directly.
89+
## Useful when your storage location is configured with STS-based access.
90+
## Valid values: true or false (case-insensitive). Default: false.
91+
#use_boto_sts = false

0 commit comments

Comments
 (0)