Skip to content

Commit 693d01b

Browse files
thomasyu888claude
andauthored
[SYNPY-1508]: add tutorial for downloading files by Synapse ID concurrenty (#1337)
* SYNPY-1508: add tutorial for downloading files by Synapse ID concurrently Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Update download data tutorial * update tutorial doc * Update script name --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent e06c715 commit 693d01b

4 files changed

Lines changed: 111 additions & 0 deletions

File tree

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
[](){ #tutorial-downloading-a-file }
2+
# Downloading data by Synapse ID
3+
4+
This tutorial shows how to download any set of files from Synapse using their
5+
Synapse IDs. Rather than syncing an entire project or folder, this approach lets
6+
you target exactly the files you need and download them **concurrently** — even
7+
directing each file to a different local directory.
8+
9+
10+
## Tutorial Purpose
11+
In this tutorial you will:
12+
13+
1. Build a mapping of Synapse IDs to local download directories
14+
1. Download all files concurrently using the async API
15+
16+
17+
## Prerequisites
18+
* Make sure that you have completed the following tutorials:
19+
* [Folder](./folder.md)
20+
* [File](./file.md)
21+
* The target directories (`~/temp/subdir1`, etc.) must exist before running the
22+
script. Create them or replace them with directories of your choice.
23+
24+
25+
## 1. Build a mapping of Synapse IDs to download directories
26+
27+
Create a dictionary that maps each Synapse ID to the local path where that file
28+
should be saved. Files can be directed to different directories as needed.
29+
30+
```python
31+
{!docs/tutorials/python/tutorial_scripts/download_data_by_synid.py!lines=13-30}
32+
```
33+
34+
35+
## 2. Download all files concurrently
36+
37+
Use `File.get_async()` together with `asyncio.gather` to kick off every download
38+
at the same time and wait for them all to finish.
39+
40+
```python
41+
{!docs/tutorials/python/tutorial_scripts/download_data_by_synid.py!lines=31-43}
42+
```
43+
44+
<details class="example">
45+
<summary>After all downloads finish you'll see output like:</summary>
46+
```
47+
Retrieved 12 files
48+
```
49+
</details>
50+
51+
52+
## Source code for this tutorial
53+
54+
<details class="quote">
55+
<summary>Click to show me</summary>
56+
57+
```python
58+
{!docs/tutorials/python/tutorial_scripts/download_data_by_synid.py!}
59+
```
60+
</details>
61+
62+
## References used in this tutorial
63+
64+
- [File][synapseclient.models.File]
65+
- [File.get_async][synapseclient.models.File.get_async]
66+
- [syn.login][synapseclient.Synapse.login]
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
"""
2+
Here is where you'll find the code for the downloading files by synapse ids tutorial.
3+
"""
4+
5+
import asyncio
6+
7+
from synapseclient import Synapse
8+
from synapseclient.models import File
9+
10+
syn = Synapse()
11+
syn.login()
12+
13+
# A mapping of Synapse IDs to the local directory each file should be downloaded to.
14+
# Files can be directed to different directories as needed.
15+
SYN_IDS_AND_PATHS = {
16+
"syn60584250": "~/temp/subdir1",
17+
"syn60584256": "~/temp/subdir1",
18+
"syn60584248": "~/temp/subdir1",
19+
"syn60584252": "~/temp/subdir1",
20+
"syn60584258": "~/temp/subdir1",
21+
"syn60584260": "~/temp/subdir1",
22+
"syn60584257": "~/temp/subdir1",
23+
"syn60584251": "~/temp/subdir1",
24+
"syn60584253": "~/temp/subdir1",
25+
"syn60584390": "~/temp/subdir1",
26+
"syn60584405": "~/temp/subdir2",
27+
"syn60584400": "~/temp/subdir3",
28+
}
29+
30+
31+
async def main():
32+
# Build a list of concurrent download tasks — one per Synapse ID
33+
tasks = []
34+
for syn_id, path in SYN_IDS_AND_PATHS.items():
35+
tasks.append(File(id=syn_id, path=path).get_async())
36+
37+
# Download all files concurrently and wait for every one to finish
38+
results = await asyncio.gather(*tasks)
39+
40+
print(f"Retrieved {len(results)} files")
41+
42+
43+
asyncio.run(main())

docs/tutorials/python_client.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ By the end of these tutorials you'll have:
3030
- A [Team](./python/team.md) created with one or more members
3131
- Methods to [upload data in bulk](./python/upload_data_in_bulk.md)
3232
- Methods to [download data in bulk](./python/download_data_in_bulk.md)
33+
- Methods to [download files by Synapse ID](./python/download_data_by_synid.md)
3334
- Methods to [move files and folders](./python/move_files_and_folders.md)
3435
- Methods to [migrate data to other storage locations](./python/migrate_data_to_other_storage_locations.md)
3536

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ nav:
4545
# - Team: tutorials/python/team.md
4646
- Upload data in bulk: tutorials/python/upload_data_in_bulk.md
4747
- Download data in bulk: tutorials/python/download_data_in_bulk.md
48+
- Download data by Synapse ID: tutorials/python/download_data_by_synid.md
4849
# - Creating JSON Schema: tutorials/python/schema_operations.md
4950
- Working with JSON Schema: tutorials/python/json_schema.md
5051
# - Move Files and Folders: tutorials/python/move_files_and_folders.md

0 commit comments

Comments
 (0)