|
1 | 1 |
|
2 | 2 | # DiDa.jl — simple work with distributed data |
3 | 3 |
|
4 | | -This packages provides a relatively simple Distributed Data manipulation and |
5 | | -processing routines for Julia. |
| 4 | +This packages provides simple Distributed Data manipulation and processing |
| 5 | +routines for Julia. |
6 | 6 |
|
7 | 7 | The design of the package and data manipulation approach is deliberately |
8 | 8 | "imperative" and "hands-on", to allow as much user influence on the actual way |
9 | | -the data are moved and stored in the cluster as possible. It uses the |
10 | | -`Distributed` package and its infrastructure of workers, and provides a few |
11 | | -very basic primitives that lightly wrap the `Distributed` package functions |
12 | | -`remotecall` and `fetch`. |
| 9 | +the data are moved and stored in the cluster as possible. It is based on the |
| 10 | +[`Distributed`](https://docs.julialang.org/en/v1/stdlib/Distributed/) package |
| 11 | +and its infrastructure of remote workers. The basic `Distributed` package |
| 12 | +functions `remotecall` and `fetch` are then wrapper (very lightly) to create a |
| 13 | +simple yet powerful data manipulation interface. |
13 | 14 |
|
14 | 15 | There are also various extra functions to easily run distributed data |
15 | | -transformations, MapReduce-style algorithms, store and load the data on worker |
16 | | -local storage (e.g. to prevent memory exhaustion) and others. |
| 16 | +transformations, MapReduce-style algorithms, store and load the data remotely |
| 17 | +on worker-local storage (e.g. to prevent memory exhaustion) and others. |
17 | 18 |
|
18 | 19 | To start quickly, you can read the tutorial: |
19 | 20 |
|
|
0 commit comments