dags/llms.txt at main · OpenSourceEconomics/dags · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
# dags

> A Python library for creating executable DAGs (Directed Acyclic Graphs) from interdependent functions. It automatically determines execution order based on function signatures and enables efficient composition of complex computational pipelines.

dags works by analyzing function signatures to build a dependency graph. If a function has a parameter named `x`, and another function is named `x`, then the first function depends on the second. This allows you to build combined functions at runtime without specifying the execution order in advance.

## Core API

- [concatenate_functions](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/dag.py): Combine multiple functions into a single callable that executes them in the correct order based on their dependencies. Supports multiple return types (tuple, list, dict), optional aggregation, and signature enforcement.
- [create_dag](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/dag.py): Build a networkx.DiGraph representing the dependency structure of functions without creating the combined function.
- [get_ancestors](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/dag.py): Find all functions that a target depends on (with optional `include_targets` flag).

## Signatures

- [get_free_arguments](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/annotations.py): Get parameter names of a function, excluding partialled arguments. Lives in `annotations.py` but is about function signatures.
- [rename_arguments](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/signature.py): Rename function parameters using a mapping dict, useful for integrating functions with different naming conventions. Can be used as a decorator.

## Annotations

- [get_annotations](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/annotations.py): Get type annotations from a function, handling functools.partial and Python 3.14 edge cases. Returns a dict including `"return"` key.

## Tree Utilities (`dags.tree`)

The `dags.tree` module provides utilities for working with nested dictionary structures using qualified names (`"__"` delimiter) and tree paths (tuples).

### Conversion Functions

- [QNAME_DELIMITER](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): The delimiter used for qualified names (default: `"__"`).
- [qname_from_tree_path](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Convert a tree path tuple to a qualified name string.
- [tree_path_from_qname](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Convert a qualified name string to a tree path tuple.
- [flatten_to_qnames](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Convert nested dict to flat dict with qualified names (e.g., `{"a": {"b": 1}}` → `{"a__b": 1}`).
- [unflatten_from_qnames](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Convert flat qualified names back to nested dict.
- [flatten_to_tree_paths](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Flatten nested dict to flat dict with tuple keys.
- [unflatten_from_tree_paths](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Convert flat dict with tuple keys back to nested dict.
- [qnames](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Return list of qualified name keys from a nested dict.
- [tree_paths](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/tree_utils.py): Return list of tree path keys from a nested dict.

### Tree DAG Functions

- [concatenate_functions_tree](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/dag_tree.py): Like `concatenate_functions` but works with nested function dicts, takes nested inputs, and returns nested results.
- [create_dag_tree](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/dag_tree.py): Create a DAG from nested function dictionaries and input structure.
- [create_tree_with_input_types](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/dag_tree.py): Create a nested input structure template with type annotations based on functions and targets.
- [functions_without_tree_logic](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/dag_tree.py): Flatten a nested function dict and rename all parameters to qualified absolute names, producing a flat dict suitable for `concatenate_functions`.
- [one_function_without_tree_logic](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/dag_tree.py): Convert a single function from relative parameter names to qualified absolute names given its tree path and namespace.

### Tree Validation

- [fail_if_paths_are_invalid](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/validation.py): Validate all tree paths for trailing underscores and repeated top-level elements.
- [fail_if_path_elements_have_trailing_underscores](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/validation.py): Check that no non-leaf path element ends with an underscore.
- [fail_if_top_level_elements_repeated_in_paths](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/validation.py): Fail if any top-level namespace element appears deeper in the hierarchy.
- [fail_if_top_level_elements_repeated_in_single_path](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/validation.py): Same check for a single path.

## Exceptions

- [DagsError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Base exception for all dags-specific errors.
- [CyclicDependencyError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Raised when the DAG contains a cycle.
- [MissingFunctionsError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Raised when target functions are not provided.
- [AnnotationMismatchError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Raised when type annotations conflict between functions.
- [NonStringAnnotationError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Raised when a non-string annotation is encountered (with `set_annotations=True`).
- [InvalidFunctionArgumentsError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Raised when there's an issue with function signatures or arguments (e.g., too many positional args, duplicate args).
- [ValidationError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/exceptions.py): Base exception for validation errors (parent of tree validation exceptions).
- [RepeatedTopLevelElementError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/exceptions.py): Raised when top-level elements are repeated in tree paths.
- [TrailingUnderscoreError](https://github.com/OpenSourceEconomics/dags/blob/main/src/dags/tree/exceptions.py): Raised when path elements have trailing underscores.

## Quick Example

```python
import dags

def a(x):
    return x ** 2

def b(a):
    return a + 1

def c(a, b):
    return a + b

combined = dags.concatenate_functions(
    functions={"a": a, "b": b, "c": c},
    targets=["c"],
    return_type="dict",
)

result = combined(x=5)  # Returns {"c": 51}
```

## Installation

```bash
pip install dags
```

## Optional

- [Source Code](https://github.com/OpenSourceEconomics/dags/tree/main/src/dags): Full source code repository.
- [Tests](https://github.com/OpenSourceEconomics/dags/tree/main/tests): Test suite with usage examples.