You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+19-11Lines changed: 19 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,29 +1,27 @@
1
1
# Pipeline Component System (PCS)
2
2
3
-
TODO
4
-
5
3
A strange programming framework
6
4
7
5
The name is inspired by Entity Component System (ECS)
8
6
9
7
## Why?
10
8
11
-
Why create this? I often find myself not liking the programs I create, and then end up rewriting them to be better, but they still end up quite brittle. This is a programming framework to make code cleaner and hopefully more maintainable. Have I succeeded in my goal? I have been using this framework for some time now and it has definitely helped my development speed and mental fatigue a lot! So yes, it has helped me achieve my goal!
9
+
I often find myself not liking the programs I create, and then end up rewriting them to be better, but they still end up quite brittle. This is a programming framework to make code cleaner and hopefully more maintainable. Have I succeeded in my goal? I have been using this framework for some time now and it has definitely helped my development speed and mental fatigue a lot! So yes, it has helped me achieve my goal!
12
10
13
11
## Introduction
14
12
15
-
I will first discuss the few simple components which make up this framework, then connect them together, explaining choices I took along the way. If you wish to see an example of how this all ties together, look at `examples/example.py`.
13
+
I will first discuss the few simple elements which make up this framework, then connect them together, explaining choices I took along the way. If you wish to see an example of how this all ties together, look at `examples/example.py`.
16
14
17
15
### Component
18
16
19
-
Think of the component as your global database. Each piece of persistent data (literal or object) is stored here. It is a dataclass, and it is the only dataclass (unless you want to nest them ofcourse). The reason for this design choice is that this way, we ALWAYS know where the data is. We do not have to guess which class owns what, like in those OOP messes.
17
+
Think of the component as your global database. Each piece of persistent data (static or dyamic) is stored here. It is a dataclass, and it is the only dataclass (unless you want to nest them ofcourse). The reason for this design choice is that this way, we ALWAYS know where the data is. We do not have to guess which class owns what, unlike OOP soups.
20
18
21
-
The Component distinguishes between 2 data types: config (constant / defined at the start then not changed after initialization) and runtime (dynamic data which changes during runtime). The config variables can only be of primitive types (a restriction which comes from omegaconf, which this project depends on). Whereas the config class is
19
+
The Component distinguishes between 2 data types: `conf` (constant / static / defined at the start then remains read-only after initialization) and `runtime` (dynamic data which changes during runtime). The config variables can only be of primitive types (a restriction which comes from omegaconf, which this project depends on).
22
20
23
21
```python
24
22
@dataclass
25
23
classConfig: # Note: the name is not important
26
-
i: int# Only primitive types in the config class
24
+
i: int# Only primitive types in the config class (whatever OmegaConf is capable of)
27
25
f: float
28
26
s: str
29
27
result: float
@@ -33,6 +31,7 @@ class Dynamic:
33
31
di: int# Dynamic class can also take complex types
34
32
35
33
data = parse_arguments_cli(Config, Dynamic)
34
+
data.seal() # Makes the `Config` part of the component read-only
36
35
37
36
print(data.i) # Print's the Config class' 'i'
38
37
print(data.di) # Print's the Runtime class' 'i'
@@ -67,25 +66,34 @@ A pipeline takes a component, and a list of systems, then automatically passes t
pipeline.execute() # Execute pipeline a second time
76
79
```
77
80
78
81
When a system returns a dictionary, the keys of the dict are interpreted to be the names of the component variables to replace with the value of the respective key. So the final 2 systems in the Systems examples will replace the `result` field.
79
82
80
-
Note that this helps us avoid having to pass parameters around, as it is done automatically for us, which cleans up the code base tremendously, as we have a concise pipeline definition, and when we call `Pipeline.execute`, we execute the 3 functions.
83
+
Note that this helps us avoid having to pass parameters around, as it is done automatically for us, which cleans up the code base tremendously, as we have a concise pipeline definition, and when we call `Pipeline.execute`, we execute the 3 functions sequentially.
81
84
82
85
### Other handy tools
83
86
84
-
`parse_arguments_cli` will read your argvs using argparse and give you a component object ready to use. So you may run your file like so: `file.py --args-files="file1.yaml,file2.yaml" --rest a=1 --rest b=2`. Consecutive files will overwrite the previous entries, and `--rest` take precendence always, but each `--rest` takes precedence over the previous.
87
+
`parse_arguments_cli` will read your `argv`s using argparse and give you a component object ready to use. So you may run your file like so: `file.py --args-files="file1.yaml,file2.yaml" --rest a=1 -r b=2`. Consecutive files will overwrite the previous entries, and `--rest/-r` take precendence always, but each `--rest` takes precedence over the previous.
0 commit comments