Add custom hook

Author: FreeYourSoul

CHANGELOG.md

@@ -1,7 +1,12 @@
 # Changelogs
 
+## Version 1.0.0 
+(Released : April 2, 2021) : tag 1.0.0
+* Complete documentation
+* Test and implement custom visitor hook
 
-## Version 1.0.0
+## Version 0.0.0
+(Alpha released : March 27, 2021) : tag 0.1.0
 * First implementation: 
     * Execution graph implementation
     * Visitor implementation
@@ -15,9 +20,9 @@
 
 ## Documentation todo
 
-* Add doc about node / graph creation
-* Add documentation about graph nodes
-* Add documentation about custom hook
+* ~~Add doc about node / graph creation~~
+* ~~Add documentation about graph nodes~~
+* ~~Add documentation about custom hook~~
 
 ## Feature todo
 
@@ -26,4 +31,4 @@
 ## Testing todo
 
 * ~~Test Composed Visitor~~
-* Custom hook
+* ~~Custom hook~~

README.md

@@ -29,6 +29,8 @@ It provide:
 * [networkx](https://networkx.org/) : FreExGraph is a layer on top of networkx 
 * [tqdm](https://github.com/tqdm/tqdm) : provide a nice way to display progression on visitation
 
+---
+
 # Documentation
 
 The goal of this library is to provide a standardized way to represent an execution graph and to visit it via visitor. Some visitors are provided by FreExGraph but the more important thing is the ability to very easily provide its own visitor and it's own node type.
@@ -39,7 +41,7 @@ The goal of this library is to provide a standardized way to represent an execut
 
 In order to use FreExGraph properly, it is required to provide your own node type. A node is a class that inherit from FreExNode class. It provides the following interface:
 ```python
-from freeexgraph import FreExNode, AbstractVisitor
+from freexgraph import FreExNode, AbstractVisitor 
 
 class MyCustomNode(FreExNode):
   def accept(self, visitor: AbstractVisitor) -> bool:
@@ -62,6 +64,7 @@ Here is a complete example that can be used as a good start to show how you shou
 ```python
 
 # We have Three different kind of node : Alpha, Beta, Gamma
+from freexgraph import FreExNode, AbstractVisitor 
 
 class MyNodeAlpha(FreExNode):
   def accept(self, visitor: AbstractVisitor) -> bool:
@@ -97,7 +100,7 @@ class MyBaseVisitor(AbstractVisitor):
 # My actual Visitor implementation, this one just need a specific action 
 # to be made on MyNodeGamma node type. The rest has a default behaviour.
 
-class CustomVisitor(MyBaseVisitor)
+class CustomVisitor(MyBaseVisitor):
   def visit_default(self, node: FreExNode):
     print("This is the behavior for Alpha and Beta nodes")
     return True
@@ -114,19 +117,98 @@ class CustomVisitor(MyBaseVisitor)
 
 After creating node types (see above), we can create an execution graph that will use those nodes.
 
+A graph is created with the class `FreExGraph`, which doesn't require any parameter, as follow:
+```python
+from freexgraph import FreExGraph 
+
+execution_graph = FreExGraph()
+```
+
 > it is important to note that the character ':' is forbidden in node id, as it is used for internal node (fork uniqueness and/or root_node)
 
-**add_node:**
+When the graph instance is done, you need to add nodes to them. The two following methods makes it possible.
 
-**add_nodes:**
+**add_node:** Of the method of signature `add_node(self, node: FreExNode)`. This method will add a node to the graph. If no parents are set in the given node, 
+the node will be linked with the root node that is automatically generated by the FreExGraph. If parents are set, edges will be made. 
+* Node id HAS to not be present in the graph 
+* A node's parent HAS to already be present in the graph. 
+* The id of the graph cannot contains ':' which is a forbidden character.
+* No infinite looping of the node are accepted.
 
+In the case any of those rules is not respected. An Assertion error is thrown.
+
+example:
+```python
+# The graph below would look like that.
+#
+#            id1
+#             |
+#            id2 ____.
+#             |       \
+#             |       id4
+#             |       / |
+#            id3 ----'  |
+#             |         |
+#            id5 -------` 
+#
+execution_graph = FreExGraph()
+id1 = f"id1"
+id2 = f"id2"
+id3 = f"id3"
+id4 = f"id4"
+id5 = f"id5"
+
+execution_graph.add_node(FreExNode(id1))
+execution_graph.add_node(FreExNode(id2, parents={id1}))
+execution_graph.add_node(FreExNode(id4, parents={id2}))
+execution_graph.add_node(FreExNode(id3, parents={id2, id4}))
+execution_graph.add_node(FreExNode(id5, parents={id4, id3}))
+```
+
+**add_nodes:**: It can be cumbersome to ensure the ordering of the nodes you want to add (with the parents order). In order to avoid this issue, you can use add_nodes with the signature `add_nodes(self, nodes: List[AnyFreExNode])`.  
+This method is going to re-order the nodes depending on their parents in order to add them (calling add_node internally) properly.
+`add_node` being called. All the rules applicable on add_node has to be respected with add_nodes (unicity of id and so on...)
+
+
+example
+```python
+# we want to make the same graph as the one tested above for add_node
+execution_graph = FreExGraph()
+node_1 = FreExNode(id1)
+node_2 = FreExNode(id2, parents={id1})
+node_3 = FreExNode(id3, parents={id2, id4}
+node_4 = FreExNode(id4, parents={id2})
+node_5 = FreExNode(id5, parents={id4, id3}))
+
+# adding them in complete disorder is okay
+execution_graph.add_nodes([node_5, node_2, node_1, node_3, node_4])
+```
 
 ### Graph Node
 
 It is possible to embed a graph into another thanks to a graph node. Any visitation going through a graph node is going to be propagated to the inner graph.
 
-< TODO >
+To make one you need to create a FreExGraph (that will be in the GraphNode).  
+example :
 
+```python
+    # We will do the following graph
+#
+#           id1___,
+#            |     \
+#            |     id_graph (graph_node)
+#            |     /
+#           id2---`
+#
+execution_graph = FreExGraph()
+
+inner_graph = FreExGraph()
+# fill inner_graph as you want 
+
+execution_graph.add_node(NodeForTest(id1))
+execution_graph.add_node(GraphNode(id_graph, parents={id1}, graph=inner_graph))
+execution_graph.add_node(NodeForTest(idc, parents={id1, id_graph}))
+```
 
 ### Fork
 
@@ -149,7 +231,7 @@ _Given the following graph in `execution_graph`:_
 #
 
 # If we decide to fork (with the fork id f1) from the node id4
-valid_basic_execution_graph.fork_from_node(FreExNode(uid="id4", fork_id="f1"))
+execution_graph.fork_from_node(FreExNode(uid="id4", fork_id="f1"))
 
 # The result would be the following:
 
@@ -170,27 +252,35 @@ Obviously if you want the node id4::f1 (which is of type FreExNode as you asked)
 
 It is also possible to provide a join node. It will be a node used as join for the fork in order to not have to fork the whole graph from the source of the fork. It is usefull if you have to multiply a big chunk of execution graph because one node has to change some internal values (in experimentation fields, it can be useful for parameter explorations).
 
-> **IN CASE OF MAP REDUCE CASES DO NOT USE FORKS**. It is possible to do so with a join_id.. But it is prefered to do manually your map reduce (with add_node / add_nodes) than to use fork.
+> **Try avoiding forks** : This is a mecanism that can be useful in certain cases (the main one would be parameter exploration on an experimentation) But when it comes to map reduce for example, it is advised to manually fo the nodes you want instead (improve readibility of what you are doing when making your graph). A chaining of fork can start being very hard to understand for the user.
 
-Join is do-able by adding the
+But if you want to do a map reduce with a fork, it is do-able by setting the join_id to the `fork_from_node` method. The join_id has to be an existing node on which, for every parents that are part of the fork has only this join node as child.
+See [test using this mechanism](https://github.com/FreeYourSoul/FreExGraph/blob/ae707cf0fcb8486bde783cd0c7fe67217a56b3d2/test/fork_test.py#L41-L66) for more details
 
 ## Visitors
 
 ### Abstract Visitor hooks
 
 Abstract visitor provide some default hooks that can be overridden from custom visitors in order to implement more complex logic depending on the graph visit.
-* `hook_start()` : This hook is called when the visitation of the graph start (an interesting way to use this hook could be to reinitialize your visitor in case of re-use).
-* `hook_end()` : This hook is called when the visitation of the graph end.
-* `hook_fork_started(n: FreExNode, fork_id: str)` : This hook is called when a fork has been entered (when visiting the first node of a fork).
+* `hook_start()`: This hook is called when the visitation of the graph start (an interesting way to use this hook could be to reinitialize your visitor in case of re-use).
+* `hook_end()`: This hook is called when the visitation of the graph end.
 * `hook_start_graph_node(gn: GraphNode)` : This hook is called when a graphnode recursion start (graph node given as parameter of the hook).
 * `hook_end_graph_node(gn: GraphNode)` : This hook is called when a graphnode visitation end (graph node given as parameter of the hook).
 
-Custom hooks can be implemented if you must trigger a specific action that depend on the business data stored in your node.   
-To do so, in the `__init__` of your custom visitor, use the method `register_custom_hook(predicate: Callable, hook: Callable)` : This method will trigger the provided hook if the given predicate return true for a node.   
+Custom hooks can be implemented on any visitor if you want to trigger a specific action that depend on the business data stored in your node.   
+
+To do so, on a visitor instance, use the method `register_custom_hook(predicate: Callable, hook: Callable)` : This method will trigger the provided hook if the given predicate return true for a node. Custom hooks are all executed just before the visitation of each node.   
 
 **Example of a custom hook:** 
 ```python
-# TODO
+visitor_test.register_custom_hook(
+    predicate=lambda n: n.id.startswith("id3"), hook=hook
+)
+visitor_test.register_custom_hook(
+    predicate=lambda n: not n.id.startswith("id3"),
+    hook=hook_2,
+)
+visitor_test.visit(execution_graph.root)
 ```
 
 ### Standard Visitor provided by FreExGraph
@@ -243,13 +333,28 @@ v = ValidateGraphIntegrity()
 # visitation does assertion to verify if the graph is correct
 v.visit(graph_above.root())
 ```
-Those visitor implement the start hook that reinitialize their state as if they were new freshly created visitor. Which is why an instance of a standard visitor can be re-used. To implement this kind of behaviour in your own visitors, check out [visitor hooks](#abstract-visitor-hooks).
+Those visitors implement the start hook that reinitialize their state as if they were new freshly created visitor. Which is why an instance of a standard visitor can be re-used. To implement this kind of behaviour in your own visitors, check out [visitor hooks](#abstract-visitor-hooks).
+
+### Progression visit
+
+tqdm is implemented in AbstractVisitor. 
+It makes possible to have a progress bar while visiting your graph. To do so, when instantiating the AbstractVisitor from `super()` within your custom visitor.
+Just set the boolean parameter `with_progress_bar` in the constructor to True.  
+```python
+class MyCustomVisitor(AbstractVisitor):
+    def __init__(self):
+        super().__init__(with_progress_bar=True)
+
+    # rest of your custom visitor implementation
+    ...
+```
 
 ### Reverse visit
-It is possible to revert the order of visitation of your visitor by setting its attribute `is_reversed` of your visitor (works with any kind of visitor), example :
+It is possible to revert the order of visitation of any visitor by setting its attribute `is_reversed` of the visitor (works with any type of visitor inheriting from AbstractVisitor).  
+example :
 ```python
 # we assume a graph called `execution_graph` that represent the following graph:
-# id_1 ---> id_2 ---> id_3 ---> id4 ---> id3bis ---> id5
+# id1 ---> id2 ---> id3 ---> id4 ---> id3bis ---> id5
 
 v = FindFirstVisitor(lambda node: node.id.startswith("id3"))
 
@@ -263,6 +368,8 @@ assert v.found()
 assert v.result.id == "id3bis"
 ```
 
+---
+
 ## Installation
 
 * Via pip

freexgraph/__init__.py

@@ -25,9 +25,13 @@
 Module to manipulate an execution graph
 """
 
-from freexgraph._version import __version__
-
-from freexgraph.freexgraph import FreExGraph, FreExNode, AnyVisitor
+from freexgraph.freexgraph import GraphNode, FreExGraph, FreExNode, AnyVisitor
 from freexgraph.visitor import AbstractVisitor, VisitorComposer
 
 import freexgraph.standard_visitor
+
+
+def version() -> str:
+    from freexgraph._version import __version__
+
+    return __version__

freexgraph/_version.py

@@ -21,4 +21,4 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-__version__ = "0.1.0"
+__version__ = "1.0.0"

freexgraph/freexgraph.py

@@ -218,7 +218,7 @@ def add_node(self, node: AnyFreExNode) -> None:
         """
         assert (
             node.fork_id is not None or ":" not in node.id
-        ), f"Node cannot contains a ':' in its name {node.id}"
+        ), f"Node cannot contains a ':' in its id {node.id}"
         assert not self._graph.has_node(
             node.id
         ), f"{node.id} is already in the execution graph"