Merge pull request #83 from VirtualPlantLab/hard-dependency-just-from-dep

Multiscale hard dependencies

Author: VEZY

docs/src/model_coupling/model_coupling_modeler.md

@@ -59,4 +59,115 @@ PlantSimEngine.dep(::Process2Model) = (process1=Process1Model,)
 
 ## Soft coupling
 
-A model that takes outputs of another model as inputs is called a soft-coupled model. There is nothing to do on the modeler side to declare a soft-dependency. The detection is done automatically by PlantSimEngine using the inputs and outputs of the models.
+A model that takes outputs of another model as inputs is called a soft-coupled model. There is nothing to do on the modeler side to declare a soft-dependency. The detection is done automatically by PlantSimEngine using the inputs and outputs of the models.
+
+## Handling dependencies in a multiscale context
+
+ If a model requires some input variable that is computed at another scale, providing the appropriate mapping will resolve name conflicts and enable proper use of that variable and there will be no extra steps for the user or the modeler.
+
+ In the case of a hard dependency that operates at a different scale from its parent, the same principle applies and there are also no extra steps on the user-side. 
+ 
+ On the other hand, modelers need to bear in mind a couple of subtleties when developing models that possess hard dependencies that operate at a different organ level from their parent : 
+
+ The parent model directly handles the call to its hard dependency model(s), meaning they are not explicitely managed by the dependency graph.
+ Therefore only the owning model of that dependency is visible in the graph, and its hard dependency nodes are internal.
+ 
+ When the caller (or any downstream model that requires some variables from the hard dependency) operates at the same scale, variables are easily accessible, and no mapping is required. 
+
+ If an inner model operates at a different scale/organ level, a modeler must declare hard dependencies with their respective organ level, similarly to the way the user provides a mapping. 
+
+ Conceptually :
+
+```julia
+ PlantSimEngine.dep(m::ParentModel) = (
+    name_provided_in_the_mapping=AbstractHardDependencyModel => ["Organ_Name_1",],
+)
+```
+
+ Here's a concrete example in [XPalm](https://github.com/PalmStudio/XPalm.jl), an oil palm model developed on top of PlantSimEngine. 
+ Organs are produced at the phytomer scale, but need to run an age model and a biomass model at the reproductive organs' scales.
+
+```julia
+ PlantSimEngine.dep(m::ReproductiveOrganEmission) = (
+    initiation_age=AbstractInitiation_AgeModel => [m.male_symbol, m.female_symbol],
+    final_potential_biomass=AbstractFinal_Potential_BiomassModel => [m.male_symbol, m.female_symbol],
+)
+```
+
+The user-mapping includes the required models at specific organ levels. Here's the relevant portion of the mapping for the male reproductive organ :
+
+```julia
+mapping = Dict(
+    ...
+    "Male" =>
+    MultiScaleModel(
+        model=XPalm.InitiationAgeFromPlantAge(),
+        mapping=[:plant_age => "Plant",],
+    ),
+    ...
+    XPalm.MaleFinalPotentialBiomass(
+        p.parameters[:male][:male_max_biomass],
+        p.parameters[:male][:age_mature_male],
+        p.parameters[:male][:fraction_biomass_first_male],
+    ),
+    ...
+)
+```
+
+The model's constructor provides convenient default names for the scale corresponding to the reproductive organs. A user may override that if their naming schemes or MTG attributes differ.
+
+```julia
+function ReproductiveOrganEmission(mtg::MultiScaleTreeGraph.Node; phytomer_symbol="Phytomer", male_symbol="Male", female_symbol="Female")
+    ...
+end
+```
+
+But how does a model M calling a hard dependency H provide H's variables when calling H's `run!` function ? The status the user provides M operates at M's organ level, so if used to call H's run! function any required variable for H will be missing.    
+
+PlantSimEngine provides what are called Status Templates in the simulation graph. Each organ level has its own Status template listing the available variables at that scale.
+So when a model M calls a hard dependency H's `run!` function, any required variables can be accessed through the status template of H's organ level.
+
+Using the same example in XPalm : 
+
+```julia
+# Note that the function's 'status' parameter does NOT contain the variables required by the hard dependencies as the calling model's organ level is "Phytomer", not "Male" or "Female"
+
+function PlantSimEngine.run!(m::ReproductiveOrganEmission, models, status, meteo, constants, sim_object)
+    ...
+    status.graph_node_count += 1
+
+    # Create the new organ as a child of the phytomer:
+    st_repro_organ = add_organ!(
+        status.node[1], # The phytomer's internode is its first child 
+        sim_object,  # The simulation object, so we can add the new status 
+        "+", status.sex, 4;
+        index=status.phytomer_count,
+        id=status.graph_node_count,
+        attributes=Dict{Symbol,Any}()
+    )
+
+    # Compute the initiation age of the organ:
+    PlantSimEngine.run!(sim_object.models[status.sex].initiation_age, sim_object.models[status.sex], st_repro_organ, meteo, constants, sim_object)
+    PlantSimEngine.run!(sim_object.models[status.sex].final_potential_biomass, sim_object.models[status.sex], st_repro_organ, meteo, constants, sim_object)
+end
+```
+
+In the above example the organ and its status template are created on the fly.
+When that isn't the case, the status template can be accessed through the simulation graph :
+
+```julia
+function PlantSimEngine.run!(m::ReproductiveOrganEmission, models, status, meteo, constants, sim_object)
+
+    ...
+
+    if status.sex == "Male"
+
+        status_male = sim_object.statuses["Male"][1]
+        run!(sim_object.models["Male"].initiation_age, models, status_male, meteo, constants, sim_object)
+        run!(sim_object.models["Male"].final_potential_biomass, models, status_male, meteo, constants, sim_object)
+    else
+        # Female
+        ...
+    end
+end
+```

src/dependencies/dependencies.jl

@@ -88,12 +88,12 @@ function dep(mapping::Dict{String,T}; verbose::Bool=true) where {T}
     # First step, get the hard-dependency graph and create SoftDependencyNodes for each hard-dependency root. In other word, we want 
     # only the nodes that are not hard-dependency of other nodes. These nodes are taken as roots for the soft-dependency graph because they
     # are independant.
-    soft_dep_graphs_roots = hard_dependencies(mapping; verbose=verbose)
+    soft_dep_graphs_roots, hard_dep_dict = hard_dependencies(mapping; verbose=verbose)
     # Second step, compute the soft-dependency graph between SoftDependencyNodes computed in the first step. To do so, we search the 
     # inputs of each process into the outputs of the other processes, at the same scale, but also between scales. Then we keep only the
     # nodes that have no soft-dependencies, and we set them as root nodes of the soft-dependency graph. The other nodes are set as children
     # of the nodes that they depend on.
-    dep_graph = soft_dependencies_multiscale(soft_dep_graphs_roots, mapping)
+    dep_graph = soft_dependencies_multiscale(soft_dep_graphs_roots, mapping, hard_dep_dict)
     # During the building of the soft-dependency graph, we identified the inputs and outputs of each dependency node, 
     # and also defined **inputs** as MappedVar if they are multiscale, i.e. if they take their values from another scale.
     # What we are missing is that we need to also define **outputs** as multiscale if they are needed by another scale.

src/dependencies/dependency_graph.jl

@@ -8,7 +8,7 @@ mutable struct HardDependencyNode{T} <: AbstractDependencyNode
     scale::String
     inputs
     outputs
-    parent::Union{Nothing,HardDependencyNode}
+    parent::Union{Nothing,<:AbstractDependencyNode}
     children::Vector{HardDependencyNode}
 end
 
@@ -39,9 +39,9 @@ A graph of dependencies between models.
 - `roots::T`: the root nodes of the graph.
 - `not_found::Dict{Symbol,DataType}`: the models that were not found in the graph.
 """
-struct DependencyGraph{T}
+struct DependencyGraph{T,N}
     roots::T
-    not_found::Dict{Symbol,DataType}
+    not_found::Dict{Symbol,N}
 end
 
 # Add methods to check if a node is parallelizable:

src/dependencies/get_model_in_dependency_graph.jl

@@ -28,4 +28,16 @@ function get_model_nodes(dep_graph::DependencyGraph, model)
     end
 
     return model_node
+end
+
+function get_model_nodes(dep_graph::DependencyGraph, process::Symbol)
+    process_node = Union{SoftDependencyNode,HardDependencyNode}[]
+
+    traverse_dependency_graph!(dep_graph) do node
+        if node.process == process
+            push!(process_node, node)
+        end
+    end
+
+    return process_node
 end