Skip to content

Commit fb23c2f

Browse files
committed
updating documentation
1 parent dfbc9b6 commit fb23c2f

2 files changed

Lines changed: 28 additions & 19 deletions

File tree

docs/source/analyzing_instances.rst

Lines changed: 22 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,39 @@
11
.. _traces-label:
22

33
Analyzing Instances
4-
================
4+
===================
55

66
Workflow execution instances have been widely used to profile and characterize
77
workflow executions, and to build distributions of workflow execution behaviors,
88
which are used to evaluate methods and techniques in simulation or in real
99
conditions.
1010

11-
The first axis of the WfCommons project targets the analysis of actual workflow
12-
execution instances (i.e., the workflow execution profile data and characterizations)
13-
in order to build **recipes** of workflow applications. These recipes contain
14-
the necessary information for generating synthetic, yet realistic, workflow
15-
instances that resemble the structure and distribution of the original workflow
16-
executions.
11+
The WfCommons project targets the analysis of actual workflow execution instances
12+
(i.e., the workflow execution profile data and characterizations)
13+
in order to build :ref:`workflow-recipe-label` of workflow applications.
14+
These recipes contain the necessary information for generating synthetic, yet
15+
realistic, workflow instances that resemble the structure and distribution of
16+
the original workflow executions.
1717

1818
A `list of workflow execution instances <https://wfcommons.org/instances>`_
1919
that are compatible with :ref:`json-format-label` is kept constantly updated
2020
in our project website.
2121

22-
Workflow Execution Instances
23-
-------------------------
22+
.. _wfinstances-label:
23+
24+
WfInstances
25+
-----------
2426

2527
A workflow execution instance represents an actual execution of a scientific
2628
workflow on a distributed platform (e.g., clouds, grids, HPC, etc.). In the
2729
WfCommons project, an instance is represented in a JSON file following the
28-
schema described in :ref:`json-format-label` section. This Python package
30+
schema described in :ref:`json-format-label`. This Python package
2931
provides an *instance loader* tool for importing workflow execution instances
3032
for analysis. For instance, the code snippet below shows how an instance can
3133
be loaded using the :class:`~wfcommons.trace.trace.Trace` class: ::
3234

3335
from wfcommons import Trace
34-
trace = Trace(input_trace='/path/to/trace/file.json')
36+
trace = Trace(input_trace='/path/to/instance/file.json')
3537

3638
The :class:`~wfcommons.trace.trace.Trace` class provides a number of
3739
methods for interacting with the workflow instance, including:
@@ -41,16 +43,21 @@ methods for interacting with the workflow instance, including:
4143
- :meth:`~wfcommons.trace.trace.Trace.roots`: gets the roots of the workflow (i.e., the tasks without any predecessors).
4244
- :meth:`~wfcommons.trace.trace.Trace.write_dot`: writes a dot file of the instance.
4345

46+
.. note::
47+
Although the analysis methods are inherently used by WfCommons (specifically
48+
WfChef) for :ref:`generating-workflows-recipe-label`, they can also be used
49+
in a standalone manner.
50+
4451
The Instance Analyzer
45-
------------------
52+
---------------------
4653

4754
The :class:`~wfcommons.trace.trace_analyzer.TraceAnalyzer` class provides
4855
a number of tools for analyzing collection of workflow execution instances. The
4956
goal of the :class:`~wfcommons.trace.trace_analyzer.TraceAnalyzer` is to
5057
perform analyzes of one or multiple workflow execution instances, and build
5158
summaries of the analyzes per workflow' task type prefix.
5259

53-
.. note::
60+
.. warning::
5461

5562
Although any workflow execution instance represented as a
5663
:class:`~wfcommons.trace.trace.Trace` object (i.e., compatible with
@@ -89,7 +96,7 @@ summary showing the best fit probability distribution for runtime of the
8996
...
9097
}
9198

92-
Workflow analysis summaries can then be used to develop :ref:`workflow-recipe-label`,
99+
Workflow analysis summaries are used by WfChef to develop :ref:`workflow-recipe-label`,
93100
in which themselves are used to :ref:`generate realistic synthetic workflow instances
94101
<generating-workflows-label>`.
95102

@@ -113,7 +120,7 @@ plots (runtime, and input and output files) into the :code:`fits` folder using
113120
from os.path import isfile, join
114121

115122
# obtaining list of instance files in the folder
116-
INSTANCES_PATH = "/Path/to/some/instance/folder/"
123+
INSTANCES_PATH = "/path/to/some/instance/folder/"
117124
instance_files = [f for f in listdir(INSTANCES_PATH) if isfile(join(INSTANCES_PATH, f))]
118125

119126
# creating the instance analyzer object

docs/source/parsing_logs.rst

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@
33
Parsing Workflow Execution Logs
44
===============================
55

6-
The most common way for obtaining instances from actual workflow executions is to parse
7-
execution logs. As part of the WfCommons project, we are constantly developing
8-
parsers for commonly used workflow management systems.
6+
The most common way for obtaining **workflow instances** from actual workflow
7+
executions is to parse execution logs. As part of the WfCommons project, we
8+
are constantly developing parsers for commonly used workflow management systems.
9+
The parsers provided in this Python package automatically scans execution logs
10+
to produce instances using :ref:`json-format-label`.
911

1012
Each parser class is derived from the abstract
1113
:class:`~wfcommons.trace.logs.abstract_logs_parser.LogsParser` class. Thus, each
@@ -18,7 +20,7 @@ Makeflow
1820

1921
`Makeflow <http://ccl.cse.nd.edu/software/makeflow/>`_ is a workflow system for
2022
executing large complex workflows on clusters, clouds, and grids. The Makeflow
21-
language is similar to traditional Make, so if you can write a Makefile, then you
23+
language is similar to traditional "Make", so if you can write a Makefile, then you
2224
can write a Makeflow. A workflow can be just a few commands chained together, or
2325
it can be a complex application consisting of thousands of tasks. It can have an
2426
arbitrary DAG structure and is not limited to specific patterns. Makeflow is used

0 commit comments

Comments
 (0)