Skip to content

Commit dfbc9b6

Browse files
rafaelfsilvarafaelfsilva
committed
updating documentation
1 parent adb6e08 commit dfbc9b6

3 files changed

Lines changed: 45 additions & 114 deletions

File tree

‎docs/source/index.rst‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,8 @@
55

66
`WfCommons <https://wfcommons.org>`__ is a community framework for
77
enabling scientific workflow research and development. This Python package
8-
provides a collection of tools for:
9-
10-
- Analyzing instances of actual workflow executions;
11-
- Producing recipes structures for creating workflow recipes for workflow
12-
generation; and
13-
- Generating synthetic realistic workflow instances.
8+
provides methods for analyzing instances, deriving recipes, and generating
9+
representative synthetic workflow instances.
1410

1511
.. figure:: images/wfcommons.png
1612
:scale: 90 %

‎docs/source/introduction.rst‎

Lines changed: 43 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,60 +1,67 @@
11
The WfCommons Project
22
=======================
33

4-
The `WfCommons project <https://wfcommons.org>`_ is a community framework
4+
The `WfCommons project <https://wfcommons.org>`_ is an open source framework
55
for enabling scientific workflow research and development by providing foundational
66
tools for analyzing workflow execution instances, and generating synthetic, yet
77
realistic, workflow instances that can be used to develop new techniques, algorithms
88
and systems that can overcome the challenges of efficient and robust execution of
9-
ever larger workflows on increasingly complex distributed infrastructures. The
10-
figure below shows an overview of the workflow research life cycle process that
11-
integrates the three axis of the WfCommons project:
9+
ever larger workflows on increasingly complex distributed infrastructures.
10+
11+
The figure below shows an overview of the research and development life cycle that
12+
integrates the four major components WfCommons: (i) workflow execution instances
13+
(**WfInstances**), (ii) workflow recipes (**WfChef**), (iii) workflow generator
14+
(**WfGen**), and (iv) workflow simulator (**WfSim**).
1215

1316
.. figure:: images/wfcommons.png
1417
:align: center
1518

1619
The WfCommons conceptual architecture.
1720

18-
The *first axis* (**Workflow Instances**) of the WfCommons project targets the
19-
collection and curation of open access production workflow executions from
20-
various scientific applications shared in a common instance format (i.e.,
21-
:ref:`json-format-label`). We keep a `list of workflow execution instances
21+
**WfInstances.**
22+
The WfInstances component provides a collection and curation of open-access
23+
production workflow instances from various scientific applications, all made
24+
available using a common format (i.e., :ref:`json-format-label`).
25+
A workflow instance is built based on logs of an actual execution of a scientific
26+
workflow on a distributed platform (e.g., clouds, grids, clusters) using a
27+
workflow system. We keep a `list of workflow execution instances
2228
<https://wfcommons.org/instances>`_ in our project website.
2329

24-
The *second axis* (**Workflow Generator**) of the WfCommons project targets
25-
the generation of realistic synthetic workflow instances based on workflow execution
26-
profiles extracted from execution instances. We are constantly seeking for additional
27-
workflow execution instances for refining or developing new workflow recipes for
28-
the WfCommons's workflow generator.
29-
30-
The *third axis* (**Workflow Simulator**) of the WfCommons project fosters the
31-
use of simulation for the development, evaluation, and verification of scheduling
32-
and resource provisioning algorithms (e.g., multi-objective function optimization,
33-
etc.), evaluation of current and emerging computing platforms (e.g., clouds, IoT,
34-
extreme scale, etc.), among others. We keep a `list of open source workflow
35-
management systems simulators and simulation frameworks
36-
<https://wfcommons.org/simulation>`_ that provide support for the WfCommons
37-
JSON format in our project website.
30+
**WfChef.**
31+
The WfChef component automates the construction of synthetic workflow generators
32+
(recipes) for any given workflow application. The input to this component is a set
33+
of real workflow instances described in the :ref:`json-format-label` (e.g.,
34+
instances available in WfInstances).
3835

39-
This Python package provides a collection of tools for:
36+
**WfGen.**
37+
The WfGen component targets the generation of realistic synthetic workflow instances.
38+
WfGen takes as input a workflow recipe produced by WfChef for a particular application
39+
and a desired number of tasks. WfGen then automatically generates synthetic, yet
40+
realistic, randomized workflow instances with (approximately) the desired number of
41+
tasks.
4042

41-
- Analyzing instances of actual workflow executions;
42-
- Producing recipes structures for creating workflow recipes for workflow
43-
generation; and
44-
- Generating synthetic realistic workflow instances.
43+
**WfSim.**
44+
The WfCommons project fosters the use of simulation for the development, evaluation,
45+
and verification of scheduling and resource provisioning algorithms (e.g.,
46+
multi-objective function optimization, etc.), evaluation of current and emerging
47+
computing platforms (e.g., clouds, IoT, extreme scale, etc.), among others.
48+
We do not develop simulators as part of the WfCommons project. Instead, the WfSim
49+
component catalogs open source WMS simulators that provide support for
50+
:ref:`json-format-label`. We keep a `list of open source workflow
51+
management systems simulators and simulation frameworks
52+
<https://wfcommons.org/simulation>`_ on our project website.
4553

4654
.. _json-format-label:
4755

48-
The WfCommons JSON Format
49-
---------------------------
56+
WfFormat
57+
--------
5058

5159
The WfCommons project uses a common format for representing workflow execution
52-
instances and generated synthetic workflows instances, so that workflow simulators and
53-
simulation frameworks (that provide support for WfCommons format) can use
54-
such instances interchangeably. This common format uses a JSON specification
55-
available in the
56-
`WfCommons JSON schema GitHub <https://github.com/wfcommons/workflow-schema>`_
60+
instances and generated synthetic workflows instances. Workflow simulators and
61+
simulation frameworks that support WfFormat can then use both types of instances
62+
interchangeably. WfFormat uses a JSON specification available in the
63+
`WfFormat Schema GitHub <https://github.com/wfcommons/workflow-schema>`_
5764
repository. The current version of the WfCommons Python package uses the schema
5865
version :code:`1.1`. The schema GitHub repository provides detailed explanation
59-
of the WfCommons JSON format (including required fields), and also a validator
60-
script for verifying the compatibility of instances.
66+
of WfFormat (including required fields), and also a validator script for verifying
67+
the compatibility of instances.

‎docs/source/user_api_generator.rst‎

Lines changed: 0 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -13,75 +13,3 @@ wfcommons.generator.generator
1313
:members:
1414
:undoc-members:
1515
:show-inheritance:
16-
17-
wfcommons.generator.workflow.blast\_recipe
18-
--------------------------------------------
19-
20-
.. automodule:: wfcommons.generator.workflow.blast_recipe
21-
:members:
22-
:undoc-members:
23-
:show-inheritance:
24-
25-
wfcommons.generator.workflow.bwa\_recipe
26-
------------------------------------------
27-
28-
.. automodule:: wfcommons.generator.workflow.bwa_recipe
29-
:members:
30-
:undoc-members:
31-
:show-inheritance:
32-
33-
wfcommons.generator.workflow.cycles\_recipe
34-
---------------------------------------------
35-
36-
.. automodule:: wfcommons.generator.workflow.cycles_recipe
37-
:members:
38-
:undoc-members:
39-
:show-inheritance:
40-
41-
wfcommons.generator.workflow.epigenomics\_recipe
42-
--------------------------------------------------
43-
44-
.. automodule:: wfcommons.generator.workflow.epigenomics_recipe
45-
:members:
46-
:undoc-members:
47-
:show-inheritance:
48-
49-
wfcommons.generator.workflow.genome\_recipe
50-
---------------------------------------------
51-
52-
.. automodule:: wfcommons.generator.workflow.genome_recipe
53-
:members:
54-
:undoc-members:
55-
:show-inheritance:
56-
57-
wfcommons.generator.workflow.montage\_recipe
58-
----------------------------------------------
59-
60-
.. automodule:: wfcommons.generator.workflow.montage_recipe
61-
:members:
62-
:undoc-members:
63-
:show-inheritance:
64-
65-
wfcommons.generator.workflow.seismology\_recipe
66-
-------------------------------------------------
67-
68-
.. automodule:: wfcommons.generator.workflow.seismology_recipe
69-
:members:
70-
:undoc-members:
71-
:show-inheritance:
72-
73-
wfcommons.generator.workflow.soykb\_recipe
74-
--------------------------------------------
75-
76-
.. automodule:: wfcommons.generator.workflow.soykb_recipe
77-
:members:
78-
:undoc-members:
79-
:show-inheritance:
80-
81-
wfcommons.generator.workflow.srasearch\_recipe
82-
------------------------------------------------
83-
84-
.. automodule:: wfcommons.generator.workflow.srasearch_recipe
85-
:members:
86-
:undoc-members:
87-
:show-inheritance:

0 commit comments

Comments
 (0)