Merge pull request #27 from score-p/scorep_5.0

Scorep 5.0

Author: AndreasGocht

.gitmodules

@@ -1,3 +0,0 @@
-[submodule "scorep_substrate_vampir_groups_writer"]
-	path = scorep_substrate_vampir_groups_writer
-	url = https://github.com/score-p/scorep_substrate_vampir_groups_writer.git

README.md

@@ -2,7 +2,8 @@
 scorep is a module that allows tracing of python scripts using [Score-P](http://www.vi-hps.org/projects/score-p/).
 
 # Install
-You need to have scorep-config in your `PATH` variable.
+You need at least Score-P 5.0, build with `--enable-shared`.
+Please make sure, that `scorep-config` is in your `PATH` variable.
 
 Then simply run
 ```
@@ -26,22 +27,27 @@ and
 
 There is also a small [HowTo](https://github.com/score-p/scorep_binding_python/wiki) in the wiki.
 
-# Compatibility
-## Working
-* python3 
-* python2.7
-* mpi using mpi4py
+Since version 0.9 it is possible to pass the traditional Score-P commands to the Score-P bindings, e.g.:
 
-## Not Working
-* python multiprocessing
-    * Score-P does currently not support any non MPI or non SHMEM communication. So the different processes will not know from each other. You might want to take a look to https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html .
+```
+python -m scorep --mpp=mpi --thread=pthread <script.py>
+```
 
-## Partialy working
-* threaded applications
-    * Please have a look to a Score-P trunk package with revision at least 13560 here:
-      http://scorepci.pages.jsc.fz-juelich.de/scorep-pipelines/.
-    
-# User instrumentation
+To see all flags simply call:
+
+```
+scorep --help
+```
+
+## MPI
+
+To use trace an MPI parallel application, please specify
+
+```
+python -m scorep --mpp=mpi <script.py>
+```
+
+## User instrumentation
 
 In some cases, the user might want to define a region, log some parameters, or just disable tracing for a certain area. To do so the module implements a few functions:
 
@@ -52,7 +58,6 @@ scorep.user.region_end(name)
 
 These functions allow the definition of user regions. `name` defines the name of a region. Each `user.region_begin` shall have a corresponding call to `user.region_end`.    
 
-
 ```
 scorep.user.enable_recording()
 scorep.user.disable_recording()
@@ -67,16 +72,52 @@ scorep.user.parameter_string(name, string)
 ```
 
 These functions allow passing user parameters to Score-P. These parameters can be int, uint and string. `name` defines the name of the parameter, while `val` or `string` defines the value that is passed to Score-P. 
+Please be aware, that the scorep module still needs to be preloaded, i.e. by using `python -m scorep`.
 
-# Tracing
-The tracing uses Score-P User instrumentation. The python trace module was reworked, to pass the region names to ```SCOREP_USER_REGION_BEGIN``` and ```SCOREP_USER_REGION_END``` instead of printing. All other features of the tracing module where striped.
+## Additional Flags
 
-# MPI
+The Score-P bindings now also support a `--nopython` flag, which disables the python instrumentation.
+This might be helpful, if only user instrumentation is required, or only some instrumented libraries shall be traced.
 
-This version of the Score-P python tracing plugin supports MPI. To use it please specify `--mpi`:
+## Backward Compatibility
+
+In order to maintain backwards Compatibility, the following flags are set per default:
 
 ```
-python -m scorep --mpi <script.py>
+python -m scorep --compiler --thread=pthread <script.py>
 ```
 
-This will ensure, that the right Score-P libs are in `LD_PRELOAD`. It will re-execute the tracing script, with the preloaded libs.
+The traditional `--mpi` does still work, and is similar to the following call:
+
+```
+python -m scorep --compiler --thread=pthread --mpp=mpi <script.py>
+```
+
+To disable compiler instrumentation please specify:
+
+```
+python -m scorep --nocompiler <script.py>
+```
+
+For other thread schemes just specify `--thread=<something>`. E.g. :
+
+```
+python -m scorep --thread=omp <script.py>
+```
+
+Please be aware the `--user` is always passed to Score-P, as this is needed for the python instrumentation.
+
+# Compatibility
+## Working
+* python3 
+* python2.7
+* mpi using mpi4py
+* threaded applications
+
+
+## Not Working
+* python multiprocessing
+    * Score-P does currently not support any non MPI or non SHMEM communication. So the different processes will not know from each other. You might want to take a look to https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html .
+
+# Tracing
+The tracing uses Score-P User instrumentation. The python trace module was reworked, to pass the region names to `SCOREP_USER_REGION_BEGIN` and `SCOREP_USER_REGION_END` instead of printing. All other features of the tracing module where striped.

scorep/__init__.py

@@ -1 +1 @@
-__all__ = ["trace", "scorep_bindings", "user"]
+__all__ = ["trace", "scorep_bindings", "user", "subsystem"]

scorep/__main__.py

@@ -5,97 +5,102 @@
 
 import scorep.trace
 import scorep.helper
+import scorep.subsystem
 
 
 def _usage(outfile):
     outfile.write("""TODO
 """ % sys.argv[0])
 
 
-global_trace = None
-
 cuda_support = None
 opencl_support = None
 
+
 def _err_exit(msg):
     sys.stderr.write("%s: %s\n" % ("scorep", msg))
     sys.exit(1)
 
 
-def set_init_environment(mpi):
+def set_init_environment(scorep_config=[], keep_files=False):
     """
     Set the inital needed environmet variables, to get everythin up an running.
     As a few variables interact with LD env vars, the programms needs to be restarted after this.
     The function set the env var `SCOREP_PYTHON_BINDINGS_INITALISED` to true, once it is done with
     initalising.
 
-    @param mpi indicates if mpi is used.
+    @param scorep_config configuration flags for score-p
+    @param keep_files whether to keep the generated files, or not.
+    @return temp_dir to be deleted once the script is done
     """
 
-    """
-    look for vampir_groups_writer, which will produce a groups file for vampir.
-    This allows collored traces.
-    """
-    vampir_groups_writer_lib = "libscorep_substrate_vampir_groups_writer.so"
-    vampir_groups_writer = scorep.helper.find_lib(vampir_groups_writer_lib)
-
-    if vampir_groups_writer:
-        scorep.helper.add_to_ld_library_path(
-            os.path.dirname(vampir_groups_writer))
-        if ("SCOREP_SUBSTRATE_PLUGINS" not in os.environ) or (
-                os.environ["SCOREP_SUBSTRATE_PLUGINS"] == ""):
-            os.environ["SCOREP_SUBSTRATE_PLUGINS"] = "vampir_groups_writer"
-        else:
-            os.environ["SCOREP_SUBSTRATE_PLUGINS"] += ",vampir_groups_writer"
+    if ("LD_PRELOAD" in os.environ) and (
+            "libscorep" in os.environ["LD_PRELOAD"]):
+        raise RuntimeError(
+            "Score-P is already loaded. This should not happen at this point")
 
-    if mpi:
-        if ("LD_PRELOAD" not in os.environ) or (
-                "libscorep" not in os.environ["LD_PRELOAD"]):
+    subsystem_lib_name, temp_dir = scorep.subsystem.generate(
+        scorep_config, keep_files)
+    scorep_ld_preload = scorep.helper.generate_ld_preload(scorep_config)
 
-            ld_preload, scorep_subsystem = scorep.helper.generate_ld_preload()
-            scorep.helper.add_to_ld_library_path(
-                os.path.dirname(scorep_subsystem))
-
-            os.environ["LD_PRELOAD"] = ld_preload + " " + os.environ["LD_PRELOAD"]
+    scorep.helper.add_to_ld_library_path(temp_dir)
 
+    preload_str = scorep_ld_preload + " " + subsystem_lib_name
+    if "LD_PRELOAD" in os.environ:
+        print("LD_PRELOAD is already specified. If Score-P is already loaded this might lead to errors.", file=sys.stderr)
+        preload_str = preload_str + " " + os.environ["LD_PRELOAD"]
+    os.environ["LD_PRELOAD"] = preload_str
     os.environ["SCOREP_PYTHON_BINDINGS_INITALISED"] = "true"
 
 
 def scorep_main(argv=None):
-    global target_code
+    # print(sys.flags)
     if argv is None:
         argv = sys.argv
-    try:
-        opts, prog_argv = getopt.getopt(argv[1:], "v",
-                                        ["help", "version", "mpi"])
-
-    except getopt.error as msg:
-        sys.stderr.write("%s: %s\n" % (sys.argv[0], msg))
-        sys.stderr.write("Try `%s --help' for more information\n"
-                         % sys.argv[0])
-        sys.exit(1)
-
-    mpi = False
 
-    for opt in opts:
-        key, value = opt
-        if key == "--help":
-            _usage(sys.stdout)
-            sys.exit(0)
+    scorep_config = []
+    prog_argv = []
+    parse_scorep_commands = True
+
+    keep_files = False
+    no_default_threads = False
+    no_default_compiler = False
+    no_python = False
+
+    for elem in argv[1:]:
+        if parse_scorep_commands:
+            if elem == "--mpi":
+                scorep_config.append("--mpp=mpi")
+            elif elem == "--keep-files":
+                keep_files = True
+            elif "--thread=" in elem:
+                scorep_config.append(elem)
+                no_default_threads = True
+            elif elem == "--nocompiler":
+                scorep_config.append(elem)
+                no_default_compiler = True
+            elif elem == "--nopython":
+                no_python = True
+            elif elem[0] == "-":
+                scorep_config.append(elem)
+            else:
+                prog_argv.append(elem)
+                parse_scorep_commands = False
+        else:
+            prog_argv.append(elem)
 
-        if key == "--version":
-            sys.stdout.write("scorep_trace 1.0\n")
-            sys.exit(0)
+    if not no_default_threads:
+        scorep_config.append("--thread=pthread")
 
-        if key == "--mpi":
-            mpi = True
+    if not no_default_compiler:
+        scorep_config.append("--compiler")
 
     if len(prog_argv) == 0:
-        _err_exit("missing name of file to run")
+        _err_exit("Did not find a script to run")
 
     if ("SCOREP_PYTHON_BINDINGS_INITALISED" not in os.environ) or (
             os.environ["SCOREP_PYTHON_BINDINGS_INITALISED"] != "true"):
-        set_init_environment(mpi)
+        set_init_environment(scorep_config, keep_files)
 
         """
         python -m starts the module as skript. i.e. sys.argv will loke like:
@@ -111,23 +116,14 @@ def scorep_main(argv=None):
                 new_args.append(elem)
         os.execve(sys.executable, new_args, os.environ)
 
-    scorep_bindings = None
-    if mpi:
-        try:
-            scorep_bindings = importlib.import_module(
-                "scorep.scorep_bindings_mpi")
-        except ModuleNotFoundError:
-            raise ModuleNotFoundError(
-                "The MPI bindings are missing. Did you build Score-P with '--without-mpi'?")
-    else:
-        scorep_bindings = importlib.import_module("scorep.scorep_bindings")
+    scorep_bindings = importlib.import_module("scorep.scorep_bindings")
 
     # everything is ready
     sys.argv = prog_argv
     progname = prog_argv[0]
     sys.path[0] = os.path.split(progname)[0]
 
-    global_trace = scorep.trace.ScorepTrace(scorep_bindings, True)
+    tracer = scorep.trace.ScorepTrace(scorep_bindings, not no_python)
     try:
         with open(progname) as fp:
             code = compile(fp.read(), progname, 'exec')
@@ -139,32 +135,30 @@ def scorep_main(argv=None):
             '__package__': None,
             '__cached__': None,
         }
-        
-        global_trace.runctx(code, globs, globs)
+
+        tracer.runctx(code, globs, globs)
     except OSError as err:
         _err_exit("Cannot run file %r because: %s" % (sys.argv[0], err))
-    except SystemExit:
-        pass
+    finally:
+        scorep.subsystem.clean_up(keep_files)
+
 
 def main(argv=None):
     import traceback
     call_stack = traceback.extract_stack()
     call_stack_array = traceback.format_list(call_stack)
     call_stack_string = ""
     for elem in call_stack_array[:-1]:
-        call_stack_string+=elem
-    _err_exit("Someone called scorep.__main__.main(argv).\n"
-              "This is not supposed to happen, but might be triggered, if your application calls \"sys.modules['__main__'].main\".\n"
-              "This python stacktrace might be helpfull to find the reason:\n%s" % call_stack_string)
+        call_stack_string += elem
+    _err_exit(
+        "Someone called scorep.__main__.main(argv).\n"
+        "This is not supposed to happen, but might be triggered, if your application calls \"sys.modules['__main__'].main\".\n"
+        "This python stacktrace might be helpfull to find the reason:\n%s" %
+        call_stack_string)
+
 
 if __name__ == '__main__':
     scorep_main()
-
 else:
-    '''
-    If Score-P is not intialised using the tracing module (`python -m scorep <script.py>`),
-    we need to make sure that, if a user call gets called, scorep is still loaded.
-    Moreover, if the module is loaded with `import scorep` we can't do any mpi support anymore
-    '''
-    scorep_bindings = importlib.import_module("scorep.scorep_bindings")
-    global_trace = scorep.trace.ScorepTrace(scorep_bindings, False)
+    if ("SCOREP_PYTHON_BINDINGS_INITALISED" not in os.environ) or (os.environ["SCOREP_PYTHON_BINDINGS_INITALISED"] != "true"):
+        print("scorep needs to be loaded using \"python -m scorep <script>\". Please be aware that scorep might not work correctly!", file=sys.stderr)