Skip to content

Fix to be able to execute qmap config file specifying cores or memory for a single job #3

@dmartmillan

Description

@dmartmillan

It seems that when we want to execute a qmap script specifying the number of cores or the memory to use for a paarticular job it doesn't run it with them and it runs with the general parameters. Here, it's an example of a jobs file:

[pre]
. "/home/$USER/miniconda3/etc/profile.d/conda.sh"
conda activate biomarkers

[params]
cores = 7
memory = 32G

[jobs]
python $PWD/launch_pad/engine2/get_cnv-features.py
python $PWD/launch_pad/engine2/get_hla-features.py
python $PWD/launch_pad/engine2/get_somatic-features.py ## cores=40

The code that it gets the parameters is this one:

qmap/qmap/manager.py

Lines 328 to 333 in c73c355

for i, c in job.items():
cmd = c.split('##')
params = None
if len(cmd) > 1: # if the job has specific _params
params = jobs_file.parse_inline_parameters(cmd[1]) # job specific _params
job_list.append((i, SubmittedJob(i, self.out_folder, cmd[0].strip(), params, pre_commands=pre, post_commands=post)))

It generates the .env file for the specific job but it seems that it isn't running accordingly the parameters. Also, it generates the .bash file but it doesn't get the right .env. An exampe of a .bash file generated:

#!/bin/bash
#SBATCH --no-requeue
set -e

source "/home/dmartinezm/hartwig_biomarkers/fire_engine2_20220530/execution.env"
if [ -f "/home/dmartinezm/hartwig_biomarkers/fire_engine2_20220530/1.env" ]; then
	source "/home/dmartinezm/hartwig_biomarkers/fire_engine2_20220530/1.env"
fi

. "/home/$USER/miniconda3/etc/profile.d/conda.sh"
conda activate biomarkers
python $PWD/launch_pad/engine2/get_hla-features.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions