Skip to content

unpack_singularity builds and immediately discards an intermediate SIF #506

Description

@agitter

Given the proot problems in #505, we may want to reconsider how we pull and then build sif files

spras/spras/containers.py

Lines 473 to 484 in 07707bf

# Adding 'docker://' to the container indicates this is a Docker image Singularity must convert
image_path = Client.pull('docker://' + resolved.image, name=str(unpacked_dir / sif_file))
base_cont_path = unpacked_dir / Path(base_cont)
# Check whether the directory for base_cont_path already exists. When running concurrent jobs, it's possible
# something else has already pulled/unpacked the container.
# Here, we expand the sif image from `image_path` to a directory indicated by `base_cont_path`
if not base_cont_path.exists():
Client.build(recipe=image_path, image=str(base_cont_path), sandbox=True, sudo=False)
print(f'Resolved singularity image to sandbox: {base_cont_path}', flush=True)
return base_cont_path # sandbox directory

We could use a single Client.build instead.

The two stage version we have could also introduce a race condition.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions