Skip to content

Docker adaptor can retrieve an image from the local docker service#246

Draft
mih wants to merge 1 commit into
datalad:masterfrom
mih:bf-199
Draft

Docker adaptor can retrieve an image from the local docker service#246
mih wants to merge 1 commit into
datalad:masterfrom
mih:bf-199

Conversation

@mih

@mih mih commented Oct 10, 2023

Copy link
Copy Markdown
Member

Closes: #199

@bpinsard would be great if you could give this a spin!

Demo

❯ datalad create busydemo
❯ cd busydemo
❯ datalad containers-add -u dhub://busybox:latest busy
❯ datalad drop .datalad/environments --reckless availability
❯ git annex info | grep 'local annex'
local annex keys: 0
local annex size: 0 bytes
❯ cat .datalad/config
[datalad "dataset"]
        id = b7adee52-a65a-43fc-a85b-c0d5e2d5b67c
[datalad "containers.busy"]
        image = .datalad/environments/busy/image
        cmdexec = {python} -m datalad_container.adapters.docker run {img} {cmd}
❯ datalad containers-run -n busy uname
[INFO   ] Saved busybox:latest to /tmp/busydemo/.datalad/environments/busy/image/tmpzrdd7mj2
[INFO   ] Making sure inputs are available (this may take some time)
[INFO   ] == Command start (output follows) =====
Linux
[INFO   ] == Command exit (modification check follows) =====
run(ok): /tmp/busydemo (dataset) [/home/mih/env/datalad-dev/bin/python -m ...]

Closes: datalad#199

Demo

```
❯ datalad create busydemo
❯ cd busydemo
❯ datalad containers-add -u dhub://busybox:latest busy
❯ datalad drop .datalad/environments --reckless availability
❯ git annex info | grep 'local annex'
local annex keys: 0
local annex size: 0 bytes
❯ cat .datalad/config
[datalad "dataset"]
        id = b7adee52-a65a-43fc-a85b-c0d5e2d5b67c
[datalad "containers.busy"]
        image = .datalad/environments/busy/image
        cmdexec = {python} -m datalad_container.adapters.docker run {img} {cmd}
❯ datalad containers-run -n busy uname
[INFO   ] Saved busybox:latest to /tmp/busydemo/.datalad/environments/busy/image/tmpzrdd7mj2
[INFO   ] Making sure inputs are available (this may take some time)
[INFO   ] == Command start (output follows) =====
Linux
[INFO   ] == Command exit (modification check follows) =====
run(ok): /tmp/busydemo (dataset) [/home/mih/env/datalad-dev/bin/python -m ...]
```
@qlty-cloud-legacy

Copy link
Copy Markdown

Code Climate has analyzed commit 24a11ce and detected 11 issues on this pull request.

Here's the issue category breakdown:

Category Count
Security 11

View more on Code Climate.

@codecov

codecov Bot commented Oct 10, 2023

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 41.93548% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.23%. Comparing base (c505e52) to head (24a11ce).
⚠️ Report is 81 commits behind head on master.

Files with missing lines Patch % Lines
datalad_container/adapters/docker.py 27.27% 16 Missing ⚠️
datalad_container/containers_run.py 77.77% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #246      +/-   ##
==========================================
- Coverage   94.69%   93.23%   -1.46%     
==========================================
  Files          25       25              
  Lines        1093     1124      +31     
==========================================
+ Hits         1035     1048      +13     
- Misses         58       76      +18     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bpinsard

Copy link
Copy Markdown
Contributor

Works for me, it seems to retrieve the layers correctly.

I currently no longer have a specific usecase for the docker adapter, but that is a great performance enhancement.

@yarikoptic

Copy link
Copy Markdown
Member

sorry, not grasping atm the whole machinery... if code works for you -- go ahead with the merge.

@yarikoptic yarikoptic added the patch Increment the patch version when merged label Oct 12, 2023
@mih

mih commented Oct 12, 2023

Copy link
Copy Markdown
Member Author

Thanks for the feedback! I am not sure whether I will have the chance to try the practicality of this implementation sufficiently. The next week will tell me that. For now, I will convert this to a draft PR.

If I get this chance to try this out sufficiently I will finish it out and repropose.

If this message is read after two weeks from now, and the PR is still in draft. Please close without waiting for me. Thanks!

@mih mih marked this pull request as draft October 12, 2023 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch Increment the patch version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docker: do not get layers if present in local docker service

3 participants