feature: allow dropkick to bundle files next to the service executable

[inickles]

while thinking of ways to supply an AWS key policy to use as a runtime argument to permission-slip https://github.com/oxidecomputer/permission-slip/pull/37, which doesn't make for a great command line argument, i was wondering if there was a way to include a file with the service that i could reference from cmd arg instead.

Ex: --key-policy-path policy.json vs --policy '{"image this was huge and you":"get the idea...'

for now i think i'll just an env var, but i could see this kind of feature being useful for services that would like to bundle config files with them.

[iliana]

I'm in favor of doing this, just need to work out the design. There's sort of a spectrum of what we could do here.

For context, the dropkick "API" consists of:

• command line arguments
• options in a Cargo.toml [package.metadata.dropkick] table

So for any new input what I end up considering is:

• Is this an option that is something a user would do as a one-off for testing, or would this setting be something a user would expect to "live" with the codebase?
  • If it's supposed to live with the codebase, when would a user use this instead of include_str!()?
• How is the input shaped? How is that parsed from either TOML or command line arguments?

And specifically for this input:

• Where does the file go?
  • Whatever we do, it'll end up in /nix/store somewhere, but we can use either symlinks or bind mounts¹ to provide it a known, non-hash-based path.
  • Does the user need to specify the location on disk, and if so, is that a great way for the user to create interesting problems for themselves? ("here, put this in /usr/bin/env, it'll be great, i swear")
  • Do we allow dropping a file anywhere in /dropkick? Does the user get to specify a filename or do we just assume it from the input?
• If we support this option in [package.metadata.dropkick], is the file relative to your current working directory, or to the Cargo.toml? (Probably the latter, while the CLI flag would be the former.)

---

What I'd like to propose is this interface:

CLI: --copy-file PATH copies PATH (relative to the user's CWD) into the Nix store, and creates a bind mount to /dropkick/$BASENAME, where $BASENAME is the base filename of the path. There is no option to change the path on the instance with this mode to keep the CLI easier to comprehend.

Cargo metadata: This metadata would copy data/whatever.json and data/some-other-thing.json, relative from the package Cargo.toml, to the Nix store and bind mount them to /dropkick/whatever.json and /dropkick/other.json, respectively:

[package.metadata.dropkick]
port = 42069

[[package.metadata.dropkick.copy-files]]
from = "data/whatever.json"

[[package.metadata.dropkick.copy-files]]
from = "data/some-other-thing.json"
to = "other.json"

Thoughts?

Footnotes

1. Believe it or not, bind mounts are easier in NixOS configs. ↩

[inickles]

Lots of good questions and considerations.

What I'd like to propose is this interface:

CLI: --copy-file PATH copies PATH (relative to the user's CWD) into the Nix store, and creates a bind mount to /dropkick/$BASENAME, where $BASENAME is the base filename of the path. There is no option to change the path on the instance with this mode to keep the CLI easier to comprehend.

Cargo metadata: This metadata would copy data/whatever.json and data/some-other-thing.json, relative from the package Cargo.toml, to the Nix store and bind mount them to /dropkick/whatever.json and /dropkick/other.json, respectively:

Those both sound great and I think even just the CLI option without being able to change dst filename would work well for what I'm looking for, which is scripts I intend to use via shell access, from time to time for maintenance. The path from CLI being relative to current PWD makes perfect sense. I assume absolute paths would just work.

After having filed this, and considering I plan on using them from a shell, I figure I can just SCP the files I need for now, which I think will be an easy workaround for my current needs.

[inickles]

An idea for this: The Oxide user_data param for creating an Oxide instance seems like an easy way for users to include their specific data & configurations for an image, via cloud-init. I'm using this in my Github ephemeral runner project to configure a new instance on first boot with the runner config. This could easily be exposed via dropkick cli param. It is a way for users to do whatever, such as install new packages, add config files, etc.

At first I thought this would tie in nicely to the CreateInstance API, but I think we'd want to do this in Nix instead, because we want it on the image whether we deploy the Oxide instance or not. Also, by hooking into that CreateInstance API then it assumes an Oxide mage and won't work for AWS builds, so Nix seems like the natural place for it.

[inickles]

As far as I can tell, this cloud-init might an alternative to the nixie-tubes.nixosModules.ssh-init module, but I'm not sure, as that sounds like that'd require multiple cloud-init user data files, which is stretching my knowledge of it.