Improve CI/CD for the repo.

[tok-kkk]

Object

We want to have a test suite build into our CI system which helps us reduce the amount of manual testing. The test suite should cover most of the CLI command and tests on different VPSs and systems.

Design

Testing Accounts

We should create separate accounts or sub-users of our main account and limit the service access. This ensures the safety of our VPS main account. We also need to make sure the account credential is private. (i.e. set up as Github secrets or environment variables)

Testing cases

CI should be triggered on every new commit. Both CircleCI and GitHub Actions allow you to skip certain commits if you have [skip ci] in the title or description. source.It should cover both positive and negative test cases. And tests against different VPS should be running parallel if that's possible. It should test the following cases:

• Test the CLI binary is working and returns the expected version.
• up a couple of nodes with different options and destroy them with different options.
• Deploy a node, testing following cases before destroy
  • check start, stop and restart commands are able to control the darknode status.
  • check list command show the correct info of the darknode
  • check update command is able to compare the versions and do an update when outdated.
  • try resize the darknode to an upper tier and a lower tier and check the darknode status
  • check exec command with different options

Discussion

• Github Actions VS CircleCI
  I personally favour CircleCI as we already have everything set up in CircleCI and it has some features which Github Actions doesn't have. Github Action is also good in terms of github integration, the setup should be more easy and friendly than CircleCi. We can achieve the same result with Github Actions, but with more developing time.

• Not sure how can we test the ssh command in CI.

[loongy]

Are these tests going to be run by GitHub Actions, or are they expected to be run by the user on-demand? We should specify that the VPS accounts we use for this must be separated from the main VPS accounts we use so that compromising the CI will not compromise the main accounts.

[loongy]

This issue should probably describe the exact commands we will be testing, in what order. For example: we will do darknode up, then darknode version, then darknode down (obviously we want a few different workflows, many of which will be a bit more complex than this).

[loongy]

@tok-kkk let's proceed with GitHub Actions, just because it allows us to keep as much as possible on GitHub, and we can cut down on an external service. GitHub Actions also seems to be a little more reliable recently. RE: the testing ssh command, we can omit that from testing in these tests.

[loongy]

CI should be triggered on every new commit.

This might be a bit extreme, since these tests are probably going to take a decent amount of time to actually execute in full. I would recommend that we only run them on attempts to merge into master (eg. PRs into master).