feat: add monitoring to cronjobs

[MarceloRobert]

Changes

• Added healthcheck_id parameter to relevant commands used in the cronjobs
• Added map from healthcheck ids to healthcheck.io uuid, the ids reflect the defined cronjobs and are set via environment_variable
• Added wrapper to run functions with healthcheck.io pings for start, success and failure
• Applied healthcheck changes to main cronjob execution

How to test

For this PR, the ideal way to test would be with a healthcheck account and uuid so you can monitor the pings. Follow the instructions there to create an account and a check.

• In the code, set one of the cronjobs to notifications --action=fake-report and set a small interval, such as */2 * * * * (run every 2 minutes)
• You won't need any other cronjob for testing, but if you want to make more than one check you're welcome
• Check that the cronjobs are not going to send email reports accidentally
• Get your healthcheck id and use it in the cronjobs
• Start the cronjobs either via docker compose up backend --build or locally with poetry run ./manage.py crontab add (docker is the recommended way)
• In the healthcheck.io dashboard, see the details of your healthcheck and monitor the received pings

The cronjobs should even if the ping fails or the monitoring_id is empty

Required production changes

• Create 6 different checks for the 6 monitored cronjobs
• Set up on Discord a desired webhook (could reuse the already-present ones)
• Set up in the checks the integration to webhook, POSTing to the discord webhook when a check goes down. Example of body:

{
  "avatar_url": "https://avatars.githubusercontent.com/u/11725450?s=200&v=4",
  "username": "KernelCI Healthcheck Notifications",
  "content": "Healthcheck.io Alert: $NAME service went down!"
}

Also set the request headers to Content-Type: application/json

• Disable the default email notification integration in the checks

Closes #1792

[bhcopeland]

Awesome to see healthchecks.io being used. Have you setup an account? We need to make sure one global KCI account is used with me/denys (@nuclearcat) (and you guys on it). Lets talk about this in tomorrows meeting

[MarceloRobert]

Awesome to see healthchecks.io being used. Have you setup an account? We need to make sure one global KCI account is used with me/denys (@ nuclearcat) (and you guys on it). Lets talk about this in tomorrows meeting

Yeah, I have setup a personal account so I can test this later. Since this change won't stop the cronjobs that don't have monitoring we can go checking the accounts without blocking the PR.

[alanpeixinho]

nit: we can use urllib.parse.urljoin

[MarceloRobert]

urljoin actually was a bit worse because if you join url/middle with end, it will result in url/end instead of url/middle/end. Formatting the strings seems to be safe for this operation

[MarceloRobert]

@bhcopeland all set up and tested, this is now ready for review. If you want to move forward with the healthcheck.io integration, I left some instructions on what to do.

cc @nuclearcat