Monitoring: adding long time storage for metrics

[Tbaile]

Firewall needs a long term storage for metrics, at the moment Netdata does the job but it has the following limitations:

• only 6 hours
• very old build
• hard to customize

We landed on Victoria Metrics due to the nice space usage and easy replacement for Prometheus, which does the same as Netdata.
However a tool that scrapes the data from the firewall and send it to Victoria is needed, that's where Telegraf comes in.
From Telegraf the data to be sent to the Victoria Metrics is the following:

• generic machine info (CPU, MEM, etc...)
• MWAN status
• services status
• pings

Victoria can through vmalert mimic what AlertManager does, which is in firewall's case:

• notify downtimes
• notify alerts

Alerts is kept in storage by Victoria Metrics itself.

[gsanchietti]

Testing image: 8.8.0-nethsecurity-8.8.20260515144748.b7046f2a9 or later

Main changes:

• switch monitoring/alerting from Netdata to Victoria Metrics via Telegraf
• the new Metrics page replaces the Netdata charts
• when data storage is configured, Victoria stores metrics under /mnt/data/victoriametrics, otherwise they are kept in RAM

Test case 1

• Install a clean machine with the testing image
• Open Metrics page
• Verify the page loads and charts show data
• Fill the boot partition with dd until DiskSpaceCritical fires
• Wait for the alert evaluation window to pass
• Verify the alert is visible in the Alerts section of the UI

Test case 2

• After test case 1
• Free the space on boot partition
• Enable an enterprise subscription inside the machine
• Execute again test case 1
• The alert should be visible on my.nethesis.it