Skip to content

Host monitoring

Project Loom’s k3s node (s) run on Ubuntu VPS boxes, monitored by Netdata Cloud. This is host-level monitoring, not a workload in this repo (no manifest for it, and Argo CD never reconciles it).

Netdata’s agent runs as a systemd service directly on each VPS, outside k3s. It reports OS-level metrics (CPU, memory, disk, network) and auto-detects the container runtime and any other services running on the box. It does not monitor application-level metrics inside the cluster (no Prometheus / ServiceMonitor stack exists yet, see the repo’s own CLAUDE.md for why that’s deferred).

Netdata Cloud stores no metrics itself, it only live-queries each box’s agent. Free tier caps out at 5 connected nodes; check current node count against that before adding another box.

At app.netdata.cloud, sign up (or log in) and create a Space if one doesn’t exist yet. Inside it, open (or create) a Room, all boxes being claimed below should join the same Room so they show up together in one dashboard.

From the Room, use Connect Nodes, it generates a personalized one-line install+claim command containing your claim token and Room ID. This command is per-account; don’t reuse a copied command verbatim between boxes, always copy it fresh from the UI for each one (the token is the same per Space, but confirm the Room ID matches before running).

For each of the 3 boxes:

Terminal window
ssh <user>@<box-host>
# paste the command from Connect Nodes, e.g.:
curl https://get.netdata.cloud/kickstart.sh > /tmp/netdata-kickstart.sh
sh /tmp/netdata-kickstart.sh --claim-token <TOKEN> --claim-url https://app.netdata.cloud --claim-rooms <ROOM_ID>

Verify the install locally on the box before moving on:

Terminal window
sudo systemctl status netdata # active (running)
curl -s http://localhost:19999/api/v1/info | head # agent responding

Back in the Room’s dashboard, all 3 nodes should show as Live within a minute of the install completing. If a node stays offline, check outbound connectivity from the box (the agent needs to reach Netdata’s Cloud endpoints) and re-check the claim token/Room ID used in step 2.

Metric storage lives entirely on each box’s local dbengine, not on Cloud. Default is a 3GiB cap split across tiers. Write the [db] section directly with a heredoc instead of opening an editor:

Terminal window
sudo tee /etc/netdata/netdata.conf > /dev/null <<'EOF'
[db]
db = dbengine
dbengine tier 0 retention size = 1GiB
dbengine tier 0 retention time = 14d
dbengine tier 1 retention size = 1GiB
dbengine tier 1 retention time = 3mo
dbengine tier 2 retention size = 1GiB
dbengine tier 2 retention time = 2y
EOF
sudo systemctl restart netdata

Each tier stops at whichever limit (size or time) is hit first; setting either to 0 disables it. Any option not listed here falls back to Netdata’s built-in default, so this file only needs to hold your overrides.

Confirm the new caps took effect from the Cloud dashboard’s per-node Metrics tab, or locally with curl -s http://localhost:19999/api/v1/info.