Troubleshooting the agent

Unhealthy or never reported

Symptom	Likely cause	Fix
Never reported on a new instance	The image does not run full cloud-init.	Use a cloud image with full cloud-init, or install it by hand.
Never reported, metadata endpoint disabled	The agent talks only to the metadata service, which is off for the instance. Installing it by hand does not help.	Turn the endpoint on (the instance’s modify-metadata-options action with metadataOptions.httpEndpoint: enabled), or launch with it on.
Unhealthy after it was healthy	The agent stopped: the guest is down, out of memory or disk, or the agent’s units failed.	Check the instance is running; look inside with the serial console; read the logs.
Every agent unit failed with 203/EXEC	The binary has the wrong SELinux label.	ls -Z /usr/local/bin/stratum-instance-agent should show bin_t; sudo restorecon -v /usr/local/bin/stratum-instance-agent. The self-heal timer fixes this within about 10 minutes.
On other build for long	The agent cannot update itself.	Refresh the agent.
Sessions and commands fail, heartbeat healthy	The session channel is down.	systemctl status stratum-instance-agent-session in the guest; the session timer restarts it within a minute.

Refresh the agent

Refresh agent re-installs the build the platform serves and its full unit set, over the agent’s live session channel. No relaunch, no floating IP.

stratumctl instances refresh-agent --instances i-abc123,i-def456 --project p-web

Console: the instance’s Agent tab, Refresh agent. Up to 100 instances at a time; each result says how it ran:

Method	Means
agent-root	The agent ran the install as root and checked the new binary.
sudo-fallback	An older agent: the install ran as stratum-ssh through passwordless sudo. Fails with “that user has no passwordless sudo” if the image does not grant it; then use the steps below or relaunch.

An instance whose agent never started has no channel to refresh through.

From inside the guest

Re-run the platform’s install script, the same one a self-update uses:

curl -fsS http://169.254.169.254/latest/stratum/agent-bootstrap | sudo sh
# without curl:
wget -qO- http://169.254.169.254/latest/stratum/agent-bootstrap | sudo sh

Or reinstall the package:

sudo dnf reinstall stratum-instance-agent              # dnf family
sudo apt-get install --reinstall stratum-instance-agent   # apt family

Install by hand

For a guest that never got an agent (an image without full cloud-init): download the binary from Admin › Platform › Catalog › Instance Agent, which also shows its SHA-256, then:

scp stratum-instance-agent admin@<guest>:/tmp/
sha256sum /tmp/stratum-instance-agent                   # compare with the console
sudo install -m0755 /tmp/stratum-instance-agent /usr/local/bin/stratum-instance-agent
sudo /usr/local/bin/stratum-instance-agent install      # writes, enables and starts the units
stratum-instance-agent version
systemctl list-timers 'stratum-instance-agent*'

Check the checksum: a truncated copy fails in ways that look like an SELinux problem. The agent still needs the metadata endpoint on: it talks to nothing else.

Logs inside the guest

systemctl list-timers 'stratum-instance-agent*'
systemctl status 'stratum-instance-agent*'
journalctl -u stratum-instance-agent-heartbeat -n 50
journalctl -u stratum-instance-agent-session -n 50
journalctl -u stratum-instance-agent-inventory -n 50
journalctl -u stratum-instance-agent-selfheal -n 50

Log	Look for
heartbeat	Metadata service errors; the self-update check and its result.
session	The channel connecting and each session opened.
inventory	Package collection errors behind stale security posture.
selfheal	Repairs it made.