We renamed the product. Somewhere in that change, the process manager's working directory became /home/ubuntu/glacro, while the deploy script kept writing to /home/ubuntu/glacro. The site went down for about an hour and our checks reported everything was fine.
Why the homepage lied
The old process was still running, still holding port 3000, still serving the previously built HTML from memory. So a request to / returned 200. But every hashed asset it referenced had been deleted from disk by the new build, so every request to /_next/static/* returned 400. A 200 on the homepage was true and completely uninformative.
What we check now
The deploy now reads the freshly built HTML, extracts a real asset URL from it, and fetches that. If the running process cannot serve an asset from the build that was just produced, the deploy fails and the run goes red. It also reads the application names out of the process manager config rather than hardcoding them — the drift detection had its own copy of the old names, which is why it silently matched nothing and reported no drift.
# Fetch a chunk the new build actually references
CHUNK=$(grep -o '/_next/static/chunks/[^"]*\.js' index.html | head -1)
curl -fsS "http://localhost:3000$CHUNK" > /dev/null || exit 1The wider lesson was about coupling: an infrastructure path had been derived from the product name, so renaming the product moved a directory. Paths are now fixed and documented as unrelated to branding.