Cloudflare alerts, and a container that never went to sleep
Value
3.0
■■■□ of 4
Time
43m
active
Elapsed
3h 20m
wall-clock span
Commits
4
from this session
Cost
$7.12
model spend
Tokens
47k
uncached in, plus out
Tool calls
117
by the agent
Outcomes
12
7 left open
Summary
Written by the summariserConfirm a Cloudflare billing budget alert exists, review last month's usage, and add usage alerts for Workers requests, Workers CPU and Durable Objects. Later, find out why Containers were charged and stop the SMTP probe container running 24/7.
Rob asked the agent to confirm his Cloudflare budget alerts, review last month's usage, and add usage alerts for Workers requests, Workers CPU and Durable Objects. Budget alerts were already in place, and three usage alerts were created. Cloudflare's API rejected every CPU product name the agent tried, so no CPU usage alert exists, and a test alert was sent through the Workers requests policy. Rob then asked why Containers were charged. The agent found that the prod SMTP probe container had been running continuously since 4 October despite a five-minute idle setting, and inferred from the code that a two-minute cleanup timer kept waking it. Rob chose to leave the instance running rather than delete the whole prod application. The change was merged and released to production as 0.35.1 after two CI failures were fixed, and the container showed inactive after the deploy.
What came out of it
- Usage alerts created and enabled for Workers requests (5M), Durable Objects requests (1M) and Durable Objects duration (1M GB-s)
- Test alert sent through the Workers requests policy; alert history shows it went out by email at 17:22 UTC
- Existing budget alerts confirmed at $1, $2, $10 and $20, all enabled
- Last month's usage reviewed: Containers were the only charge ($1.39); other metrics well inside included amounts
- MSS-0453 created to move SMTP probe security-state pruning to the daily cron; change pushed to main as [commit] (staging only)
- Dependency security patch pushed as [commit]: sharp 0.35.4 to 0.35.5 and source-map-js 1.2.1 to 1.2.2, to clear CI's package audit
- Identifier policy baseline entries for the new lockfile added in [commit] after CI's policy check failed
- Release PR #91 (version 0.35.1) squash-merged as [commit] after its checks passed; production release finished 19:55 UTC
- After the deploy, the prod SMTP probe container reported inactive with no running instances
- MSS-0453 notes appended; its second acceptance criterion (a test of the clean-up path) dropped by Rob's decision
- MSS-0454 logged for the local just check policy-check failure
- Local backlog commits [commit] and [commit] created; not pushed
I started by wanting to check my Cloudflare billing alerts and add some usage alerts for Workers and Durable Objects. The budget alerts were already there, at $1, $2, $10 and $20, and all enabled. Last month everything else sat well inside its included amounts, and the only charge was $1.39 for Containers. I added usage alerts for Workers requests and both Durable Objects metrics, set well above normal, and sent a test through the Workers requests alert. The alert history shows it went out by email. Cloudflare wouldn't accept any product name for Workers CPU, so there's no CPU usage alert. My dollar budgets are the only cover for CPU spend.
The Containers charge needed more digging. I asked why I'd been billed when I thought nothing was running, and it turned out something was. It was the SMTP probe in my DMARC reporting project. The probe exists so TrustHeader can check whether a domain's mail servers present the certificates its DNS says they should, and a Worker can't do that check alone. The probe had been running continuously since 4 October, even though its code asks for a five-minute idle timeout. From reading the code, not from logs, I think a two-minute cleanup timer wakes it each time, and every wake-up resets the idle countdown, so it never sleeps. That's a best guess until I can confirm it.
Cloudflare can't stop a single container instance. The only way to stop this one straight away would have been deleting the whole production application, which would have broken the probe until the next deploy, so I didn't do that. Instead I logged a task to move the cleanup onto the daily cron. The change went through CI after two fixes: a security patch for two packages, and a policy check that needed the new lockfile recorded. It shipped to production as 0.35.1, and the probe showed as inactive straight after. A deploy restarts the container anyway, so the real test is tomorrow's daily run at 12:00 UTC. The container should go idle, and daily Container billing should drop to near zero.
Two smaller things are still open. The local check fails on the policy step even though CI passes, so I'll log that as a separate task, and two local commits haven't been pushed yet.
Left open
- Workers CPU has no usage alert because Cloudflare rejected every CPU product name tried; the dollar budget alerts are the only cover
- Receipt of the test alert email in the inbox was not confirmed in the transcript; only the alert history shows it was sent
- MSS-0453 stays In Progress until the 12:00 UTC daily run on 9 October shows the container going idle and daily Container billing dropping to near zero; this needs a local cf check
- The container was cause-inferred from code, not confirmed from logs
- The prod application was not deleted, so the container keeps costing about $0.05 a day until the next check
- just check still fails at policy-check locally (tracked as MSS-0454); CI passes
- Local commits [commit] and [commit] are not pushed