Jobs every engineer should automate with agentsHow we handed off our dev chores to scheduled agents (with prompts)Some engineering tasks are too repetitive to be worth prompting every time by hand, but too fuzzy for a script. We've been using Scouts at PostHog to do these instead – scheduled agents with long-term memory that run on a cron trigger.¹ Here are 6 chores we’ve automated this way, with prompts you can copy to build your own version of each. 1. Validating small fixes in prodEveryone's merging more code than ever. And the smaller the PR, the more annoying it is to stop what you’re doing to check it in prod – especially if you already did that in dev. Take this agent-generated string length limit fix that Andy Maguire from our self-driving team merged. It's a tiny change that takes more effort to validate than to push: he'd have to check back for when it hits prod, open error logs, and verify traffic is still flowing. Instead, Andy queued it to merge at 09:19 UTC and moved on to other tasks. It reached prod about an hour later, and the next morning a scheduled agent confirmed the fix held across 200K sandbox-creation log lines using a prompt similar to our PR follow-up skill. It's even more useful in the opposite case. One time, an agent fixed an error that looked like it came from our browser SDK, affecting about 45k users a week. A later validation report showed the error was firing more than before from our Node SDK, which has its own copy of the same utility – exactly the kind of thing a human might not have remembered to check. Steal this prompt:
2. Capturing feature requestsWe use agents to find recurring themes in user feedback from in-app surveys, support tickets, Discord, Gong, and Slack. A lesson we recently re-learned was the importance of prioritizing that feedback. When our sales team set up a scheduled agent to post to #feature-request-feed, Raquel from our Blitzscale team said that these automations had been less than useful in her experience: Tyler Crandall, the TAM who created the feed, updated the prompts to factor in urgency. Each one now gets rated based on the call transcript and customer history, which makes up for the facial expressions and voice cues we'd rely on if we did this manually. Plus, the agent can reference its own ratings as context for future runs, too: Steal this prompt:
3. Adding event trackingThere are times where even we forget to add instrumentation for product analytics. We added a rule in our It works through recently merged PRs and looks for user-facing actions that are missing event tracking. When it finds one, it cross-checks our live PostHog data to confirm the event never fires, and then files a report with a fix. A coding agent then picks up that report and opens a PR to add the instrumentation. It's useful to have a separate agent doing the actual implementation work. The scheduled agent's original report recommended adding 2 different capture calls in the frontend for this PR. The coding agent, which has more specialized context, found that multiple sources (including MCP and API callers) sent the same request. It chose a better pattern to instrument these as a single capture on server-side, with session IDs and access method. Steal this prompt:
4. Diagnosing flaky testsFlaky tests are a huge bottleneck for engineers working with AI. They cause intermittent CI failures (usually from hard-to-control factors like timing, shared state, or dependencies). These two were the biggest obstacles slowing down our engineers according to our Q2 internal DevEx survey: To help with this, we added a scheduled agent that checks for newly quarantined Trunk flaky tests every 3 hours and creates (or updates) reports that are actionable enough to be turned into PRs. Since July, it's filed 72 diagnostic reports, and 11 of those have led to merged fixes like this one, which a coding agent wrote using our skill for fixing flaky tests. Even though this loop isn't fully automatable, we still consider it a win because it lowers the cost and friction for a human to start a fix. The diagnostic reports are always kept up to date in the background so that anyone can quickly pick one up at any time. And that's way more likely to happen than if our engineers had to start each one from scratch. Steal this prompt:
5. Triaging our own agent feedbackAnother type of feedback we take seriously is the responses from our internal coding agents using our dev environment, which we call Our monorepo's AGENTS.md instructs agents to use hogli devex:feedback to send a report whenever a The problem with all of these is that if you prompt agents to send reports, they'll always try to send something. We already have a few thousand reports written by our agents since we shipped it in July, and that’s way too many for anyone on the team to read. Raul, an engineer on our DevEx team, used to have an automation to triage these by grouping reports into themes, checking if the root issue has already been fixed, roughly ranking by frequency and impact, then committing the notes to a file in our private DevEx repo. Instead, he now just checks on these once a month for a couple of hours, and the reports still lead to low-hanging fixes from across the team, like rewording a preflight warning or updating how code owners are labeled. It's a good example of how some tasks are still better suited for AI-assisted human review, even if they could be solved with scheduled automations. Steal this prompt:
6. Self-improving promptsWith enough data, some of our scheduled agents can even suggest improvements to their own prompts. The key is that they have to include real evidence from past runs, and proposals have to go through human review. We also have a versioned skill store that makes it easy to track these changes. For example, one scheduled agent proactively checks that our product docs are up to date. It once flagged that we should remove the "in beta" notices from two of our docs after a cleanup PR deleted both features' flags. What the agent didn't know was that even though both features were visible to all PostHog users, the team didn't consider them GA yet. When a human reviewer dismissed its reports with handwritten feedback, the agent filed a report suggesting a fix to its own skill:
Steal this:
Words by Jina Yoon, who is still waiting for agents that can fold her laundry. 🕰️ Scheduled reads
💼 Tasks we can’t automate1 Some people call these "background agents," "ambient agents," or "always-on agents." Others describe them as a type of loop or cron. |