PostHog AI product-building newsletter

Forwarded this email? Subscribe here for more 38 things we've learned about building AI products Lessons from 4 years of building AI products at PostHog Ian Vanagas Oct 12 READ IN APP We’ve built 10 different AI products at PostHog, including PostHog AI, our install Wizard, Replay Vision, Scouts, and more. They’re used by nearly 100,000 people every month. This is what we’ve learned. AI can make your product worse. It can be slow and unreliable, or solve a problem nobody has (like chatting with your docs, any AI can do this). Just because it’s AI does not mean it’s worth building. Consider whether you need to build AI at all. It’s often better to build for AI first, like an MCP server. These are easier to build and maintain. 5x more people use our MCP server daily than PostHog AI. Look for jobs to be done, not demos. The best opportunities for AI are tasks that take 30+ seconds, use language or interfaces users don’t know (like SQL), or tasks users repeat 20+ times. Onboarding is a great first target. Our AI install wizard does two hours of work in ~8 minutes. Those who use it convert to paid 5x more often and activate 2x faster. Don’t ask “how do we add AI to this?” ask “what is the agent-first version of this?” Doing this forces you to rethink the jobs to be done for an agent-first future and helps you avoid being disrupted by an AI-native competitor. Building 2030-shaped products may require you to start fresh and break from years of cumulative UI and assumptions that are no longer valid or relevant. Watch for customers rebuilding your product with AI. This reveals what features they actually want. When one of our largest customers built their own prototype of “PostHog 2.0” with our MCP and Claude Code, we knew we needed to add more customizable “canvas” artifacts to PostHog. Validate AI bets like you would new products. Write down how it will be judged and how much runway it gets. Ask how disappointed your team would be to lose it (the PMF survey). We did this for one product and only 23% said they would be very disappointed. Bad results, but useful. Write down how your product loses to an AI-native threat and then fund your version of the attacker. Even the best support product loses to one where AI answers all the tickets for example, which is probably why Intercom pivoted so hard into Fin. Answer the question: How does this beat Claude? It’s what people default to. Your moat is context: your product’s functionality, your users’ data, and the tools and skills you’ve built. Your harness is not your moat. PostHog AI went through three of them in a year. We now have MCP as the canonical interface and let people pick their own harness. Don’t spend innovation points here. Don’t build an assistant. Assistants are what you build when intelligence sits next to your product. Systems are what you build when intelligence becomes part of it. Build those instead. Focus on building shared AI infrastructure. Make prompts, streaming, consent, evals, tasks, and analytics primitives teams can extend and build into their products. Pick a consistent UX pattern to avoid death by a thousand widgets and then get out of people’s way. Product teams know their users best, not the AI team. Let agents run code. Code execution gives agents capabilities beyond what your predefined tools might enable them to do. PostHog AI can write SQL, run Python scripts, and ship PRs. Sandbox them to make this safe. Model upgrades will bulldoze your assumptions. We’ve restructured PostHog AI due to cheap reasoning (o4-mini) and reliable tool use (Claude 4). Multimodal models made Replay Vision possible. The harness, prompts, and context you built for previous models might not be relevant for current ones. Skills are features for agents. They combine tools, docs, and rules into a recipe for a job. If an agent has the tools but not the recipe, it must look up how to use your product every time. Every docs-search call is evidence of a missing ski