Tech talk: The AI stack is more modular and multi-modal than ever |
With a changing landscape of models, standards, and systems, no two AI tech stacks are alike. This open ecosystem is the emerging standard of architecture, something last week’s Open Intelligence Summit (OIS) highlighted with speakers from NVIDIA, Amplitude, Open Hands, and Nous Research.
According to internal DigitalOcean figures, 88% of our tokens run on open-weight models, a 27% increase from May 2026. This is accompanied by a 50% growth in token volume over the last six months.
It’s clear that loyalty to a single model is fading, and a new layer is emerging to manage churn: routers. Analysis from Vals AI frames routers as a new AI primitive. They choose models based on quality and cost per request, so applications don’t need to revisit model selection every time something new launches. They also provide more opportunities for developers to select models based on use case, rather than being locked into frontier models or a specific provider.
Open standards are doing the work that closed-loop systems and frontier models used to do, and the tooling is adapting:
|
- DeepSeek opened Search and Fetch APIs that return live web content as Markdown for agents and RAG pipelines.
- Perplexity open-sourced its agent sandbox.
- Anthropic’s OpenTelemetry support means traces from different providers can share the same dashboards.
|
When models are interchangeable, differentiation moves to the layers you control: evaluation, observability, security, and cost. With the ability to fine-tune multiple layers, developers need concrete plans for AI stack management. Abstract your model calls behind a routing layer. Standardize on OpenTelemetry for agent traces. Build evals you can rerun against any model. And keep a cost dashboard open.
Our new Managed Agents and Agent Learning Services, announced at OIS, paired with our cache-aware Inference Router, allow you to build AI products and own your intelligence.
|
|
|
|
AI adoption looks broad until you check what’s measurably working.
a16z’s State of Markets II presentation found that 69% of S&P 500 companies have a live AI deployment and about 30% report a quantified result, but only around 2% track a metric for it, while the top 1% of companies by AI spend outspend the top 10% roughly eightfold. The gap is less about access to tools than about tying AI to a workflow you can measure.
ChatGPT is shifting from walls of text to answers you can interact with.
OpenAI’s GPT-6 launch introduced “Intelligent UI,” which lets the model respond with charts, buttons, forms, and quick tools like savings calculators or bill splitters, all inside the conversation. OpenAI also says GPT-6 Instant starts answering web-search questions 44% sooner on average than GPT-5.6 Instant.
|
|
|
|
Agents in Production, Part 1: Build an Overnight AI Support Agent using Managed Agents
An unattended support agent ran for 1 hour 48 minutes against a live board, triaged 99 tickets, stopped all 35 sensitive ones for a person, and sent zero replies, because the agent was never given a route to send. Learn how to scope board credentials, restrict egress to a single host, and add deterministic checks to drafts so the agent can’t act outside its brief.
How to Safely Let AI Agents Call Production APIs using DigitalOcean Action Gateway
Across 1,300 episodes, 45 to 48 of every 100 runs without a gateway ended with at least one harmful change, and the agent re-sent a refund in 66 of 75 tries. See how idempotency keys, version checks, and approval tiers brought harmful changes to 0 of 100.
What real time-to-first-token looks like on DigitalOcean Serverless Inference
Across 380 calls to six models, five landed between 0.88 and 0.98 seconds median TTFT on short prompts, but DeepSeek V3.2 hit 78.3 and 186.9 seconds on two of 80 calls. Get the measurement method and code repository to run the same audit on your own models and prompt sizes.
Should You Use Spot or On-Demand GPUs for Model Training?
At a 60-minute cadence, a Spot GPU fits single-node jobs and multi-node jobs with elastic restart. Resuming a 98.3 GB checkpoint after a simulated reclaim took 706.5 seconds on MI355X and 436.6 seconds on B300. Walk through the break-even model and decision table, and plug in your own checkpoint cost and reclaim rate.
Deployment Isn’t the Bottleneck: Inference Economics Is
The same $3-per-hour GPU can cost about $0.30 or about $3 per million tokens, depending on whether it runs at full or 10 percent utilization. Learn how cold starts, continuous batching, and KV-cache limits set cost per request, and run the arithmetic for your own model.
|
|
|
|
|
|
|
|
|
Now in Public Preview: DigitalOcean Insights
Insights gives you one place to see what your agents, applications, and infrastructure are actually doing, with metrics, logs, and traces built in and nothing to install or instrument. Follow an agent run from the model call through every tool it used and find exactly what broke, without switching tools or lining up timestamps yourself. It’s free to use during public preview.
Now Available: Inference Router Enhancements
Choosing the right model for every request shouldn’t require constant testing and manual routing. DigitalOcean’s enhanced Inference Router automatically selects models based on task performance, cost, and latency, helping you balance quality and efficiency as new models become available.
Now in Public Preview: Simulations
Simulations, a new capability of DigitalOcean Evaluations, lets you test a change through realistic multi-turn scenarios in an isolated sandbox on Harness Runtime, with the same configuration and Action Gateway tools your agent uses in production. Pull scenarios from Signals, generate them from your agent’s configuration, or describe one in plain language. An LLM judge scores every trajectory, and running a baseline against a candidate shows exactly what improved and what regressed.
Now in Public Preview: Signals
Signals watches production traffic on Managed Agents and Inference Engine and automatically highlights potential problems across user sentiment, tool execution, and infrastructure, with nothing extra to instrument. Flagged sessions become evaluation scenarios in one click. Signals is opt-in, designed to redact PII before processing, and keeps traces within your DigitalOcean Team boundary.
|
|
|
