Roark product changelog roundup

WEEKLY UPDATE September 21 – October 18, 2026 This window was a heavier one than usual, so this is the long version. Live Bench v1.3 landed with four full industry scenarios, config as code grew to cover collector settings and scheduled simulation groups, dashboards finally compare across filter groups, and custom metrics picked up a handful of upgrades I know a lot of you have been waiting on. Nº 01 Live Bench v1.3: four industry scenarios, tool use, voice The public voice-agent leaderboard now runs every stack through four full industries, scores tool use alongside speech, and ranks on a weighted quality score with a price switch. What changed: Four scenarios, five callers each. Reschedule a healthcare appointment, triage an urgent home repair, file an auto claim, and troubleshoot an internet outage. The same five callers (US quiet, Spanish-accented on a street, Indian English in a café, flustered Southern with a crying baby, fast and interrupting under construction) run every scenario, so industries and voices stay comparable. Tool use is a first-class figure. Every stack is graded on whether its agent actually called the right tools, including hallucinated calls and skipped ones. Boards for LLM and speech-to-speech stacks now show tool-call completion and tool hallucination. Voice consistency and same-speaker checks. We flag lines where an agent drifts off its usual voice or the wrong speaker lands on the agent track, and the board scores on both. Semantic word error rate for transcribers. The raw rate is still shown, but ranking uses a judged rate that forgives differences a listener would read the same way. Weighted Roark score with a price switch. Pick quality-only or price-adjusted, filter by industry or caller, and open any model or stack on its own page beside the board. 👉 Browse the live benchmarks Nº 02 Simulation groups, scheduled from config, delivered to email and Slack Run several simulation plans together and read them as one report. Keep the whole thing in config as code with scheduled delivery when the group finishes. What's new: Run groups. Group existing runs, or start several plans at once from the dashboard. One combined report with an executive summary, numbered issues, strengths, metric aggregates, and a PDF export. Groups in config as code. kind: simulationGroup collects plans by name, schedules them on the same cadence block as plans, and announces each run to any mix of email recipients and up to 10 Slack channels (PDF in the thread). Schedule any plan from config. A schedule block on kind: simulationPlan runs it on cadence. The dashboard view goes read-only when the schedule is config-owned. Full REST surface. POST /v1/simulation/run-group to start a group, GET to read its full report document, regenerate on demand. 👉 See the simulation groups guide Nº 03 Per-collector metric settings in config as code, and re-score with them A collector in kind: collector can now carry each metric's per-collector settings in the file, and you can re-score past calls using exactly those settings over the API. What you can do: Pin settings in YAML. For each metric a collector runs, declare verifiedProperties, excludedPhrases, voicemailScript or screeningBrief inline. The file is the source of truth: an apply resets anything drifted in the app, and misspellings fail validation. Re-score past calls with a policy. POST /v1/metric/collection-jobs takes an optional policyId. The job runs each metric at the version and settings that collector runs it at live, so a re-collection against an updated Property Verification setup checks only the properties you now care about. Choose which properties a collector verifies. Property Verification's "Properties to verify" is now a first-class collector setting, visible everywhere the collector is edited. 👉 Read the collector config reference Nº 04 Dashboards compare across filter groups Dashboards now take a filter that applies to every widget, plu