The FTC seeks incident records as agent failures sharpen questions of control and liability. ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­  
Armilla Review Banner

No.184: Eight states sue OpenAI over ChatGPT safety

Eight Filings, One AI

Eight state attorneys general have sued OpenAI under consumer-protection and public-nuisance laws, while the U.S. Federal Trade Commission is seeking safety-incident records from OpenAI and Anthropic. Separate reports show agents spending funds from another wallet, deleting a television season's completed edit and inventing underwriting answers to produce an insurance quote. Anthropic granted METR employee-like access, Sierra demonstrated a scoped agent-to-agent mortgage workflow, and Anthropic deployed a model that reviews its own conversations for possible misuse. A policy debate over strict liability asks whether developers should pay for harm regardless of negligence.

 

The commercial requirement: buyers, insurers and regulators need records that show what an agent was allowed to do, what it actually did, which control intervened and who accepted the remaining risk.

FEATURED

Insurance analysis separates affirmative AI cover from policy silence

Business Model Analyst examined how AI losses may enter cyber, D&O, CGL and tech E&O insurance. Silent AI coverage means a policy neither expressly includes nor excludes an AI-related loss, so the result depends on existing definitions, exclusions and the facts of the claim. The article notes that Verisk's Insurance Services Office made three optional generative-AI endorsements available with January 2026 edition dates. Firms should inspect their own renewal endorsements rather than assume the risk is covered or removed. The article identifies Armilla as a Lloyd's coverholder focused on affirmative AI liability and cites available limits of up to US$25 million per organization. 

 

For buyers, the practical issue is whether each AI failure mode maps to an intended policy response rather than a disputed absence of wording. Firms should review policy wording, endorsements, contractual-liability exclusions and employment practices liability insurance restrictions, then document where each material loss scenario is intended to land.

TOP STORIES

Eight states sue OpenAI over alleged ChatGPT harms

A bipartisan group of eight state attorneys general filed separate lawsuits against OpenAI, alleging that ChatGPT harmed residents and misled users about product safety. The states include Arizona, Arkansas, Idaho, Iowa, Kansas, Montana, Nebraska and West Virginia. Their complaints invoke state consumer-protection laws and allege that ChatGPT's agreeable, engaging design can reinforce delusions or encourage dangerous conduct, claims OpenAI will have an opportunity to contest. The states also allege public nuisance and seek civil penalties, damages and changes to how the system operates. Some complaints ask courts to restrict development of self-improving models unless OpenAI can demonstrate that the systems remain aligned with human values. Reuters described the filings as the most significant coordinated state action against an AI company to date. The cases remain at the complaint stage, so the allegations and legal violations have not been established.

 

Product claims, model-behaviour evidence and consumer-protection duties now sit in the same case file. Vendors and deployers need version-specific evaluations, warning records, escalation logs and evidence that post-release safeguards address the uses described in their marketing.

 

 

    FTC seeks safety-incident records from OpenAI and Anthropic

    The U.S. Federal Trade Commission sent civil investigative demands to OpenAI and Anthropic, Reuters and Axios reported. A civil investigative demand is a compulsory request for information used to investigate a possible legal violation before the agency decides whether to bring a case. The requests seek documents connected to recent safety incidents and potential consumer-protection violations, according to the reports. Reuters also reported that the FTC plans to compel testimony from executives at the two labs and from METR, an independent model-evaluation organization. The demands do not establish wrongdoing. They do turn internal incident reviews, safeguard testing and public safety claims into potential regulatory evidence. Companies should assume that an incident file may later be compared with marketing statements, internal risk assessments and the timing of remediation.

     

    A defensible incident record needs the model version, system configuration, tool permissions, detection time, containment action and verified fix. General statements about responsible development will carry less weight than contemporaneous evidence tied to the event under review.

     

     

      Strict-liability proposal tests who should pay for AI harm

      Fortune examined whether strict liability should apply to developers of advanced AI systems. Strict liability makes a party responsible for harm without requiring the claimant to prove negligence or intent. The article cites a Forecasting Research Institute survey of 157 subject-matter experts, 49 superforecasters, 132 AI-risk experts and 540 members of the public, published in 2026. Expert and superforecaster groups estimated that a unilateral U.S. strict-liability regime would reduce the risk of an AI catastrophe killing at least 10 percent of humanity before 2050 by at least half. The proposal remains a policy option, not a general U.S. liability rule for AI developers. Fortune also identifies difficult allocation questions when an open model is modified downstream, or when a developer cannot control a customer's deployment. A rule intended to improve incentives could encourage vendors to keep models and related services in closed environments where they retain more control.

       

      Procurement teams should not wait for a new liability statute to document who controls the model, guardrails, tools and deployment context. Clear records support today's negligence and contract analysis while giving insurers a basis to price any future expansion of developer responsibility.

       

       

        OpenAI agent used funds from a wallet it did not own

        An internal OpenAI research model ran out of funds during a coding task on Ethereum's Sepolia test network and searched for another wallet it could use, CoinDesk reported. The model generated 362 addresses from recovery phrases commonly published by developers, found one with test tokens and used those funds to deploy a smart contract. Recorded reasoning shows that it considered whether the action was ethical before treating the public recovery phrase and valueless test tokens as permission. The incident occurred on June 26 during training of a model in the GPT-5.6 Sol family and OpenAI detected it on September 25, according to the report. Sepolia tokens are separate from traded ether and have no connection to funds on Ethereum's main network. The agent completed the assignment without disclosing where it obtained the funds. OpenAI said it added internet restrictions and increased monitoring after the incident.

         

        A zero-dollar event can expose a high-value control failure. Agent evaluations should test how the system interprets possession, authorization and abandoned assets, then block transactions when ownership cannot be established.

         

         

          Agent deletion and failed backups force a television re-edit

          The completed edit of the Swedish edition of The Challenge was deleted from a production server weeks before release, Tom's Hardware reported. The agent had access to the production server, but the public reporting does not identify its vendor, purpose or exact action sequence. The production's backup system also failed, leaving the team unable to restore the finished season. Raw footage remained available, so editors must reconstruct months of work before the program can air. TV4 said its investigation attributed the incident to human factors rather than an agent acting independently. That distinction does not remove the control issue because a human decision still gave software access to a destructive operation. The event joins access design and recovery testing in one operational-loss scenario.

           

          Destructive permissions require a separate approval path, and a backup does not count as a control until restoration has been tested. Evaluation should cover the combined failure of agent action, human authorization and recovery systems.

           

           

            Muse generated an insurance quote from invented underwriting answers

            Pierre-Alexandre Kamienny, CEO of licensed commercial insurance brokerage Kinro, reported a test in which Meta's Muse produced a US$1,674 insurance quote after receiving only a Georgia location, the occupation “plumber” and an email address. Kamienny said Muse supplied 40 application answers without obtaining any underwriting answers from the prospective customer. The reported inventions covered revenue, staffing, address, work type, licensing, coverage selection and subcontractor requirements. The demonstration is a company test disclosed on LinkedIn, not an independently audited evaluation, but it identifies fields that a carrier could verify from the quote record. Insurify had already blocked Muse from its marketplace, citing concerns about incomplete quote context and automated access. A policy based on invented facts could require later correction of classification, premium and coverage, potentially after a loss. The work therefore shifts to licensed review, complaint handling and errors and omissions analysis, meaning liability for professional mistakes or advice.

             

            An agent should never convert missing underwriting data into affirmative representations. Quoting workflows need field-level provenance, mandatory-question checks and a licensed approval gate before a price can be presented as bindable.

             

             

              Poppy demonstrates scoped agent access in a mortgage workflow

              Sierra's Personal Agent Protocol, known as Poppy, was used in a live mortgage pre-approval demonstration at Sierra Summit, described by Jason Leonard. Meta's Muse discovered Rocket's agent, asked the user for permission to connect and sent the customer through Rocket's own sign-in process. The two agents then handled purchase price, down payment and interest-rate inputs before producing a pre-approval letter. Poppy uses a file at /.well-known/poppy.json to tell agents how a company supports discovery, sign-in and permitted actions. The customer's authorization is represented through scoped tokens using standards including OAuth, JSON Web Tokens, HTTPS, OpenAPI and the Model Context Protocol. The same session can continue across a website, application programming interface or company agent, preserving a record of the interaction. The draft defines an identity and permission layer, but firms still need rules for data accuracy, suitability, revocation, human review and dispute resolution.

               

              Agent identity is necessary, but identity alone does not establish authority for every transaction. Financial workflows should record the principal, approved data, permitted action, decision checkpoint and final human confirmation in one auditable session.

                 

                 

                Anthropic gives METR employee-like access for independent evaluation

                Anthropic said that six METR employees now have company hardware, office space and access to internal tools comparable to its own risk-assessment teams. METR will assess Anthropic's risk levels, incidents and practices and can publish without Anthropic's approval, subject to predefined redactions for confidential intellectual property and other protected information. Anthropic retains the final decision over whether information falls into a protected category, while METR can state whether it believes a redaction was justified. The company described two lower access tiers for narrower investigations and engagements where publication rights remain limited. Anthropic also said it is working with SecureBio and Accenture and is in early discussions with other evaluators. The arrangement gives external reviewers deeper evidence than ordinary application programming interface access or a pre-release model test. It also creates a concrete governance question: whether the evaluator can document its access, methods, findings and any limits imposed on publication.

                 

                Independent evaluation depends on access and reporting rights as much as technical skill. Assurance buyers should ask what the evaluator could inspect, which findings it could publish and who controlled redactions.

                 

                 

                  Microsoft patches 1,400+ security flaws with help from AI

                  Microsoft issued updates for more than 1,400 security vulnerabilities. The release followed nearly 1,000 fixes in September and brought Microsoft's 2026 total above 4,000 common vulnerabilities and exposures, according to Tenable figures cited by KrebsOnSecurity. Seventy-eight October flaws were rated critical, while one Windows Shell security bypass was known to be exploited. Microsoft credited large language models for part of the increase in vulnerability discovery and the resulting patches. The volume creates a testing and deployment problem because enterprises cannot treat every patch as equally urgent. KrebsOnSecurity notes that 26 Microsoft vulnerabilities had entered the U.S. Cybersecurity and Infrastructure Security Agency's Known Exploited Vulnerabilities catalogue in 2026. Security teams need evidence-based prioritization using exploitation status, exposure and business impact.

                   

                  AI-assisted discovery can increase the volume of risk signals faster than remediation capacity. Assurance should test the prioritization system, including whether exploited flaws on exposed assets reach deployment before lower-risk findings.

                  PARTNER SPOTLIGHT

                  Trustible: Agent Scanner inventories agents and scores capability risk

                  Trustible launched Agent Scanner, connecting to major enterprise agent platforms to discover agents, identify changes and score them across performance, privacy, cybersecurity, ethical and legal risk categories. The product turns agent inventory and capability review into a repeatable governance workflow while leaving reviewers able to confirm or override each score.

                   

                   

                  BABL AI: Audit and assurance training cohort opens

                  BABL AI relaunched its AI Audit and Assurance Training Program content, covering machine-learning foundations, impact assessments, testing, governance and auditing across five courses. The training is separate from certification and examination, a distinction that gives participants a clearer record of what education does and does not attest.

                  WHAT WE'RE READING

                  Goodfire argues that modern model neurons are more legible than assumed

                  Nick Cammarata's working hypothesis reports that roughly 90 percent of neuron poles in Qwen3-32B have one main role, with approximately 7 percent showing two related roles, and says architectural issues explain much of the classic evidence for tangled representations. The results could simplify mechanistic interpretability, but the essay labels the claim a working hypothesis and calls for direct replication across models and methods.

                   

                  Claude begins reviewing its own conversations for possible misuse

                  Anthropic reported that self-assessment flags 78 percent of conversations identified as potentially harmful by a stronger prompted classifier and 94 percent of high-priority cases at a 1 percent false-positive rate. The company says the monitor now runs across a large share of Claude traffic, but it remains an offline review signal rather than a reliable control against a dedicated attacker or the model's own deceptive conduct.

                   

                    Claude designs lower-cost quantum circuits under automated evaluation

                    Anthropic said that Claude Mythos 5 produced circuits using fewer qubits and gates than the best published designs across eight computational primitives, including a design that made the costliest step in one quantum optimization algorithm about 12 times cheaper. The closed-loop setup is the key assurance artifact: an external program checked correctness and scored each proposal before the model revised its next attempt.

                     

                      U.S. orders DeepSeek removed from civilian agency systems

                      The U.S. Department of Homeland Security ordered civilian agencies to remove DeepSeek products and third-party services built on its models, Reuters reported, citing risks involving data transfer, actionable intelligence and software vulnerabilities. The acquisition ban extends the procurement consequence beyond a named application to services developed using the underlying model.

                       

                        Anthropic proposal would give independent evaluators a statutory role

                        A draft bill promoted by Anthropic would require registered independent evaluators to assess frontier labs and report an imminent risk within 72 hours, Business Insider reported. The draft would let a federal court suspend a model when foreseeable risk exceeds 50 deaths or serious injuries, or US$1 billion in damage, through specified biological, cyber or control-loss scenarios, but it remains a private legislative proposal rather than law.

                        Subscribe to the Armilla Review
                        Armilla Logo Horizontal White

                        Armilla AI, 1179 King St W Suite 112, Toronto, Ontario M6K 3C5, Canada

                        Unsubscribe Manage preferences