Summary: Alaska Court System’s AVA probate chatbot grew from a planned three-month project into roughly fifteen months of work because the court required 100% accuracy and repeatedly encountered hallucinations. The launch scope narrowed to a cited document navigator, while evaluation fell from 91 questions to 16 because full testing was too labor-intensive.
January 3, 2026: NBC News ran a field report out of the Alaska Court System. To help residents through probate without a lawyer, the courts had built a generative AI chatbot named AVA — the Alaska Virtual Assistant. The plan called for three months of work. Fifteen months later, most of the ambition shed along the way, the launch is now slated for late January 2026, “if all goes as planned.”
The story’s value is not the headline “AI project struggles” — it is the detail. A Q&A bot anchored to official court documents looks like the safest shape generative AI can take. Bolt a 100% accuracy requirement onto it, though, and the engineering looks nothing like the demo. For any team building AI products for law, medicine, or government services, AVA sits about as close to a textbook case as reality offers.
From Three Months to Fifteen
AVA came out of LawDroid, the legal tech firm founded by lawyer Tom Martin, with seed money from the National Center for State Courts (NCSC), which also stationed consultant Aubrie Souza on the effort. The problem being targeted is concrete: transferring a deceased person’s estate means heavy paperwork, most families have no attorney, and the court’s self-help resources and helpline run permanently short-staffed.
Nothing about the technical route is exotic: an enhanced retrieval-augmented generation pipeline that grounds every answer in the court’s existing self-help web content, complete with citations to verifiable sources and suggested follow-up questions. The team benchmarked OpenAI and Anthropic models against each other repeatedly for “accuracy and tone,” and constructed a knowledge base to keep the model from leaning on outside material. Thomson Reuters Institute coverage from as far back as February 2025 already described AVA as “in the final testing phase before launching” — nearly a year before NBC’s account.
What Hallucination Looks Like in Probate Law
The failures deserve a slow read from anyone building vertical-domain bots. AVA conjured resources out of thin air — one version directed users to seek help from an Alaska law school alumni network that does not exist. Stacey Marz, administrative director of the Alaska Court System, stated the bar without varnish: “With a project like this, we need to be 100% accurate, and that’s really difficult with this technology.” Souza’s phrasing: they “had trouble with hallucinations, regardless of the model.”
Tone proved to be its own minefield. Early builds radiated empathy, offering condolences to every grieving user — right up until one fed-up user replied, “I’m tired of everybody in my life telling me that they’re sorry for my loss.” The team stripped the sympathy out entirely. Developer Martin’s read on model behavior is just as quotable: some models are poor rule-followers and “kind of want to prove that they’re the smartest guy in the room” — the exact trait a legal tool cannot afford.
The Reality of Testing and Cost
For accuracy evaluation the team assembled a 91-question probate test suite; running the full set proved too labor-intensive, so it came down to 16 questions. Marz called the whole effort “just so very labor-intensive.” The cost structure thus reads at a glance: inference is nearly free — roughly 11 cents covered 20 AVA queries under one configuration — while human review, content preparation, and iteration consume the actual budget.
The launch design wears its retreat in every corner. The official FAQ states flatly that AVA “uses artificial intelligence (AI) to answer your questions and may make mistakes.” Every answer ships with links back to the court’s self-help probate pages so users can check against the source of truth. And the bot no longer tries to replicate the full role of human facilitators — in Marz’s words, “We’re not confident that the bots can work in that fashion.”
Three Lessons for High-Stakes Deployment
AVA’s fifteen months compress into three lessons for product teams.
First, scope reduction is a legitimate engineering move. AVA earned its launch ticket by retreating from “virtual facilitator” to “document navigator”; the smaller scope is not failure but an honest answer to the accuracy bar. Second, evaluation is a recurring labor cost. Cutting 91 questions to 16 was not corner-cutting but resource reality — and it implies post-launch regression testing needs a standing owner, not a one-time acceptance gate. Third, never underestimate a “simple” grounded Q&A project. A Deloitte survey found fewer than 6% of local government practitioners rank AI as a service-delivery priority — and AVA supplies the explanation: the obstacle is not reluctance but the accuracy threshold stacked on top of review cost, which keeps most projects outside the door.