AI Business Concepts doesn't have client case studies yet. Instead of inventing them, this page documents real experiments — run in real accounting work and on sample data, anonymized, with the results shown as they actually happened, including what didn't work. Client results will be published here as projects complete, with permission.
Every entry here follows the same rule: if it's a number, it either came from an experiment you can see on this page, or it's cited industry research. No averages across clients that don't exist. No results we can't show you.
Sixth session on an ongoing build diary: a month-end close application I'm building myself, session by session, with an AI assistant. The earlier sessions are further down this page. This time I asked a second, independent AI to review the project's filed documents cold — no conversation history, just what's on file.
The review came back with real findings, three of which would have caused genuine problems later. Fourteen items got fixed. The instinct was to report that and move on. Instead I re-tested the fixes themselves, narrowly, against the specific claims they made — and they had introduced seven new defects. Three places claimed a task was "done" while a to-do list in the same document still listed it outstanding. A document that had just declared itself "the single source of truth for counts" had a wrong count in its own text. Something edited an hour earlier was still described as unchanged. A finding count didn't match its own findings list. A numbering scheme meant to demonstrate that order isn't identity was itself out of sequence.
A repair pass is a mechanism, and it fails the same way every other mechanism fails: by asserting correctness instead of demonstrating it. If you have AI clean up a schedule, a mapping or a workpaper, the cleanup is not evidence. Test the specific claims the cleanup made. That was the fifth instance of this same pattern I found in a single session — in a numbering scheme, a document mirror, a certification rule, a piece of analysis, and finally in the act of fixing the first four.
The honest position isn't "provably clean." A third check would probably still find something; the returns diminish without reaching zero. The claim that's actually true is narrower: checked twice, ready to hand off, not proven perfect. This is my own build, not a client engagement, and no organization is named anywhere in it.
Fifth session, same build diary. This one built a key-dates calendar — meetings, statutory filings, and prep-start dates derived from them — from a single approved source document.
Building a calendar that mixes confirmed and inferred dates exposed a gap I hadn't planned for: a date confirmed from a filed document and a date derived from a rule render identically in a plain date field. Nothing distinguishes "known" from "probably true," and a reader trusts both equally, which is backwards. The fix was making provenance a required field rather than a documented best practice — every new entry must declare, before it saves, whether its date came from an approved document, was derived and is unverified, or was typed in by hand. That answer is written permanently alongside the date.
The accounting parallel is direct: "confirmed per statement" and "estimated pending statement" need to be distinguishable in the record, not just in someone's memory of how the number was produced. If the system can't tell the two apart, it will eventually be trusted for something it doesn't actually know. It had to be required rather than optional because a similar expectation already existed as unenforced guidance elsewhere in the same project — and had already been silently skipped once under time pressure. A rule that depends on someone remembering it under pressure isn't a control. A rule the entry form itself enforces is.
The same session's continuity test technically passed and still got its first answer wrong — a stale snapshot was read before the live source connected, and it reported two problems that didn't exist. It corrected itself a minute later, unprompted. "It passed" alone would overstate what happened on the first try. Separately, the two-person sync test on this application still hasn't been run, five sessions in.
Fourth session on the close application. No organizational data of any kind was involved — which makes this one fully replicable by anyone reading it.
I opened the AI project that supposedly held four sessions of work and got back "no knowledge." The cause was a naming collision between the project (a knowledge container) and the connected folder (where everything actually lived). Nothing was lost. But it exposed that I had no written rule about which container was authoritative, and no mechanism to keep the answer true over time. So the session became a governance one: a one-page operating model declaring the folder the single source of truth, a context file at the folder root that the tool reads automatically at the start of each session, the previous session's documents finally filed, an open data-classification question resolved, and a fingerprint manifest that flags which documents have drifted since they were last synced.
Two things the AI got wrong along the way, both corrected only after I pushed back: it implied a connected folder would stay attached to every future session (it doesn't — folders are selected per session), and it told me a staging folder was safe to delete (it isn't — that folder is what the drift check compares against). Confidently wrong is the failure mode to plan for, not obviously wrong. If you are about to start multi-session AI work, decide in writing which artifact is authoritative before session two.
All of it was unverified at the time of writing. The real test is whether a cold session, opened fresh, reads these files and resumes correctly — that ran in a later session and is documented above. Team sync remained unproven four sessions in.
An inherited quarterly reporting pack in real accounting work — undocumented, unknown age, no tie-out controls, single preparer. Diagnosed, remediated, and then run live for a quarter. Anonymized throughout: no organization, no real figures, magnitude bands only.
I built roughly 88 live tie-out checks on a pack that had never had one — layer-to-layer amount ties for every month of every data block, mapping-exception lines that name what to fix, and an "amount at risk" column so exceptions get triaged by cost rather than counted. It ended at 86 green, with one documented accepted variance and one informational count. Then it missed something. One key existed as two rows in the data with the balance split between them; refreshing from source wrote the full figure into both, overstating a reporting line by a five-figure amount for the quarter. The reports and the checks read the same two inflated rows, so they agreed with each other perfectly.
What caught it was recomputing the headline figures directly from the raw source, in code, independently of the model — then comparing. After the fix, every line matched to the cent, quarter and year to date, with prior quarter plus current quarter equalling year to date on every line. The rule this produced: a control that reads the same data as the thing it controls has a blind spot by construction. Verifying a model against itself is not verification. With one preparer and no second reviewer, an independent recomputation isn't a nice-to-have — it's carrying part of the review.
There is still no standing duplicate-key control in that pack. The defect that caused the only material error of the engagement would not currently be caught a second time. Publishing the gap rather than the clean version.
Third session on the close application, and the first one that built anything real. Across the two prior planning sessions I had fixed one rule: the system of record stays in our own environment — files we control — specifically so nothing gets trapped in a vendor's tool. No organizational data was involved; the app holds close-process metadata only.
I built the first component, hit a wall — the thing I'd built couldn't actually be shared with a team — and solved it by moving the backbone to a managed third-party database. A good fix. It also quietly overturned the "keep it in our own environment" rule, and I made that reversal without my own blueprint in front of me. At the end of the session I fed the two prior diaries back in and asked the AI to reconcile the new build against them and flag anything that superseded an earlier decision. It caught the contradiction immediately. I then made the reversal deliberately — written down as a formal supersession, with the tradeoff spelled out: I gain live team sharing and automation, I give up some of the "nothing trapped" custody.
Externalize the memory. Each AI session is amnesiac; you are the continuity. A written decision log plus an occasional "where does the new work contradict the old?" pass is cheap, and it does exactly what an auditor's workpapers or an engineer's changelog does — it catches drift. Changing your mind is fine. Changing it without noticing is the risk.
The root cause was a process failure: I started the session cold instead of loading my prior design first. The whole reversal traces back to that. This was a pilot build, not a production rollout — at the time of writing the session's deliverables weren't even filed in the project folder, and the vendor's data-processing agreement was unsigned and unreviewed.
Second session on the close application, run as a deliberate comparison. Whether the "right" way to build a working tool with AI is to design it thoroughly up front or build it iteratively as you use it — tested by doing both, on two different tools. Tool one: a daily task-and-meeting tracker built on the fly with an AI coding assistant, no blueprint, roughly ten hours over nine days. Tool two: this month-end close application, designed across two planning sessions without a single line of it written yet.
Building on the fly was useful almost immediately and taught me what I actually wanted by making me live with it. Features earned their place by solving a real morning annoyance. Planning first produced a coherent scope, a written record of what I chose not to build so scope can't creep back in, and a security and data-handling decision made before any real number was at risk. Each cost something. On the fly, I redid one screen layout five times and had no upfront check on architecture or security. Planning first, after two sessions nothing existed yet.
Neither method is correct in the abstract. Match it to the stakes and the blast radius: a personal productivity tool, build on the fly; a system touching financial data, multiple entities and a year-end audit, plan it first. The only real mistake is using the low-stakes method on a high-stakes build. The same session showed why the domain expert has to be in the room — asked to cut features tied to funding types the organization doesn't have, the AI correctly removed them, but it had filed functional expense allocation (required on every nonprofit's Form 990 and audited statements) right next to those optional fund-accounting features. A careless cut would have deleted a filing requirement along with the extras. AI groups features by surface resemblance; a practitioner groups them by reason for being.
All time figures here are my own estimates, not tracked. And a polished design is the cheap part — it doesn't count for anything until it survives contact with a real close. This was entirely a planning conversation: no organizational data was involved.
First session on the close application. Whether an AI assistant could take a Controller's full requirements and produce a genuinely usable design for a month-end close application — architecture, security assessment, data pipeline spec — without any real financial data entering the tool. One working session, roughly two to three hours. Inputs were requirements, the ERP vendor's public documentation, and the organization's AI-use policy. No GL balances, no vendor or customer data, no dollar amounts.
The session produced a five-module blueprint, a two-page IT integration request, a security assessment mapped against the AI policy, and a detailed GL export/import specification. The architecture it landed on — your own cloud storage as the system of record, structured workbooks as the database, a compact query layer so large files aren't re-read — mirrors how commercial close platforms are built, on infrastructure a small team already owns. Then it missed something I knew from experience: its first-pass export design assumed the standard GL detail report carries vendor and customer names. It doesn't. I flagged it; the corrected design joins small master lists on the stable source numbers.
An AI is a fast architect, not a substitute for domain judgment. The more useful result was one I didn't ask for: pressed for an honest security assessment, the tool ranked five operational risks it judged larger than the training-data question everyone opens with — led by connector write permissions, a tool connection that can act rather than just read. The training-data question is the one everyone asks and rarely the biggest real risk. A tool that tells you your question was the wrong one is more trustworthy than one that only reassures.
A design is not a working system until it survives a real close, and at this point nothing had been built. The direct ERP connection scoped in this session remained unproven — a header-compatibility question was still open — so the plan deliberately starts on file exports instead.
If you work in accounting, you've seen the vendor pitch decks: hours saved, percentages improved, no methodology in sight. You're right not to trust them.
This page is the alternative — experiments you can inspect, run by someone who closes books for a living. When client results appear here, they'll be real, attributed with permission, and shown with the same honesty.
Book a free 30-minute discovery call — or start with the free accounting AI playbook.