Measured, not demoed.
I use AI agents every working day and I am building one that has to be right about money. This page keeps the line between those two sentences honest: what is in production use, what is in progress, what I will not claim.
In production use, daily
Since 2025 I've worked inside a large production app with agent tooling as a standard part of the loop: Buildprint, an agentic engineering platform for Bubble that connects the app to Claude Code, and Claude for specs, data analysis and documentation.
- Root cause. A refund webhook flipping refunded invoices back to paid. Duplicate invoices appearing inside a seven-day window. A customer link traced through 289 references to find where the new platform broke it.
- Security. An agent-assisted review of a partner webhook surfaced a hardcoded shared secret. Escalated, rotated.
- Forensics. Sweeps across every live project for fee and pricing errors, batched into CSVs finance could act on.
- Documentation. Flows other engineers relied on during the migration, generated and then checked by hand.
The rule: the agent proposes, the data decides. Every output is checked against production records before anything changes.
Earlier AI work
- Koncile.ai, 2023. Document intelligence for finance teams: AWS Textract OCR turning unstructured invoices into structured data, with automated line-item reconciliation against purchase orders. Pre-LLM. Same problem shape as now.
- Agriculture Technologies, 2022. Pest counts estimated from photos users took of their gardens, with image workflows and historical comparison.
Payout Reconciler
An agent that links bank payouts to invoices. It comes directly from the production problem on the payments page, rebuilt in code on synthetic data so it can be published.
Design. Rules first: amount, reference, date window and fee tolerance resolve most cases deterministically. The model is called only for the ambiguous remainder, the split payments, partial refunds and mangled references that cost humans the most time. Confidence thresholds route low-confidence matches to a review queue. Writes are idempotent, incoming Stripe webhooks are signature-verified, and every proposed link is written to an audit log with its reasoning.
Evaluation. A labelled synthetic dataset built from the failure modes I handled in production. The harness reports auto-link rate, precision, false-link rate, latency and cost per hundred payouts, comparing rules-only against rules plus model. A match the system cannot explain counts as a failure.
Stack. Python 3.12, FastAPI, Claude API with tool use and structured outputs, an MCP server exposing the matcher's tools, Postgres, GCP Cloud Run, pytest, GitHub Actions.
The repository goes up on github.com/SoiCollins as it's built; the demo and evaluation write-up get linked here when the numbers exist. Not before.
What I won't claim
- That I've shipped an LLM system to production. The reconciler is the first, and it's published only once it has numbers.
- That Python, TypeScript, React or Docker are languages and tools I have years in. I'm learning all four now, daily. JavaScript is where my code history is.
- That I have automated-testing experience on Bubble. My strength is manual QA against production data; evals are how I'm turning that into automated discipline.
- A machine-learning research background. I build systems around models and measure them. I don't train them.
Got something to build?
Contract, full-time, or a short paid pilot.