Last quarter, a tax advisory firm in Milan asked me to replace their ChatGPT workflow. Their constraint: zero client data leaves the building. GDPR, professional secrecy, and their own peace of mind.
I specced a single workstation: RTX 4090 (24 GB VRAM), 64 GB RAM, Ubuntu Server. Installed Ollama, pulled Mixtral-8x7B-Instruct and a fine-tuned Italian legal model. Connected their document repository via a local RAG pipeline (LangChain + ChromaDB, all on-prem).
Total recurring cost: $0 in API calls. Inference latency: ~2 seconds per query on 4k-token contexts. The partners now draft tax opinions, summarize contracts, and cross-reference regulations without ever pinging OpenAI.
What the stack actually requires - Hardware: One NVIDIA RTX 40-series GPU (4090 ideal, 4080 works for smaller models). No cloud credits, no subscription. - Software: Ollama (free, MIT license), Linux or Windows. Model weights downloaded once. - Time to first productive query: ~4 hours if you know the stack; ~2 days if you’re learning. - Maintenance: Monthly `ollama pull` for updates. Backups of the vector store.
The economics nobody talks about Kairo’s estimates put initial capex at €0 if you already own a capable GPU, otherwise ~€2,000 for a 4090 build. Monthly opex: €0. Time investment: 5 hours/week for the first month, then 30 minutes/month.
Break-even vs. GPT-4 API: month three for a 5-person firm doing 200 queries/day.
Getting the first client took 45 days Not because the tech is hard. Because professionals don’t buy “AI.” They buy “I don’t want my clients’ tax returns on Microsoft’s servers.” My pipeline: referrals from a notary I’d helped two years ago, one LinkedIn post showing a local inference benchmark, zero cold outreach.
Where it breaks - Models still hallucinate on niche jurisprudence. Fix: strict RAG with citation enforcement. - Non-technical partners fear “breaking it.” Fix: a one-page runbook and a shared Obsidian vault with prompt templates. - GPU memory limits context window. Fix: chunking strategy + recursive summarization for long dossiers.
If you run a practice and want to test this on your own files I’ve published the exact hardware spec sheet, Ollama modelfile, and RAG config I used for the Milan firm. No email gate. Download it here: [link to GitHub gist / Notion page].
Questions? Drop a comment, I read every one.
