Key Takeaways

• The Shift to Pragmatic AI Architecture: The industry is moving away from a “biggest model wins” mentality toward finding the most cost-effective model that is “good enough” for the job. Experts predict that over 90% of AI tokens will come from open-weight models within the next two years.
• Solving the Data Sovereignty Tradeoff: Regulated, data-sensitive fields—such as legal, healthcare, and banking—no longer have to compromise between robust cloud-based performance and strict on-premises security. Advanced open-weight models now make zero-egress, local deployments completely viable.
• Empirical Proof in Contract Review: Empirical testing of 11 open-weight models on complex contract scenarios proved that a compact 9-billion-parameter model (Qwen3.5:9b) could achieve a perfect 1.00 recall rate on critical issues, rivaling frontier cloud models.
• The Reality of Language and Speed Tradeoffs: Faster processing speeds often compromise accuracy. Furthermore, all tested models showed a noticeable drop in recall and a 15–80% increase in latency when processing German-language employment contracts compared to English variants.
• Hardware Architecture Bottlenecks: Spec sheets can be deceptive. While dense model speeds scale predictably, Mixture-of-Experts (MoE) models face severe memory-bandwidth bottlenecks rather than compute constraints on standard reference hardware like the NVIDIA L4, requiring physical testing before enterprise sizing.

The industry is quietly repricing the AI decision

For the past two years, the AI conversation has been dominated by one question: which frontier model is smartest? That question is losing relevance. As CNBC reported this month, the industry is shifting from “biggest model wins” to “cheapest model that’s good enough for the job wins.” Perplexity CEO Aravind Srinivas frames the real product not as a single model, but as the system that routes each task to the right model at the right cost, letting a cheaper model handle the bulk of the work and calling in something stronger only when it’s actually needed.

The number attached to that shift is the one worth sitting with. Benchmark general partner Peter Fenton predicts that over the next 18 to 24 months, more than 90% of all AI tokens generated could come from open-weight models: models that any organization can download, fine-tune, and run entirely on its own infrastructure. This shift is being driven not just by economics, but also by a growing global demand for digital sovereignty, particularly in Europe.

That’s not a hypothetical for every industry. It’s a decision some organizations are already in a position to make.

Why this matters beyond legal

Organizations that cannot allow data to move outside their own secure infrastructure (e.g., due to privileged legal files, patient records, banking-secrecy-protected account data, government or defense information) have historically lived with an uncomfortable tradeoff: use the most capable AI model available, and accept that your data goes through a third party’s cloud API to get it, or protect your data, and settle for a materially weaker model running locally.

Law firms are a clean example of this tension. Attorney-client privilege and, in Switzerland, obligations under the FADP make sending client documents to an external API a real governance question, not a formality. But the same tradeoff sits in front of many other industries, including:

1. Legal — contract/NDA review, catching liability and termination-clause issues before signature.
2. Healthcare — clinical note summarization and patient record triage without PHI leaving the hospital network.
3. Banking — KYC/AML document screening under banking-secrecy obligations.
4. Insurance — claims document review and policy-wording analysis at scale.
5. Government/public sector — citizen-data processing where cross-border cloud transfer is legally restricted.
6. Defense/intelligence — classified document analysis with zero external network egress.
7. Pharma/life sciences — clinical trial document and regulatory submission review under data protection rules.
8. HR/people operations — large-scale employment contract audits across a multinational workforce.
9. Real estate/notarial services — property contract and deed review, relevant to Swiss notarial practice.
10. Manufacturing/industrial — supplier contract and IP-sensitive engineering document review, protecting trade secrets.

Wherever “we can’t send this data outside our own infrastructure” is a hard requirement rather than a preference, this shift toward capable open-weight models is directly relevant.

The premise worth testing, then, is simple. Has open-weight AI actually gotten good enough to close that gap? Or is “open-weight catching up” still just an industry talking point?

The claim vs. the test

Rather than take that claim on faith, or lean on generic model leaderboards, we went looking for something more specific to our use case. General-purpose leaderboards like the Hugging Face Open LLM Leaderboard and LMSYS Chatbot Arena rank models on broad reasoning and chat benchmarks, and hardware tools like CanIRun.ai can tell you whether a model will fit on a given GPU. Neither tells you whether a model will catch the specific clause a Swiss employment contract is missing, or how long that will take on your hardware. That gap is what our benchmark set out to close: O&B, through CAIO Jeff Gutierrez‘s benchmark work in collaboration with Patrick Scheller of CodeCubicle.ch, tested whether open-weight models can meet the bar for a genuinely demanding task: reviewing legal contracts and correctly catching the issues that matter.

We picked contract review deliberately. It’s unforgiving (missing a critical issue in an employment contract or NDA has real consequences), and it’s a reasonable stand-in for the kind of document-heavy, accuracy-critical work that shows up across every one of the industries above.

What we tested

We ran eleven open-weight models, across the Qwen, Mistral, Gemma, Llama, and GLM families, against a set of contract review scenarios in both English and German: NDAs (including a university-specific variant) and employment contracts, each seeded with known flaws. Every model’s output was scored two ways: a keyword-recall layer checking whether it caught the ground-truth issues, and an LLM-as-judge layer providing a second, independent read. We set the bar at 80% recall on critical-severity issues per language. Below that, a model isn’t reliable enough for real use, full stop. Every run was scored, including failures. Nothing was filtered out after the fact.

Accuracy was only half of what we measured. A model that’s perfectly accurate but takes minutes per document isn’t practical at real firm volume, so we measured processing speed on the same reference hardware for every model too.

What we found

A few things stand out. First, a 9-billion-parameter open-weight model (a fraction of the size of a frontier model) hit perfect recall across every scenario we ran. That’s the headline result, and it’s a direct, real-world data point behind the “open-weight is catching up” claim, not an extrapolation.

Second, speed and accuracy don’t automatically move together. qwen3.5:9b, our accuracy leader, processes a document in about 42 seconds on our reference hardware: solid, but not the fastest model we tested. llama3.1:8b and glm4:9b are the speed leaders, returning results in 22 to 28 seconds, over six times faster than the slowest model we tested (gemma4:31b, at 267 seconds), but both give up real accuracy on German employment contracts to get there. On the NDA-in-English scenario specifically, every model we tested tied at 1.00 recall, so accuracy alone can’t separate them there; glm4:9b was the fastest of the eleven on that scenario (24.3 seconds), narrowly ahead of llama3.1:8b (29.3 seconds). Which model is “best” depends on whether a use case can tolerate a slower, more accurate model, or needs faster turnaround and can compensate with a human review step.

Third, and just as important, we’re publishing the failures too. Every model we tested, including our best performer on other scenarios, struggled specifically with German-language employment contracts, landing at 0.57 recall, well below threshold. That’s a real, reproducible gap, not noise. We think that honesty is the point. A benchmark that only shows wins isn’t a benchmark, it’s marketing. Knowing exactly where a model breaks down, and how fast it runs while doing it, is what lets an organization deploy it responsibly, on the workloads it’s actually proven for.

Accuracy and speed, by language

Language turned out to affect both dimensions we measured, not just accuracy.

qwen3.5:9b is the only model that holds 1.00 recall in both languages; every other model drops on German, mainly because of that same employment_de weak point, not a general German-language penalty. Speed drops too: German scenarios ran slower than English ones for 9 of the 11 models, roughly 15-80% higher latency, likely reflecting longer German contract text and output. For a firm choosing between English-only and bilingual deployment, that’s a real planning input: expect German-language workloads to need more time per document, and to demand more scrutiny on the specific document types (employment contracts, in our tests) where recall drops.

Yes, it can run on your infrastructure

The numbers above were measured on an NVIDIA L4 GPU (AWS g6.xlarge), the reference hardware for this benchmark. Scaled up to enterprise-grade hardware like an NVIDIA A100, using GPU memory-bandwidth scaling (a roughly 6.7x factor over our L4 reference), projected throughput comfortably supports production-scale contract review workloads, even for the slower, more

What this means practically: the CNBC piece also points to AI increasingly running in a hybrid model, with routine work handled locally and harder cases escalated to a larger system. For instance, we are building this into our own workflows, utilizing LLMs of different sizes and speeds to ensure we get the best performance for every task. For data-sensitive organizations, that’s the architecture that matters. Contract review, document triage, and similar recurring workloads can run entirely inside infrastructure you own, with nothing leaving the building or the local AI provider you trust.

One architectural detail is worth flagging for anyone sizing hardware for this kind of deployment: dense and mixture-of-experts (MoE) models don’t behave the same way under the L4’s constraints. A dense model activates its full parameter count on every token, so its speed scales predictably with its size. An MoE model like qwen3.6:35b only activates a small fraction of its 35 billion parameters per token (around 3.1 billion, routed through a subset of its experts), which sounds like it should run closer to the speed of a much smaller dense model. In practice, it didn’t: on our L4 reference hardware, qwen3.6:35b measured 8.3 tok/s, slower than several dense models under half its size. The reason is memory bandwidth, not compute. Independent analyses of MoE serving confirm this: in low-latency, small-batch settings, MoE inference is fundamentally memory-bandwidth bound rather than compute-bound, because token-level routing forces irregular expert-weight movement that stalls the GPU waiting on data, not math. The L4’s 300 GB/s bandwidth still has to move the relevant expert weights in and out as routing shifts token by token, so the “small active parameter count” advantage doesn’t translate into proportional speed on bandwidth-constrained hardware. It also doesn’t shrink VRAM requirements: every expert has to live in memory whether the router calls on it or not, since experts can’t be materialized on demand without crushing latency, so the deployed footprint is sized to the total parameter count, not the active one. The practical takeaway: an MoE model’s total parameter count is not a reliable predictor of either VRAM footprint or throughput, and needs to be tested on the target hardware rather than estimated from its spec sheet.

The takeaway for leadership

In business, the optimal goal has always been to use “good enough” tech that is affordable, rather than maximizing capability at all costs. Open-weight models are already good enough for many tasks today, and by 2027, they will be significantly better for many more. Open-weight AI is no longer the compromise option. It’s a legitimate choice, and the case for it isn’t just an industry prediction. Our own numbers back it up, on a workload with real stakes, measured on both accuracy and speed. That doesn’t mean every open-weight model is ready for every job today. Our results show clearly that some aren’t, at least not yet, and not in every scenario. That’s exactly why testing before adopting, on both dimensions, is the right posture, whether the work is legal contract review, clinical documentation, financial disclosures, or anything else where data can’t leave your walls.