On-Device or API? Shipping Structured Extraction With Apple 3rd-Gen Foundation Models vs a Cloud LLM

Yes, for classification, routing, and most short-field structured extraction in a mobile app, you can now skip the cloud API entirely. Apple's 3rd-gen on-device Foundation Models, shown at WWDC 2026, run a roughly 20B-parameter sparse (mixture-of-experts) model locally, for free, with no per-token cost and no network round trip. The honest answer for production is hybrid: run the on-device model first, and fall back to a cloud LLM only when the input is large, the schema is deep, or confidence is low. Below is the decision framework I actually use on React Native and native iOS projects.

Do I even need a cloud API for extraction anymore?

For short, well-bounded extraction tasks on iOS, no. If you are pulling a few fields out of a receipt, classifying a support message, or routing a user request to the right handler, Apple's on-device model handles it without sending data off the phone and without a bill attached.

The shift that makes "do I need an API" a real question in 2026 is that the on-device model got good enough at constrained generation. Earlier on-device models were fine at summarizing a paragraph but unreliable when you needed exact JSON. The 3rd-gen model pairs a larger sparse architecture with guided generation, so you define a Swift type and the runtime constrains the decoder to produce a value that matches it. That is the same idea as cloud strict modes, just running on the Neural Engine.

What still pushes me to the cloud is scale and depth: long documents that blow past the on-device context window, schemas with many nested objects, multi-step reasoning, or anything where a wrong field is expensive. For those, a frontier cloud model is still the safer call. Most apps have a mix of both, which is why I default to a hybrid design rather than picking one camp.

If you are weighing this against how cloud providers handle the same JSON problem, I wrote a companion piece on reliable JSON in production with GPT-5.2 and Claude ↗ that covers the strict-mode side in detail.

What can Apple's on-device Foundation Models actually do well?

The on-device model is strong at classification, routing, tagging, short-field extraction, rewriting, and summarization of content that already fits on the screen. It is weak at long-context work, deep nested schemas, and tasks needing broad world knowledge.

Here is how I bucket tasks after shipping a few of these:

• Great fit: intent classification, sentiment and tone tagging, routing a message to one of N handlers, extracting 3 to 8 flat fields (dates, amounts, names, categories), redacting PII before it ever leaves the device, generating short suggested replies.


• Workable with care: extracting a small array of items (line items on a short receipt), light normalization, single-step reasoning over a paragraph.


• Send to cloud: multi-page PDFs, contracts, transcripts, schemas with several levels of nesting, anything requiring up-to-date facts the model was not trained on, or financial and legal extraction where an error is costly.

What does on-device structured extraction look like in code?

You define the output as a Swift type, mark it as generable, and the framework constrains decoding to match it. There is no prompt-engineering ritual to coax JSON out of free text and no parsing of a maybe-valid string.

A typical extraction on iOS looks like this:

import FoundationModels

@Generable
struct ExtractedReceipt {
    @Guide(description: "Merchant name as printed")
    let merchant: String

    @Guide(description: "Total amount in the receipt's currency")
    let total: Double

    @Guide(description: "ISO 8601 date, e.g. 2026-06-11")
    let date: String

    @Guide(description: "Spending category")
    let category: Category
}

@Generable
enum Category {
    case food, travel, software, hardware, other
}

func extract(from text: String) async throws -> ExtractedReceipt {
    let session = LanguageModelSession()
    let response = try await session.respond(
        to: "Extract the receipt fields from this text:\n\(text)",
        generating: ExtractedReceipt.self
    )
    return response.content
}

Because the type drives generation, you get a fully typed ExtractedReceipt back, not a string you have to validate. The enum constrains category to your five cases, so you never get a hallucinated label like "groceries-ish."

In a React Native app I wrap this in a small native module and call it across the bridge. The contract is simple: pass text in, get a typed object out, and surface a confidence flag so the JS layer can decide whether to escalate.

// RN side
import { extractReceipt } from "@/native/OnDeviceExtraction";

const result = await extractReceipt(ocrText);
if (result.confidence < 0.6 || result.tokensIn > 1500) {
  return await cloudExtract(ocrText); // hybrid fallback
}
return result.data;

If you want help wiring a native module like this into an existing RN codebase, that is the kind of thing I do in mobile development ↗ work.

On-device vs cloud LLM: the actual tradeoff

On-device wins on privacy, cost, and offline support; cloud wins on capability, context length, and consistency across platforms. The right choice depends on which of those constraints is binding for your feature.

Two non-obvious points from shipping these. First, on-device is not automatically faster: the first call after launch pays a model load cost, so for a one-shot interaction a warm cloud endpoint can feel snappier. Warm up the session early if latency matters. Second, "free" is real and it changes product decisions. When extraction costs nothing per call, you can run it on every keystroke or every scroll without watching a meter, which unlocks features you would never ship on a metered API.

When should I still pay for a cloud API?

Reach for the cloud when the input is large, the schema is deep, the task needs reasoning or fresh knowledge, or you must support Android and the web with one code path. Those are the cases where the on-device model either cannot fit the input or cannot be trusted with the answer.

My rule of thumb is a short checklist. Send it to the cloud if any of these are true:

• The input exceeds the on-device context budget (long documents, transcripts, multi-page PDFs).


• The schema has more than two levels of nesting or many interdependent fields.


• A wrong extraction has real cost: money moves, legal text, medical fields.


• You need the same behavior on Android and web, so a single server-side model is simpler than maintaining two extraction paths.


• The task needs world knowledge or facts newer than the on-device model's training.

How do I build the hybrid fallback pattern?

Run the on-device model first, then escalate to the cloud only when a routing rule trips. This gives you free, private handling for the common case and frontier quality for the hard case, without paying for every request.

I gate escalation on three signals: input size, schema confidence, and an explicit "I am not sure" path. The on-device call returns its parsed object plus a confidence value; if confidence is low or the input is too big, I re-run against the cloud. I also validate the cloud result against the same schema so both paths emit identical types.

async function extractHybrid(text: string): Promise<Receipt> {
  // 1. Fast, free, private first pass on device
  const local = await onDevice.extract(text);

  const tooBig = estimateTokens(text) > ON_DEVICE_TOKEN_LIMIT;
  const lowConfidence = local.confidence < CONFIDENCE_FLOOR;

  if (!tooBig && !lowConfidence && schema.safeParse(local.data).success) {
    return local.data; // 80% of traffic, $0
  }

  // 2. Escalate the hard cases only
  const cloud = await cloudLLM.extract(text);
  return schema.parse(cloud.data); // same schema, same shape out
}

Three things keep this honest in production. Validate both paths against one schema (I use a Zod or equivalent definition shared across the bridge) so a caller never has to know which model answered. Log the escalation rate as a first-class metric; if 70 percent of requests escalate, your on-device tier is not earning its keep and you should retune the gates or the prompt. And for privacy-sensitive fields, redact on-device before any escalation, so even your cloud fallback never sees raw PII.

The payoff is concrete. On a recent build, the on-device tier absorbed the large majority of extraction calls at zero marginal cost, and the cloud bill only covered the genuinely hard inputs. That is the whole argument for hybrid: you stop paying frontier prices for routine work.

What about evaluating quality before you trust on-device?

Before you ship on-device extraction, build a small labeled eval set and measure field-level accuracy against the cloud model you would otherwise use. Trust the on-device tier only for the fields where it matches closely enough.

I keep this lightweight: 50 to 100 real examples per task, each with the correct fields hand-labeled. I run both models, score exact-match per field, and look at where on-device drifts. Usually it is one or two tricky fields (free-text categories, ambiguous dates) that fail, and the fix is either a tighter enum, a clearer @Guide description, or routing just that field to the cloud. This same evaluation discipline is what separates a demo from something you can put in front of users, and it is the part teams most often skip.

If you are deciding whether your next mobile feature should run on-device, in the cloud, or hybrid, that is exactly the kind of architecture call I am happy to talk through on my contact page ↗. It usually takes one conversation to know which path saves you the most money and headache.

The bottom line

On-device Foundation Models did not kill the cloud API; they changed the default. Start every extraction, classification, and routing task on-device, because it is free, private, and works offline. Reach for a cloud LLM only when input size, schema depth, reasoning, or cross-platform reach forces your hand, and wrap the whole thing in a hybrid fallback so each request goes to the cheapest model that can answer it correctly. Measure your escalation rate, validate both paths against one schema, and you get most of the quality of a frontier model at a fraction of the cost.