PocketMacro Is Live: What It Took to Ship On-Device Meal AI

PocketMacro is on the App Store. $4.99 once, no account, no subscription, no meal photo ever leaves your phone. Download it here.

A few months ago I wrote about putting a multimodal LLM on an iPhone so a calorie tracker could analyze meal photos without a server. That post was the proof. This one is the shipped app.

Two things changed since then. It's called PocketMacro now, not MacroSnap. And almost none of the work between the demo and the App Store had anything to do with inference.

Why the rename

MacroSnap described one gesture: snap a plate, get macros. That was accurate when the app was one screen.

It stopped being accurate once people started scanning barcodes on yogurt cups, typing "chicken and rice, about a cup," re-logging Tuesday's breakfast, and weighing in on Mondays. Photo is still the flagship path, but it's no longer the only door. PocketMacro says where the work happens without promising it only happens through the camera.

Renaming mid-flight costs you icons, bundle ids, screenshots, legal pages, and muscle memory. Shipping under a name that no longer matches the product costs more.

Portions were the real problem, not identity

"Is that chicken or tofu" is the fun problem. "How many grams" is the one that decides whether the log is worth keeping.

A model that correctly names rice and then invents 280 g when you ate 150 g will quietly wreck your day's numbers. It gets worse if you stamp that estimate with real USDA nutrition data, because now the interface looks certain while the grams are still a guess.

So the review screen tracks two separate confidences:

  • Identity confidence can come from a database match.
  • Portion confidence stays moderate unless the grams came from you, a barcode serving, or a household unit the database actually knows.

A row can read "Nutrition matched" while its weight is still flagged as model-estimated. Those are different claims and the UI stopped collapsing them into one number.

Underneath, the reconciler does Jaccard token-set matching at a 0.5 threshold between the model's ingredient names and the bundled USDA database. On a confident match it rebuilds calories and macros from the database's per-100 g values, but leaves the grams model-estimated so the uncertainty range stays wide. One exception: if the model's own household portion ("1 cup"), priced by the database serving weight, disagrees with its gram estimate by more than 1.5x, the household grams win and the item gets flagged for review. Gemma is much better at "one cup" than at "185 grams."

The matcher also learned to refuse. A slice can't price as a whole pizza. Rice can't match rice noodles. Meatloaf can't become a meatloaf sandwich. Tied candidates get rejected rather than picked arbitrarily. Most of the accuracy work turned out to be teaching the matcher when to give up, not making it cleverer.

Describe a Meal, because photos are bad at leftovers

Vision models get polite and vague about brown takeout in a plastic container, and honestly the image doesn't contain the information.

So there's a text path now. You type "pad thai with extra peanuts, maybe half the noodles." It runs through text-only generation on the same Gemma singleton, behind a model lock so text and vision inference don't collide and the open photo chat survives. Strict JSON extraction, one stricter retry if the model won't comply, then the result goes through the same reconciler, validator, and range pipeline as a photo. Every item comes back marked as an AI description estimate with model-estimated weight, so nothing borrows credibility it hasn't earned.

Language beats pixels more often than I expected it to.

The model download grew up

The original post called first-run download the fragile part. It still is. It's a lot less fragile than it was.

What's in there now:

  • A pre-flight free-disk check before any bytes move
  • Chunked SHA-256 verification, streamed, because you can't load a 2.6 GB file into memory to hash it
  • Atomic rename from .part to the final filename, so a partial download never looks installed
  • The model lives in Application Support with iOS's isExcludedFromBackupKey set, so a multi-gigabyte blob stays out of your iCloud backup
  • A one-time migration from the old Documents path, so an earlier install doesn't trigger a fresh download
  • Typed errors (out of space, stalled, integrity failed, HTTP, network, canceled) so each failure gets specific copy instead of "something went wrong"
  • A deterministic download task id, which is what makes Retry resume from the previous byte offset instead of restarting at zero

The download URL pins an immutable Hugging Face revision rather than main, so the hosted bytes can't drift out from under the size and hash recorded in the app. That one is easy to skip and very annoying to debug later.

I also cut the model picker. There used to be a heavier E4B variant as an opt-in next to E2B. It needs a RAM class most iPhones don't have, and asking someone during onboarding to weigh that against a bigger download is a decision nobody was going to make well. It ships as one model: Gemma 4 E2B, ~2.6 GB down, ~4.5 GB installed. If E4B comes back it should be a device-gated default, not a question.

How do you QA a guess?

This is the part I didn't see coming at all.

Normal testing asserts a function returns the right answer. An estimate doesn't have a right answer, so "is the AI good" isn't a test you can write. It's a measurement you have to take, on a real device, against real food.

What I ended up with is a scorecard gate. Before I'm allowed to touch the photo prompt, the description prompt, the match thresholds, or the portion heuristics, I run the corpus on the gate iPhone and diff it against the last run. The scorecard tool reads the raw observations and exits non-zero if:

  • there are fewer than 75 photo cases
  • database match precision on common foods is under 98%
  • any ambiguous match was accepted
  • food recall is under 90% for simple plates or 80% for composite meals
  • median calorie error exceeds 20% for simple meals or 30% for mixed ones
  • median portion error exceeds 25% for countable portions or 35% for plated ones

It also refuses the example template as evidence. That sounds paranoid until you picture yourself at 11pm about to ship on a run of six photos of your own dinner.

The photo corpus starts from a deterministic 45-image subset of Nutrition5k's test split, pulled as metadata plus overhead RGB only, tens of megabytes rather than the 181 GB full archive, and topped up with phone photos over time. Nutrition5k gives you measured calories, macros, and per-ingredient masses, which is the part you can't get by labeling your own lunch.

The Describe a Meal side runs a bundled 50-case corpus from an in-app diagnostics screen on the gate device. A prompt that passes in a desktop harness tells you nothing about what the quantized model on an iPhone XR will do with it.

What didn't change

  • No server
  • No account
  • No meal photo leaves the device
  • SQLite on the phone is still the source of truth
  • Export and delete are still one tap
  • $4.99, once

The economics from the first post still hold, and they're still the whole reason there's no subscription. A scan costs me nothing because your phone does the math, so there's no per-request bill for me to pass along every month. If this app disappeared tomorrow, your downloaded model and local database would keep working, because they were never waiting on me.

Airplane mode is still the test. Turn it on and use the app. Nothing changes.

Get it

PocketMacro on the App Store. $4.99 one time, iPhone. There's a 3-day free trial so you can run the loop before paying, though the clock starts at first launch and the model download eats into it, so do the setup on Wi-Fi when you have a minute. pocketmacro.app has screenshots and the full privacy story.

If you're building anything on-device with health data, photos, or journals, the tooling is genuinely good enough now. Budget most of your time for everything after the first successful inference.

/projects/pocketmacro-live-what-it-took-ship-device-meal-ai austin.amento.dev