Grape 1
Grape 1 estimates the calories, grams, protein, carbs and fat of a meal from one photo. It is a 94M-parameter image model sized to run on a phone CPU. It returns plate totals only, with no food list.
Technical ID: grape-1.0-94m-fp16.
Model
- Encoder: SigLIP2-base, patch 16, 224 x 224 (
google/siglip2-base-patch16-224). The last four transformer blocks and the pooling head are fine-tuned. - Calorie head: 64 bins over log(kcal + 1) from 5 to 5,000 kcal, plus seven threshold logits P(kcal > X) for X = 300, 500, 700, 1,000, 1,400, 2,000 and 3,000. The calorie estimate is the mean of the bin distribution.
- Grams, protein, carbs and fat come from separate regression heads.
- The photo and its horizontal mirror are both scored inside the graph, and the two results are averaged.
- Parameters: about 94M. The file is ONNX (opset 17) with FP16 weights and FP32 compute.
Input and output
| Name | Shape | Meaning | |
|---|---|---|---|
| Input | pixels |
float32 [1, 3, 224, 224] | RGB, resized to 224 x 224 with Pillow's bicubic filter (no crop), scaled as value / 127.5 - 1, channels first |
| Output | nutrition |
float32 [5] | kcal, grams, protein (g), carbs (g), fat (g) for the whole plate |
| Output | bin probabilities | float32 [64] | Calorie distribution over the 64 log bins |
| Output | threshold probabilities | float32 [7] | Monotone P(kcal > X) for the seven thresholds |
Use one photo of the whole meal, as a phone app would send it. Our evaluation used JPEGs with the longest side at 1,024 pixels and quality 90.
Files
| File | Bytes | SHA-256 |
|---|---|---|
grape-1.0-94m-fp16.onnx |
187,559,154 | b11afe48113d474c3710319b817ed5aebcae267e5ba5571349c9531bfbe9a269 |
How to run
import numpy as np, onnxruntime as ort
from PIL import Image
session = ort.InferenceSession("grape-1.0-94m-fp16.onnx", providers=["CPUExecutionProvider"])
image = Image.open("meal.jpg").convert("RGB").resize((224, 224), Image.BICUBIC)
pixels = (np.asarray(image, dtype=np.float32) / 127.5 - 1.0).transpose(2, 0, 1)[None]
kcal, grams, protein, carbs, fat = session.run(["nutrition"], {"pixels": pixels})[0]
On phones, use onnxruntime-mobile or onnxruntime-react-native with the CPU provider and 2 threads (the setting our app uses). Resize exactly like Pillow's bicubic filter. A plain bilinear resize moves the estimates by a few kcal on average.
Results
Public benchmarks (PC)
These are scores on public photo sets, computed on a PC. They do not measure accuracy on users' own phone photos. Lower MAE is better. The full protocol (PlateBench) is in the paper.
Holdout slice, read once after every model choice was frozen:
| Set | Photos | kcal MAE [95% CI] | Median relative error | Within 20% |
|---|---|---|---|---|
| SNAPMe home meals (self-reported records) | 296 | 131.2 [117.0, 146.8] | 43.7% | 24.3% |
| JFB calorie-app photos (human-corrected labels) | 200 | 147.8 [129.3, 167.4] | 27.1% | 39.5% |
For comparison on the same photos, the open CalorieK model scores 154.5 and 170.1.
Other public tracks, kcal MAE / median relative error. SNAPMe and JFB here exclude the holdout photos:
| Set | Photos | Grape 1 | Open CalorieK model |
|---|---|---|---|
| Nutrition5k overhead (weighed) | 506 | 50.6 / 17% | 105.0 / 37% |
| Nutrition5k angled (weighed) | 675 | 48.1 / 17% | 57.2 / 24% |
| SNAPMe | 1,178 | 138.3 / 42% | 158.9 / 48% |
| JFB | 800 | 140.6 / 26% | 180.6 / 32% |
| OmniFood8K (weighed) | 2,368 | 172.1 / 42% | 228.1 / 57% |
| NutritionVerse (weighed, large arranged scenes) | 889 | 534.3 / 47% | 657.2 / 66% |
CalorieK was trained only on side views, so its overhead score mostly reflects the camera view. Under its own protocol it reproduces its published 56 kcal. These results come from the research pipeline on a PC.
On a phone (one device, 20 photos)
On an iPhone 16 Pro Max (iOS 27.0, onnxruntime CPU), Grape 1's outputs matched the PC reference on all 20 test photos: kcal identical, macros within 0.001 g. The median time was 1.7 s per photo, including JPEG decode and resize. That time comes from one device and 20 public photos. It is a check of parity and speed, not of accuracy on a phone. Android was not measured.
Limitations
- It is not medical or dietetic advice. See the use restrictions in
LICENSE.md(Attachment A). Do not use it for health, eligibility or insurance decisions about a person. - It underestimates large meals. Every model we tested does this. The largest quarter of meals carries most of the error.
- Relative error is large. On SNAPMe home meals the median photo is about 44% off, and only about a quarter of photos are within 20% of the recorded value.
- It cannot see hidden ingredients. Oil, butter and sauce that the photo does not show are often missed.
- The reference values are imperfect. The public sets use weighed food, self-reported records or model-assisted labels, so part of the measured error belongs to the reference.
- One photo, one plate. The model gives one total for the whole photo.
Training
See the paper.
License and use restrictions
Weights: BigScience Open RAIL-M (
license: openrail), with Canivel LLC as licensor. The full text is inLICENSE.md. In short: commercial use and derivatives are allowed, and the use restrictions in Attachment A bind every user and redistributor, including hosted and API use.Use restrictions (Attachment A): the standard Open RAIL-M restrictions plus health restrictions that rule out:
- insulin or medication dosing;
- clinical nutrition management;
- medical diagnosis or medical decisions;
- presenting the outputs as exact or as medical or dietetic advice.
ACCEPTABLE_USE_POLICY.mdsummarizes Attachment A in plain language and adds no terms. The outputs are approximate estimates, not medical advice.Base model: a modified version of
google/siglip2-base-patch16-224(Apache 2.0).LICENSE-APACHE-2.0andNOTICEare in this repository.
Citation
@misc{canivel2026grapeplum,
title = {Distributional Readouts Make Small Vision-Language Models Better Calorie Estimators},
author = {Canivel, Danilo},
year = {2026},
eprint = {TBD},
archivePrefix = {arXiv},
primaryClass = {cs.AI}
}
Model tree for canivel/grape-1-94m-onnx
Base model
google/siglip2-base-patch16-224