Mobile AI Factory: Building Specialized Language Models for Your Mobile Device
Abstract
Mobile AI Factory is an agentic system that automatically builds task- and phone-specific fine-tuned quantized small language models, using an orchestrator LLM, real device measurements, and adaptive search to match or outperform larger models with much less GPU time.
Small language models (SLMs) running on mobile devices can keep user data private and work without connectivity. But they perform well only when fine-tuned for the target task and sized for the target device, a process that today requires repeated expert tuning. We present {\it Mobile AI Factory}, an agentic system that builds a fine-tuned, quantized SLM for a given task and phone. Given a task specification and a target phone, an orchestrator LLM chooses whether to tune hyperparameters, mine real data, or generate synthetic examples, while fixed rules roll back changes that hurt accuracy. The factory searches from small to large models across base models and quantizations, adapts its accuracy goal as it searches, and uses real measurements of the target phone across inference engines and backends. We evaluate Mobile AI Factory on ten tasks and four Android phones. On an unseen testset, its on-device models outperform two five-shot frontier models on five tasks, by up to 0.34, and match them on three more. Its search comes within 0.03 of exhaustively training all 27 model variants on an entity extraction task, while using 19times less GPU time. Phone measurements also matter: the best configuration changes across phones. Without phone measurements, Claude Opus 5.5, given the full hardware and model specifications, misses the app's requirements in 25\% of easy and moderate cases and in 60\% of hard ones. Code is available at: https://github.com/evanly-gh/SLM_Factory.
Community
In this paper, we ask whether a specialized language model can be built automatically for a given task and a given phone. Mobile AI Factory is an agentic system that fine-tunes and quantizes small language models, and uses real measurements of the target phone to choose the model, precision, inference engine and backend. Across ten tasks and four Android phones, its on-device models outperform or match five-shot frontier models on eight tasks, and the best configuration changes from phone to phone. Code is available at https://github.com/evanly-gh/SLM_Factory
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs (2026)
- Hibana: Fast Agentic Inference on Unified-Memory Devices (2026)
- Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents (2026)
- SchemaWeaver: Accelerating On-Device Mobile Agentic AI via Schema-Guided Parallel LLM Decoding (2026)
- Extending FunctionGemma for Practical On-Device Mobile Function Calling (2026)
- First-Order LoRA Training with Phone NPUs (2026)
- ATTUNE: Agentic Throughput Tuning for LLM Training (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 4
evanlyhf/MobileAIFactory-CalendarJSON-Qwen3.5-0.8B-Q4_K_M-GGUF
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper