When mobile engineers consider adding conversational intelligence to an Android application, the standard default is consuming a Cloud REST API (OpenAI, Gemini Cloud, Claude). However, when building software for real-world independent professionals—such as private in-home tutors managing minors' schedules, lesson notes, and payment records—that conventional cloud path introduces non-negotiable liabilities:
| Dimension | Cloud AI (REST APIs) | On-Device AI (MediaPipe + Gemma in HomeTutorPro) |
|---|---|---|
| Operating Cost | Recurring token fees that scale linearly with usage. High margin vulnerability. | 0.00 € per invocation. Computation runs directly on device silicon. Infinite scale at zero marginal cost. |
| Privacy & GDPR | Personal data of students (names, phone numbers, prices, addresses) sent to third-party data centers. | Privacy by Design. Zero bytes of sensitive data leave the physical device. Automatic European GDPR compliance. |
| Offline Reliability | Requires continuous, low-latency mobile connectivity. Unusable in basements or low-signal classrooms. | 100% Offline-First. Fully operational on airplane mode with zero mobile data consumption. |
| Network Latency | 400 ms – 1500 ms round-trip overhead for TLS handshakes and payload delivery. | 0 ms network overhead. Immediate deterministic fast-path or on-chip hardware inference. |
Running Large Language Models (LLMs) on resource-constrained smartphones requires a specialized inference runtime. While alternatives like llama.cpp and ExecuTorch exist in the open-source ecosystem, Google MediaPipe Tasks GenAI was selected for HomeTutorPro due to several architectural advantages:
First-Class Android NDK Acceleration: Direct GPU, OpenCL, Vulkan, and NPU pipeline optimization tailored by Google for Qualcomm Snapdragon, MediaTek Dimensity, and Tensor chipsets.
Pure Kotlin Gradle Integration: Eliminates the burden of maintaining fragile manual JNI/C++ wrappers inside the application repository.
Native Gemma Compatibility: Seamless execution of .task and .bin quantized bundles.
16 KB Memory Page Size Support: Fully compliant with Android 15 and Android 17 page size requirements in releases 0.10.27+.
To prevent vendor lock-in and keep our domain logic pure, we implemented a strict 3-tier Clean Architecture with Unidirectional Data Flow (UDF):
┌────────────────────────────────────────────────────────┐
│ PRESENTATION LAYER (UI) │
│ Jetpack Compose • ViewModels • TTS Engine • StateFlow│
└───────────────────────────┬────────────────────────────┘
│ Observes StateFlow / Dispatches Events
▼
┌────────────────────────────────────────────────────────┐
│ DOMAIN LAYER (Core) │
│ Entities • UseCases • SueAgent │
│ (100% Pure Kotlin — Zero Android SDK Dependencies) │
└───────────────────────────┬────────────────────────────┘
│ Implements Repository Interfaces
▼
┌────────────────────────────────────────────────────────┐
│ DATA LAYER (Infra) │
│ Room DB • MediaPipe GenAI C++ • Native Audio Services│
└────────────────────────────────────────────────────────┘The domain agent (SueAgent) orchestrates prompt construction and intent detection without knowing whether inference is executed by MediaPipe or a mock test double.
Small Language Models (SLMs) with 1B–2B parameters cannot store vast factual databases without severe hallucinations. RAG solves this by:
1. Retrieving: Querying the local Room (SQLite) database for relevant records (student balances, weekly schedules, shared lesson PDFs).
2. Augmenting: Injecting structured database results directly into the prompt's --- AVAILABLE DATA --- block.
3. Generating: Constraining Gemma to answer using exclusively the injected context.
User Speech -> Android SpeechRecognizer -> Text Query
│
▼
SueViewModel -> sueAgent.detectActionIntent(query)
├── [Fast-Path: SQL query in 0 ms]
└── [Generative Path: Local RAG + MediaPipe]
├── sueAgent.gatherRelevantContext(query)
│ (Selective Room DB extraction)
├── Token Budgeting & Relational Grouping
│ (Kotlin pre-compression)
├── Gemma Chat Template (2048 token window)
└── On-Device Inference -> Audio PlaybackMobile GPUs operate with strictly bounded VRAM. Dumping uncompressed database rows risks graphics subsystem stalls and Out-Of-Memory (OOM) termination. We enforce proactive compression rules before prompt construction:
Relational Aggregation: When querying shared study resources: if total files ≤ 15, inject the complete chronological breakdown; if total files > 15, group semantically by student (groupBy { it.studentId }), injecting total counts and only the latest resource per student.
Context Window Expansion (MAX_TOKENS = 2048): Configured in MediaPipeModelRepository (TEMPERATURE = 0.3f, TOP_K = 20) to allow multi-part queries (e.g. today's classes + free slots + pending debts) without truncating output sentences.
The assembled prompt follows Google's official Gemma formatting:
<start_of_turn>user
You are Sue, the intelligent on-device assistant for HomeTutorPro.
Your role is to answer the tutor's questions concisely and accurately.
NEVER invent facts. Use ONLY the data provided under --- AVAILABLE DATA ---.
--- TEMPORAL CONTEXT ---
Current date/time: Tuesday, 05 Oct 2026, 09:30
--- END OF TEMPORAL CONTEXT ---
--- WORKING HOURS (AGENDA BOUNDARIES) ---
Working hours: 08:00 to 22:00
--- END OF WORKING HOURS ---
--- AVAILABLE DATA ---
Schedule for Tuesday:
16:30–18:00: Lucia Moreno (Algebra)
18:30–19:30: Carlos (Physics)
Free slots within working hours:
08:00–16:30, 19:30–22:00
--- END OF DATA ---
User query:
What lessons do I have this afternoon and when do I have free time?
<end_of_turn>
<start_of_turn>modelQuantization is Essential: Gemma 3 INT4 reduces model footprint from 4 GB to ~550 MB on disk and ~800 MB active RAM, making on-device execution viable on standard Android hardware (2.5+ GB RAM).
Pre-Filter in Kotlin: Do not rely on LLMs to perform heavy data filtering. Use Room SQL indexes and relational grouping in Kotlin before feeding context into the model.
Decouple Inference: Wrap your ML engine behind a clean domain interface to guarantee deterministic unit testability.
💡 Architectural Rule of Thumb: Treat the on-device LLM as an articulate summarizer, never as an unconstrained logic engine or database. When combined with deterministic SQLite queries and strict token budgeting, on-device AI delivers zero latency, absolute privacy, and 0€ operating costs.