RAM Engineering for Android 15-17: Running LLMs Without OOM Crashes

Android 15-17

Memory Management

NDK

MediaPipe

Coroutines

1. The Challenge: Android 15-17 Per-App Memory Restrictions

In modern Android versions (Android 15 through Android 17), Google introduced aggressive Per-App Memory Limits and strict low-memory enforcement daemon subsystems (MemoryLimiter:AnonSwap). If an application exceeds its allocated memory ceiling, the Linux kernel terminates the process without warning.

Running an on-device Large Language Model (even a quantized INT4 bundle) requires substantial working memory:

•

Model Weights: ~500 MB – 700 MB on disk / RAM

•

KV Cache & Inference Buffers: ~150 MB – 300 MB

•

Base Application & Jetpack Compose UI: ~80 MB – 120 MB

Without careful architectural safeguards, launching an on-device model on a mid-range Android phone will inevitably cause an Out-Of-Memory (OOM) crash.

2. Strategy 1: Hardware Pre-Flight Inspection (SueDeviceCompatibility)

Never allow an on-device model to download or execute on incapable hardware. Adhering strictly to Clean Architecture, hardware inspections reside in the Data layer, mapping directly to a pure Kotlin domain entity without Android SDK leakage:

KOTLIN

Copy

// 1. Domain Layer: Pure Kotlin Entity (domain/entities/SueDeviceCompatibility.kt)
data class SueDeviceCompatibility(
    val isSupported: Boolean,
    val reason: SueUnsupportedReason? = null,
    val totalRamBytes: Long = 0L,
    val sdkVersion: Int = 0
)

// 2. Data Layer: Hardware verification in SueModelRepositoryImpl
override fun checkDeviceCompatibility(): SueDeviceCompatibility {
    val sdkVersion = Build.VERSION.SDK_INT
    if (sdkVersion < Build.VERSION_CODES.P) {
        return SueDeviceCompatibility(
            isSupported = false,
            reason = SueUnsupportedReason.UNSUPPORTED_ANDROID_VERSION,
            sdkVersion = sdkVersion
        )
    }

    val activityManager = context.getSystemService(Context.ACTIVITY_SERVICE) as? ActivityManager
    val memoryInfo = ActivityManager.MemoryInfo()
    activityManager?.getMemoryInfo(memoryInfo)
    val totalRam = memoryInfo.totalMem
    val isLowRam = activityManager?.isLowRamDevice ?: false
    val minRequiredRam = 2_500_000_000L // 2.5 GB threshold

    if (isLowRam || (totalRam in 1 until minRequiredRam)) {
        return SueDeviceCompatibility(
            isSupported = false,
            reason = SueUnsupportedReason.INSUFFICIENT_RAM,
            totalRamBytes = totalRam,
            sdkVersion = sdkVersion
        )
    }
    return SueDeviceCompatibility(
        isSupported = true,
        totalRamBytes = totalRam,
        sdkVersion = sdkVersion
    )
}

If a device does not qualify, the assistant stays inactive and the app functions normally in classic mode, preserving system stability.

3. Strategy 2: Ultra-Light Cold Start (~80 MB)

A common architectural flaw is initializing machine learning models during Application.onCreate(). In HomeTutorPro, the base app boots consuming only ~80 MB of RAM. MediaPipe binaries and model weights are strictly lazily loaded on background dispatchers only when the user explicitly triggers the voice interface.

4. Strategy 3: Parallel Warm-Up During Speech

To achieve zero perceptible latency without keeping the model perpetually pinned in memory:

•

1. Simultaneous Dispatch: When the user taps the microphone button, SueViewModel simultaneously activates SpeechRecognizer on the main thread and initiates InferenceRepository.warmUp() on Dispatchers.IO.

•

2. Concurrent Weight Mapping: As the tutor speaks for 2 to 4 seconds, model weights are mapped into memory concurrently.

•

3. Zero-Delay Execution: By the time speech finishes, the model is fully initialized and ready to ingest the RAG prompt with 0 ms of user-perceived delay.

5. Strategy 4: Smart Idle Auto-Release (Automatic RAM Reclamation)

Models must not stay resident in memory indefinitely. We implemented a multi-stage timer policy:

•

Idle Timeout (120 seconds): If the assistant remains open without user interaction for 2 minutes (IDLE_RELEASE_TIMEOUT_MS = 120_000L), weights are automatically released via inferenceRepository.release().

•

Dismiss Timeout (30 seconds): When the user dismisses the voice overlay, a 30-second cleanup timer triggers memory evacuation.

•

Microphone Safety Timeout (15 seconds): Protects against audio lockouts if microphone focus is interrupted.

KOTLIN

Copy

// Coroutine timer managing smart model lifecycle in SueViewModel
private fun startIdleReleaseTimer(timeoutMs: Long = IDLE_RELEASE_TIMEOUT_MS) {
    idleReleaseJob?.cancel()
    idleReleaseJob = viewModelScope.launch {
        delay(timeoutMs)
        inferenceRepository.release()
    }
}

6. Strategy 5: 16 KB Page Size Alignment & NDK Stability

Android 15+ devices with 16 KB memory pages require shared libraries (.so) to be compiled with 16 KB segment alignment (-Wl,-z,max-page-size=16384).

•

Pre-Aligned Binaries: HomeTutorPro utilizes MediaPipe Tasks GenAI 0.10.27+, which provides pre-aligned NDK binaries.

•

Zero SIGBUS Faults: The mmap calls on the quantized .task file map directly into 16 KB memory chunks without triggering kernel SIGBUS faults.

7. Strategy 6: OS Lifecycle Memory Trimming (onTrimMemory)

We bind the model repository to standard Android component callbacks:

KOTLIN

Copy

override fun onTrimMemory(level: Int) {
    super.onTrimMemory(level)
    when (level) {
        ComponentCallbacks2.TRIM_MEMORY_UI_HIDDEN,
        ComponentCallbacks2.TRIM_MEMORY_RUNNING_CRITICAL -> {
            // Immediate native memory reclamation
            inferenceRepository.release()
        }
    }
}

8. Summary

💡 Architectural Takeaway: By combining hardware pre-flight audits, parallel speech warm-up, automatic idle release, and OS memory trimming, HomeTutorPro runs on-device generative AI smoothly across hundreds of device models without a single OOM crash.

GitHub LinkedIn Email