2026.09.02 14:20 ~ 15:00 Otter

Surviving the Low Memory Killer with Local AI Runtimes

Android Platform English Interpretation available

Integrating on-device Small Language Models (SLMs) or running continuous background synchronization layers introduces a brutal engineering challenge: memory starvation. When an application executes high-frequency data processing or local model inference, it pushes the device's hardware limits. This frequently triggers Android’s aggressive Low Memory Killer daemon (lmkd), leading to aborted background tasks, severe thermal throttling, and sudden application terminations that completely break the user experience. To ship complex edge workloads successfully, you must treat memory management as a strict system-level allocation problem. Relying on high-level memory wrappers hides the underlying hardware constraints. This session introduces a production-ready system architecture to build a highly resilient, zero-allocation local engine capable of surviving extreme system memory pressure without dropping frames. Specifically, this talk will cover: Navigating Kernel-Level Memory Hooks: How to intercept platform allocation signals (ComponentCallbacks2) to dynamically throttle or pause background data processing loops before the OS intervenes. Bypassing the JVM Garbage Collector: Practical implementations utilizing ByteBuffer.allocateDirect and native memory mapping (mmap) to pipe heavy binary data streams straight to local databases with zero heap inflation. Optimizing NPU Hardware Execution: Strategies to route model matrices directly onto the device's Neural Processing Unit (NPU) using the Android Neural Networks API (NNAPI) instead of falling back to CPU cores. Transactional Resumption and Fallbacks: Architectural blueprints to safely preserve and resume multi-gigabyte background operations when hardware resources are abruptly restricted. Stop fighting platform memory limitations and learn how to optimize your app's core architecture for heavy local runtime execution.

Intended audience

Senior Android Engineers and Tech Leads building local data engines, heavy background synchronization systems, or on-device AI features. Performance and Reliability Engineers tasked with reducing ANRs, minimizing battery drain, and optimizing app memory footprints. Mobile Architects looking for low-level system design patterns to manage hardware resources safely at scale.

Sessions in the same time slot

View all sessions