Research Blog
Edge Intelligence & On-Device AI
The question this pillar asks is how much capability survives the move out of the data centre. Not whether a model can technically run on a phone — it can — but what quality, latency and battery cost you actually accept, and which product shapes stop working once you refuse to send data anywhere.
What the device constraint actually costs
Quantisation is where the trade is made concrete. Dropping a seven-billion-parameter model to four-bit weights brings it into the memory envelope of a current phone, at a quality cost that is small on summarisation and extraction and larger on multi-step reasoning. The articles report throughput figures per device class rather than a single headline number, because the spread between a current flagship and a two-year-old mid-range device is wide enough that one number would mislead.
Latency behaves differently on-device than in the cloud. There is no network round trip, so short interactions feel immediate, but prefill on a long prompt is slow and thermal throttling appears in sustained use in a way it never does on a server. The design consequence is that on-device inference favours many short interactions over few long ones, which is the opposite of the shape most cloud-first applications have.
Agents and search at the edge
Mobile agents raise a problem cloud agents do not have: the screen is the interface, and sending it away is exactly what an on-device agent exists to avoid. That forces observation and planning onto the device and constrains how large the planning model can be, which in turn constrains how long a task chain can get before it drifts.
The search work approaches retrieval from an adjacent direction. Rather than matching a query against an index and ranking the results, a deliberative engine drafts an answer and a knowledge graph first, derives search terms from that draft, and uses retrieval to check and correct it. The pipeline is fixed rather than an agent loop, and every intermediate artefact is exposed so the reasoning can be inspected instead of trusted.
Where the edge argument is weakest
The honest counter-argument is that most applications do not need any of this. If the data is already in a data centre and the user expects a network round trip anyway, on-device inference adds engineering cost and subtracts model quality for no user-visible gain. The edge case is strong where data cannot leave the device, where connectivity is unreliable, or where per-request cost dominates, and weak everywhere else. These articles try to say which of those applies rather than treating locality as a virtue in itself.
llamafu vs llama.rn: On-Device LLMs for Flutter and React Native
A practical comparison of llamafu, llama.rn and llama_cpp_dart for running llama.cpp models in mobile apps: what each binding covers and how to choose.
slorg vs Algolia vs Meilisearch: Plan-First Search
slorg drafts an answer, builds a knowledge graph and then searches the web. Why that differs from Algolia, Meilisearch and Typesense, and when each wins.
ukkin vs ChatGPT Operator: On-Device vs Cloud Agents
Comparing ukkin with OpenAI's Operator and Anthropic's computer use: a phone-side Android agent against cloud-model agents, on privacy, platform and capability.
Deliberative Search: When the Engine Reasons Before It Retrieves
Traditional search retrieves first and ranks later. slorg inverts the order: a fixed six-step pipeline drafts an answer and a plan before it queries the web.
Autonomous Mobile Agents: ukkin's Architecture for On-Device AI
How ukkin builds phone-side agents from plain English and drives Android apps through the Accessibility Service, and where on-device autonomy needs limits.
Running Language Models on Your Phone: The llamafu Experiment
What it takes to run an LLM on a phone with no cloud dependency: the memory arithmetic, quantisation trade-offs, and how llamafu brings llama.cpp to Flutter.