Small models on a Raspberry Pi: route before you reason

A Raspberry Pi 4B with 8 GB of RAM and no GPU is the test bench for Neurosymbolic Routing for Reliable Reasoning on Resource-Constrained Edge Devices, a paper published on arXiv on 30 September. The authors start from a problem familiar to anyone who has tried to run a language model locally: the models that fit on that kind of hardware fail precisely at the tasks computers are expected to handle well, such as arithmetic, algebra and formal logic.

The paper's thesis is that much of that unreliability can be avoided. Many queries that seem to demand reasoning are in fact structurally deterministic and allow for a fast, exact symbolic solution. Forcing a probabilistic model to approximate them, they argue, sacrifices accuracy and energy for very little in return.

How the router works

The proposal is a neurosymbolic router that classifies each incoming query and dispatches it to the cheapest correct solver. Structured tasks go to deterministic engines, and the small language model (SLM) is reserved for open-ended word problems. Put simply: multiplying 17 by 38 should not go through a language model, whereas a problem written with implicit quantities probably should.

The most interesting part is how the routing logic is built. Instead of hand-coding rules, the authors learn a deterministic finite automaton (DFA) with L, the grammatical inference algorithm Dana Angluin proposed in 1987. L works with two oracles: a membership oracle, which answers whether a given string belongs to the language, and an equivalence oracle, which confirms whether the current hypothesis is correct or returns a counterexample. Here the SLM acts as the membership oracle and a labeled dataset plays the role of the equivalence oracle.

The result is a classifier that, once learned, runs as an automaton: there are no weights to load and no inference to pay for when deciding where each query goes. A DFA can also be inspected, which is much harder with a neural classifier. On such constrained hardware, both things matter.

How it was evaluated

The evaluation used 100 previously untested prompts drawn from well-known datasets such as DeepMind Mathematics and GSM8K, among others. The former gathers synthetically generated math problems organized by areas such as algebra, arithmetic or polynomials; the latter, grade school problems written in natural language. The mix fits the goal, because it forces the router to separate queries a symbolic engine can solve directly from those that require interpreting a problem statement.

A hundred prompts is a small sample, and that is worth keeping in mind before extrapolating. Judging the approach will require a careful look at the accuracy and energy figures in the full paper, especially against the most obvious alternative: using a somewhat larger model and accepting the cost.

Who it is useful for

The clearest case is on-device deployments where privacy or lack of connectivity rules out calling a cloud model: industrial equipment, offline educational tools or embedded assistants. It also fits wherever energy consumption per query is a real constraint, such as battery-powered devices. But the underlying idea applies just as well to much larger architectures.

In the Claude ecosystem the pattern will feel familiar. When an agent in Claude Code delegates a calculation to a tool exposed via MCP instead of working it out within the model's own reasoning, it is applying an informal version of the same idea: sending the deterministic part to a deterministic engine. What this paper adds is a way to learn that decision systematically, with a result that can be read and audited, rather than leaving it to a tool description and the model's judgment.

At ElephantPink we often see the temptation to ask the largest available model for things a library solves in milliseconds. This paper takes the point to an extreme setting, but the lesson holds just as well for a GPU server: before scaling up the model, it is worth checking which queries should never reach it.