Why on-device AI can't handle supply chain planning

The useful question is not whether on-device AI can work in supply chain. It already can, in the right layer. A camera can catch a pallet mismatch. A handheld scanner can flag a pick error before the tote moves. A vibration model can keep watching a line-side asset when the network stutters. Those are bounded execution tasks: local data, immediate latency, narrow decision.

The trouble starts when the same pitch is stretched into demand forecasting, S&OP, multi-echelon inventory optimization, or what-if simulation. Those are not just larger versions of pick verification. They are planning workloads, and the on-device AI limitations for supply chain use cases become fundamental there: the system must reconcile enterprise data, run constrained optimization, preserve scenario governance, and see dependencies that do not exist on any one device.

Execution AI and planning AI do different jobs

Execution-layer AI usually answers a question that is physically close to the sensor: does this carton match the order, is this pallet damaged, is this temperature reading abnormal, is this operator about to scan the wrong slot? The model may be sophisticated, but the decision boundary is tight. The data needed at decision time is mostly local. The consequence is immediate. The answer can often be a classification, detection, ranking, or alert.

Planning-layer AI answers a different kind of question: how much inventory should sit across a network, which demand signal should drive the next cycle, what happens if a supplier misses a ship date, which constraint is binding in a regional allocation plan, and how should finance, sales, manufacturing, and logistics reconcile one operating plan? Those questions depend on item hierarchies, lead times, supplier calendars, customer priorities, open orders, promotions, capacity assumptions, substitution rules, and ERP exceptions.

That difference matters because on-device AI is typically designed around inference: compress a model, quantize it, run it close to where data is generated, and avoid a cloud round trip. Supply chain planning is not mainly an inference problem. It is a data-federation and constrained-optimization problem that may include AI models inside it, but the hard part is not merely producing one local prediction.

The hardware ceiling becomes physical fast

The ceiling is not abstract. F22 Labs describes mid-range on-device AI hardware at roughly 4-8 GB of RAM and 5-15 NPU TOPS, while flagship devices reach about 16-24 GB of RAM and 40-70 TOPS. That guide is primarily about consumer-style on-device AI rather than industrial planning platforms, but the numbers are useful because they make the constraint visible: even strong local devices have finite memory pools and accelerator profiles tuned for compact inference, not enterprise optimization runs. [1]

A planning run consumes memory in a less forgiving way. It may need historical demand by item and location, current inventory, open purchase orders, in-transit stock, capacity constraints, forecast overrides, service-level targets, bill-of-material dependencies, sourcing rules, and scenario versions. Some of that can be summarized. Some of it cannot, because the exception is exactly where the planner needs the system to behave correctly.

Floating-point throughput is only part of the issue. NPUs are good at accelerating specific model operations. A planning engine also has to solve mathematical programs, iterate through scenarios, respect constraints, and coordinate outputs across sites. That is a poor match for a single device whose main advantage is local low-latency inference.

This is why a clean edge architecture can be the wrong answer for a planning workload even when the device itself is impressive. The device may make a local decision quickly and still be structurally blind to the network-level tradeoff.

Planning workloads need a network view

Multi-echelon inventory optimization is usually the first workload to expose the category error. It does not ask one warehouse how much stock it wants. It asks how inventory should be positioned across suppliers, plants, ports, distribution centers, stores, channels, and customer commitments under service, cost, lead-time, and variability constraints. A local model at one node can see local stock and local demand. It cannot decide whether upstream inventory should be held centrally, pushed forward, reserved for a higher-priority channel, or rebalanced because another region is about to miss service.

Demand forecasting has a similar problem, though it can look deceptively edge-friendly at first. A local node may forecast a site, a lane, or a product family. Enterprise forecasting, however, has to reconcile signals across time horizons and hierarchies. A promotion forecast may affect distribution capacity. A lost-order pattern may be hidden if the system sees only shipments. A channel mix shift may appear as a local anomaly unless the model can compare the broader commercial plan.

What-if simulation pushes even harder. The value of a scenario is not that it produces one more number; it lets planners compare feasible alternatives under shared assumptions. If a supplier delay, demand upside, transport constraint, and inventory policy change all interact, the system needs common data definitions and governed scenario state. Running fragments of that logic on disconnected devices creates a reconciliation job, not a planning answer.

S&OP is the least plausible candidate for pure on-device execution because its main problem is organizational and data alignment. The workload crosses sales, finance, operations, procurement, and logistics. Even if local AI can draft a narrative or detect an exception, the operating plan has to reconcile central assumptions and preserve a decision trail.

Where each supply chain use case hits the wall

The important distinction is not whether an edge model appears anywhere in the process. It can. A good execution signal can improve a planning input. If a dock camera catches repeated pallet discrepancies, that evidence should flow into inventory accuracy, supplier performance, and exception management. But the moment the system has to decide the policy response across locations, the workload has moved above the local device.

This is also where inventory optimization examples can become misleading. An edge model can notice that a shelf, bin, or station is wrong. It cannot replace the planning layer described in a cloud-based inventory optimization workflow, where reorder logic, service targets, lead-time variability, and network tradeoffs have to be evaluated together. For that distinction, a resource such as AI inventory optimization for retailers is a better comparison point than a local vision demo.

Accuracy loss is not just a local defect

One manufacturing example illustrates the risk, though it should not be overextended. HiveMQ cites a quality inspection case where cloud accuracy was 97% and edge accuracy was 91%. That is a quality-inspection scenario, not a supply chain planning benchmark, so it does not prove that planning models would lose the same amount on-device. [2]

The analogy is still useful because planning errors compound. A small degradation in demand sensing can affect safety stock. A distorted safety-stock recommendation can affect replenishment. A replenishment miss can trigger expediting, allocation, service failures, or excess stock somewhere else. The planning layer is full of coupled decisions, so local accuracy loss is not always contained locally.

The deployment evidence points in the same direction. HiveMQ reports that 27% of organizations identify edge deployment complexity as a top barrier, and that only 34% of industrial sites have real-time data streaming in production. Those figures are about industrial edge deployment broadly, not planning specifically, but they matter because planning depends on timely, trusted data movement before optimization can even begin. [2]

If the plant, warehouse, supplier portal, ERP, and planning platform do not agree on what happened, putting more intelligence on a device does not remove the reconciliation problem. It can even add one more version of the truth unless the architecture is explicit about synchronization, exception ownership, and master-data governance.

Local query economics do not transfer cleanly to planning

Some on-device AI arguments are persuasive in their proper lane. MindStudio estimates that about 71% of ChatGPT-style queries could run locally, presenting the figure as part of a broader argument that AI economics are shifting away from cloud-only inference. The estimate is vendor-sourced and does not publish a methodology in the brief available here, so it is best treated as directional rather than audited evidence. [3]

Even if that estimate is broadly right for conversational or personal-assistant workloads, it does not settle the supply chain planning question. A local answer to a user prompt is not the same thing as optimizing inventory across a network. Planning queries are usually requests against live enterprise state: current orders, constrained capacity, supplier status, inventory records, committed forecasts, and scenario assumptions. The system has to retrieve, reconcile, compute, and govern the answer.

A planner asking, hypothetically, whether to move stock from one distribution center to another is not simply asking a language model to generate advice. The answer depends on available-to-promise rules, transport cost, target service levels, downstream demand, open replenishment, substitute items, customer priority, and whether the inventory record is clean. If those inputs are not present and current, the prettiness of the local response is irrelevant.

Cloud planning is not automatically good planning

It would be too easy to turn this into a cloud victory lap. That would be wrong. Cloud planning deployments fail too. They fail when item masters are dirty, calendars are inconsistent, planners do not trust the recommendations, ERP exceptions overwhelm the integration design, or scenario governance becomes a slideware ritual. Centralized compute does not fix bad planning discipline.

The narrower point is architectural. Cloud or SaaS planning platforms are better aligned to these workloads because they can centralize or federate the data, run heavier optimization, manage shared models, and preserve scenario state across functions. That does not make every vendor claim credible. It only means the workload belongs in an architecture that can see and compute across the enterprise.

This is why buyers evaluating o9, Kinaxis, Blue Yonder, SAP IBP, or similar planning platforms should ask less about generic AI capability and more about workload placement. Which models run where? Which data is required at decision time? Which optimization is centralized? What is merely an edge signal feeding the planning layer? What happens when local inference disagrees with the system of record?

A vendor can legitimately offer edge AI for inspection, sensing, or exception capture and also use cloud infrastructure for planning. That is not inconsistency; it is proper workload separation. The warning sign is a pitch that blurs the layers and implies that because one local model can make one fast decision, the same architecture can run S&OP or multi-echelon optimization.

How to read on-device AI claims in 2026

The buyer's test should be practical. If the use case needs local sensor data, millisecond response, offline tolerance, and a bounded output, on-device AI may be the right architecture. If the use case needs multi-site data, cross-functional reconciliation, mathematical optimization, scenario comparison, or governed planning assumptions, it belongs in a cloud or enterprise planning layer.

That test separates strong edge candidates from planning claims that are really category errors. A dock camera, scanner, or line-side anomaly model can improve execution and feed cleaner signals upstream. A forecasting engine, S&OP model, or inventory optimizer has to work across the network, not just near the event.

For teams already comparing platforms, the next useful step is not a generic AI feature inventory. It is a vendor evaluation that ties each claimed capability to data scope, compute pattern, integration burden, and decision ownership. A domain-specific framework such as this supply chain AI vendor evaluation checklist is closer to the real buying problem than asking whether a product has on-device AI.

On-device AI belongs where the data is local, the latency requirement is immediate, and the decision is bounded. Supply chain planning belongs where the system can federate enterprise data, run large-scale optimization, and govern scenarios centrally.

References

1. What Is On-Device AI? A Complete Guide for 2026, F22 Labs
2. Edge AI in Manufacturing: Why Is It Harder Than Building the Model, HiveMQ
3. On-Device AI vs Cloud AI: Why the Economics Are Shifting, MindStudio