Edge or cloud for machine learning on sensor data: how to decide

A team has a sensor, a model that works in a notebook, and one argument: should inference run on the device or in the cloud? The argument usually goes badly because it starts with preference. We start with five questions about the product, and the answer tends to fall out.

Latency

How quickly must the output act on something? A model that trips a relay when a motor draws abnormal current needs an answer in tens of milliseconds, and a cellular round trip can take hundreds, with no upper bound when coverage drops. A model that flags a slow bearing trend can wait minutes. If the output closes a control loop or protects equipment, put it on the device.

Bandwidth

Raw data is large. A vibration sensor sampling at 10 kHz on three axes with 16-bit samples produces about 60 KB every second, which is over 5 GB a day from one node. Sending that over LTE Cat 1 or LoRa is either costly or impossible. LoRaWAN payloads are limited to a few tens of bytes per message. If the link is narrow, compute features or run the model on the device and send the result.

Privacy

Audio, video and anything that identifies a person or a customer’s process are better kept on site. Sending only a class label or a score changes what can leak if a server is breached. Some customers also forbid raw plant data leaving the premises, which settles the question before engineering does.

Power

Radio time often costs more energy than computation. A small quantised model on a Cortex-M4 or M7, or an ESP32-S3, can run a keyword spotter or a vibration classifier in a few milliseconds. Transmitting a payload over cellular costs far more energy than that inference, so on battery products the edge usually wins.

Update path

Models drift, and sensors age. In the cloud you retrain and redeploy in an afternoon. On a device you need over-the-air update with rollback, a model version stored next to every result, and flash space for two copies. If the device has no safe update path, a model on it is frozen on the day it ships.

A split often works

Many products use both. The device runs a small model and sends results, and a sample of raw windows is sent when the score is borderline. The cloud retrains on those, and the new model goes back down as an update. This keeps bandwidth low and still lets the model improve.

What this does not settle

These questions pick a location, not a model. If the training data is poor, running it closer to the sensor only makes the wrong answer arrive sooner. Collect labelled data from the actual installation before arguing about where inference runs.

---

If you are deciding where a model should run in your product, book a discovery call. We cover this under AI for devices and operations and embedded firmware. For a worked example of on-device signal processing, see FFT on a microcontroller.