Physics Structure-Informed Neural Networks in the TinyML Pipeline: Antisymmetry Preservation, Deployment Accounting, and Training-Budget Sensitivity

There’s a saying floating around the hardware world that sums up one of the biggest mistakes in Edge AI right now: „Don’t kill a mosquito with a cannon.“

Note: Please, never kill insects. They are advanced living beings and deserve respect. If you hurt one by mistake, apologize for it, as I do. Sometimes this can happen indirectly, for example because we leave artificial lights on, or because a door or window is left open.

Here is a short dramatic story about what happened to me in July 2026 while I was making a fresh cup of hot tea. After pouring the tea into the cups, I suddenly heard a noise. I looked toward the hot plate on the stove and, to my shock, saw a bee that had landed exactly on the spot where the teapot had been boiling only a few seconds earlier. I mean, come on, what are the odds? Exactly at that moment, and exactly in that spot, she decided to land.

The instant she touched the hot surface, she must have experienced terrible pain. Poor her, and stupid me. I immediately tried to move her away by blowing air toward her. She seemed to be in shock, with only her mouth moving, almost as if she wanted to say, „All I wanted was to enjoy the smell of ginger, cardamom, cloves, and cinnamon that you added to that Indian chai!“

I placed her on the balcony, gave her a few drops of water and even some date syrup, but sadly she never recovered. I felt terrible that such an amazing living being had flown into my apartment and, instead of being welcomed, encountered a dangerously hot surface.

It made me think about prevention. We take precautions so that humans do not burn themselves while cooking, but what about insects that may enter through an open balcony door without announcing their visit? In 2026, we talk constantly about safety in AI, autonomous vehicles, and technology, but usually from a human perspective. Animals and insects rarely enter that discussion. I can do better. We can all do better.

Now back to the main point of this blog post.

Here’s how it happens. A team builds a prototype to detect vibrations on a machine or monitor a temperature threshold. They grab an NVIDIA Jetson because it’s powerful, it works, and it’s easy (I did the same „mistake“ during my PhD studies when I bought an NVIDIA Jetson TX2 as the platform of choice to power an AI system using only solar energy). The demo impresses everyone. But then they try to scale to 10,000 production units, and the „Bill of Materials Bomb“ goes off. Suddenly, you’re paying $219 per unit with volume discounts for a high-performance module to do a task that an $8 microcontroller could have handled. Your product is dead on arrival because you over-specified the hardware to be safe.

We all know Edge AI is a spectrum. At the high end, applications such as LiDAR fusion and large vision models genuinely need powerful GPU or NPU platforms. In the middle are embedded AI SoCs for larger sensing and vision workloads. At the low end, TinyML workloads can fit on microcontrollers with kilobytes of RAM and milliwatt-scale power budgets. The difficult part is knowing where a particular model belongs on that spectrum before committing to the hardware.

This shift toward smaller, distributed edge devices is already visible in research labs. For example, some groups are now experimenting with clusters of low-cost edge boards to enable collaborative “physical AI” systems across robots, drones, and sensor networks rather than relying on a single powerful edge computer.

This is the problem I explore in my latest paper, “Physics Structure-Informed Neural Networks in the TinyML Pipeline: Antisymmetry Preservation, Deployment Accounting, and Training-Budget Sensitivity”, published today in the Machine Learning and Knowledge Extraction (MAKE) journal (Impact Factor 8.4, Citescore 12.7), Special Issue “Next-Generation TinyML: Innovations in Models, Security, and Applications for Constrained Intelligent Systems”.

Selbstverständlich, I made the code, firmware, experiment scripts, and reproduction material for this work open-source.

The point is not that a structured neural network magically discovers the one perfect chip for a problem. The more useful question is what physical structure survives the TinyML pipeline, and which deployment consequences must be measured separately. Once the trained representation is explicit, we can calculate the memory it actually requires, estimate throughput requirements, examine implementation cost, and determine which hardware platforms are feasible.

The work builds on the Ψ-NN and Ψ-HDL line of research. The central insight is that a neural network is not only a mathematical abstraction. Once deployed, it becomes a physical workload. It occupies memory, executes instructions, moves data, consumes energy, and ultimately has to fit inside a real processor with finite resources.

Physics-structured neural networks make another connection particularly interesting. Some physical relations can be encoded directly into the architecture rather than being expressed only through a loss function. In the current paper, the relation I test is specifically the odd-in-x antisymmetry of the Burgers equation:

u(−x,t) = −u(x,t)

This is an important distinction. The paper does not claim that every possible physical constraint is automatically preserved. The empirical result concerns this specific Burgers antisymmetry. I compared the directly trained structured Ψ-NN against seven alternatives under a common Burgers protocol, including a size-matched dense network, quantization-aware training, knowledge distillation, iterative magnitude pruning with fine-tuning, structured pruning, low-rank SVD with fine-tuning, and single-shot pruning.

The structured Ψ-NN achieved about 6.0% relative L₂ error with an antisymmetry residual of approximately 0.006. The size-matched dense model reached about 28.1% relative L₂ error with an antisymmetry residual of 0.128. The quantization-aware model reached approximately 33.9% and 0.133, while the distilled dense model reached approximately 34.5% and 0.210.

The conclusion is not that compression is universally inferior. Compression can still be useful for model size, conventional prediction accuracy, available hardware kernels, or platform-specific latency. The result is narrower and more interesting: none of the seven evaluated baselines matched the structured model on the joint objective of Burgers accuracy and antisymmetry preservation under the tested protocol.

The reason is structural. The Ψ-NN does not need to rediscover antisymmetry after compression because the sign relation is already encoded in its connectivity. A second result concerns how clustering should be interpreted for deployment. A model with 13 clusters does not store only 13 parameters. That is not how deployable memory works. A clustered model has to store both the cluster centroids and a relation or index table that tells the implementation which centroid belongs at every original weight location. Thirteen FP32 centroids require only 52 bytes, but the index table still scales with the original number of parameters. Under the one-byte indexing assumption used in the paper, the resulting FP32 weight-memory reduction approaches roughly 4×, not hundreds of times.

So one of the simplest but most useful results of the work is this: centroid counting is not memory counting. Once the representation is known, it becomes possible to evaluate the actual deployment requirements more carefully. I exported the models into bare-metal Cortex-M firmware and compiled them with the ARM embedded toolchain. There is an important measurement boundary here. Flash usage and statically allocated RAM are obtained directly from the linked firmware binary. Executed instruction counts and firmware numerical outputs are evaluated under deterministic Cortex-M target-ISA emulation. Latency and energy are then derived from the instruction counts, CPI assumptions, clock parameters, and vendor electrical specifications. They are not physical-board or wall-plug measurements. Even with that limitation made explicit, the results are useful. The Ψ-NN FP32 firmware requires approximately 10,788 bytes of flash, 976 bytes of static RAM, and 25,578 instructions per inference, while preserving an antisymmetry residual of approximately 0.0049. The Ψ-NN INT8 version reduces flash to approximately 5,368 bytes and retains an antisymmetry residual of approximately 0.0047. The dense FP32 implementation, by comparison, requires approximately 34,500 bytes of flash and 74,753 instructions, with an antisymmetry residual of approximately 0.1219.

The implementation also revealed two useful surprises. First, the structured network uses slightly more static RAM than the dense network, 976 bytes versus 928 bytes, because its branch structure keeps more intermediate activations alive. Second, the INT8 structured implementation executes fewer instructions than the clustered representation because clustering introduces centroid-table lookups. This is why parameter count and mathematical compression ratios are not enough. The target implementation tells you things the high-level model cannot. Another important limitation is that structure does not automatically make a fixed-capacity model smaller. At matched parameter count, structured and dense implementations have similar flash and instruction requirements. What structure provides in the Burgers experiments is better accuracy and much stronger antisymmetry preservation at the evaluated footprint.

The same caution applies to platform selection. For the small Burgers models evaluated in the paper, the platform constraints are non-binding. Both the structured model and the dense model fit on the nRF52840. The structured architecture is therefore not what uniquely enables the microcontroller in this case. The hardware-screening methodology becomes more useful when workload size grows. The paper includes an illustrative larger multi-physics workload where the throughput requirement eventually becomes high enough to eliminate the TinyML tier. That is where a quantitative platform-screening method begins to discriminate between candidate hardware.

The intermittent-training experiments also changed my interpretation of renewable-powered learning. The original intuition was tempting: perhaps stopping and restarting training under intermittent solar power changes the optimization trajectory in a useful way. Perhaps interruption behaves like a kind of temporal regularization. Under complete lossless checkpointing, however, the answer is much simpler. If the checkpoint restores the model parameters, Adam first and second moments, scheduler state, step count, and the full optimization state, then stopping training and resuming later is mathematically just a pause.

Under this null model:

B → E = 0

The interruption schedule itself has no independent effect. What changes is the effective training budget. The schedule becomes important only when the assumptions of the null model are broken. Stochastic interruption timing still has no additional effect when checkpoint restoration remains lossless. But discarding the Adam optimizer state can produce very large errors on some problems. Rollback is another failure mode. If the checkpoint interval is too long relative to the available uptime, the system repeatedly performs optimization work that is later discarded. The practical rule is therefore straightforward: checkpoint the complete optimization state and make the checkpoint interval short enough relative to the expected uptime.

The larger question is how sensitive the learning problem is to receiving fewer optimization steps. Across eleven benchmark problems, halving the effective training budget produced error multipliers ranging from approximately 0.97× for Burgers to 23.70× for Poisson. Laplace was approximately 1.33×, Klein-Gordon 1.49×, Allen-Cahn 1.91×, Korteweg-de Vries 2.13×, Heat 2.30×, Helmholtz 3.82×, and Advection 4.45×. Wave and the memristor remained statistically unresolved under the robust estimator. One particularly interesting comparison is between Laplace, Helmholtz, and Poisson. All three are elliptic, linear, time-independent, second-order equations under the four descriptors tested, yet their budget-sensitivity factors span approximately 1.33×, 3.82×, and 23.70×.

I tested PDE class, temporal coupling, nonlinearity, and derivative order as possible predictors. In this eleven-problem dataset, none of the four descriptors reached statistical significance for budget sensitivity. This does not prove that budget sensitivity is fundamentally unpredictable. Better descriptors may exist. The practical conclusion is that if training-budget sensitivity matters for a deployment, it should be measured instead of inferred from a broad equation category.

I also tested an adaptive regularization mechanism controlled by κ. In Equation 37 of the paper, κ controls the transient amplification of the regularization term during the active training stage. For Burgers, increasing κ from 0 to 2 changes the test-MSE improvement from approximately −8.5% to −11.2% relative to the continuous baseline. Negative values mean lower test MSE and therefore better performance. The additional improvement over the full κ range is only about 2.7 percentage points, so this is a small Burgers-specific result rather than evidence that intermittent training is generally beneficial.

Finally, the sustainability result needs to be attributed correctly. The paper compares a GPU-class baseline scenario with an nRF52840-class microcontroller scenario over five years. The estimated lifecycle carbon is roughly 238 kg CO₂ for the GPU-class scenario and approximately 5.35 kg CO₂ for the microcontroller scenario, corresponding to about a 45× difference under the 24/7 deployment assumption. But approximately 99.9% of that difference is hardware-led. Training the Burgers model consumes about 0.75 kWh in the scenario. Using the grid carbon-intensity assumption gives approximately 0.356 kg CO₂, while the solar lifecycle-intensity assumption gives approximately 0.036 kg CO₂. That training-emission difference is useful, but it is very small compared with five years of deployment. The structured representation therefore does not single-handedly create the 45× carbon reduction. Both small Burgers models fit the microcontroller. The hardware substitution dominates the lifecycle comparison, while the structured architecture improves the accuracy and antisymmetry obtained at the available footprint.

The cost story follows the same logic. Comparing an approximately $249 Jetson Orin Nano-class module with an approximately $5 nRF52840-class microcontroller gives a module-price difference of about 98%. Across 10,000 devices, the hardware difference alone is approximately $2.44 million. Again, this is not a claim that the Ψ-NN itself saves $2.44 million. It is a reminder that hardware over-specification becomes extremely expensive when multiplied across a large deployment.

The broader lesson is simple. A neural network is not only a mathematical function. Once deployed, it becomes a physical workload with real memory, computation, power, cost, and carbon consequences. The key question is not only how small the model looks mathematically, but which properties survive deployment. Encode critical physical relations explicitly, count the full stored representation, evaluate the compiled implementation, separate training-budget loss from checkpoint failure, and attribute sustainability gains to the mechanism that actually produces them.

We should not try to make every neural network tiny. Some applications genuinely require powerful accelerators. The goal is to understand what a model honestly needs, preserve the physics that matters, and choose the smallest platform that genuinely satisfies those requirements. That is how we stop „killing mosquitoes with cannons“.