The promise of this tutorial: the companion tutorial on Ψ-NN ended with a triumph: a neural network that can’t break a law of physics, and that learns how to be that way on its own. But that network had a quiet problem: it could only talk to other neural networks. This tutorial picks up exactly there. First we’ll watch Ψ-HDL teach that physics-obeying brain to speak the native language of chip designers, so it can run inside a real circuit simulator. Then we’ll watch Ψ-xLSTM give the brain fast reflexes, so it can capture the split-second electrical behavior the original, slower brain was blind to.
As before, every idea stays concrete, each equation arrives gently after the picture that explains it, and each stage comes with a diagram you can follow.
A note on attribution. This tutorial rests on Ψ-NN (Liu et al., Nature Communications, 2025), the structure-discovery method recapped in §1, and the foundation everything here extends. The two methods this tutorial teaches, Ψ-HDL and Ψ-xLSTM (both IEEE Access, 2026), are my own work. See the code behind them here and here. To keep the boundary clear throughout: §1 is Liu et al.’s contribution; Parts I and II are mine.
A 30-second recap: what Ψ-NN gave us (Liu et al., 2025)
Everything in this section is the contribution of Liu et al. (Nature Communications, 2025), the method my own work extends, not part of it. Recall its three moves: Distill → Extract → Reconstruct. A teacher network learns the physics. A tidy student copies it until its many weights clump into just a few repeated values. We read the pattern hiding in those values (a wiring blueprint of „+1 same / −1 opposite / 0 unrelated“, called the relation matrix R), and we rebuild a lean network with that pattern hard-wired in. The payoff: a network that is smaller, faster, more accurate, and physically guaranteed, a symmetry it literally cannot break, like folded-and-cut paper. Hold onto one image: Ψ-NN handed us a structured brain: a network whose few surviving knobs each mean something. That structure is the hero of everything that follows.
Two new problems, two new rungs
The structured brain is wonderful, and almost useless to the people who most need it. Chip designers don’t run neural networks; they run circuit simulators. And the brain has a second, deeper limitation we’ll get to: it’s a slow thinker, blind to the very fastest electrical events. So we add two new rungs to the Ψ-NN ladder, each fixing exactly one limitation.
Part I climbs the first new rung (Ψ-HDL). Part II climbs the second (Ψ-xLSTM). Both are my own work, built directly on top of Liu et al.’s Ψ-NN method: they don’t replace it, they extend it.
PART I. Ψ-HDL: TEACHING THE BRAIN TO SPEAK „CIRCUIT“
Why a perfect model can still be unusable
Before we fix anything, here’s the world a chip designer lives in.
📦 What a „compact model,“ Verilog-A, and SPICE actually are. A circuit simulator (the famous one is SPICE) predicts how a circuit behaves by repeatedly solving equations for every component. To take part, each component needs a compact model: a small set of clean equations describing how its current responds to voltage. These are written in a hardware description language called Verilog-A, basically
current = (some equation in the voltages). The catch: the simulator evaluates each model millions of times inside an iterative loop, so the equations must be few, smooth, and fast. Traditionally a human expert spends weeks or months hand-deriving them for each new device.
Now the gap. A raw neural network is the opposite of a compact model: thousands of meaningless numbers, possibly with non-smooth pieces, slow to evaluate in a solver. So even a perfectly accurate, physics-respecting Ψ-NN brain is a closed box the simulator can’t ingest. People had shown you can hand-port a trained network into Verilog-A, but nobody had automated the path from a learned, structured model to clean HDL code. That missing automation is what Ψ-HDL provides.
The missing fourth step
The idea is almost embarrassingly direct: Ψ-HDL = Ψ-NN’s three steps + a fourth step that writes the HDL code. The full name spells it out: Physics structure-informed neural networks for Hardware Description Language generation.
Stages 1–3 are exactly Ψ-NN. The novelty is Stage 4, and the fact that the whole pipeline runs end-to-end, in minutes, with no human in the loop.
The translation trick: how structure becomes equations
This is the clever heart of Ψ-HDL, and it only works because of the tidying we did earlier. After Ψ-NN’s clustering, the student’s weights are tied: there are only a handful of distinct values, call them P1, P2, …, Pk. When weights are shared like that, the network’s output stops being an opaque tangle and collapses into a clean sum:
ŷ(x) = Σ Pₖ · Φₖ(x)
In words: each unique parameter Pₖ multiplies one intermediate function Φₖ of the inputs. If every weight were unique, no such tidy factorization would exist. It’s the sharing that makes the network readable as equations.
Now the magic. Each Pₖ becomes a named circuit parameter, and each term becomes part of the device’s current or state equation. The paper’s tiny example: if the net learned I = P1·V + P2·V², that’s literally one ohmic term plus one nonlinear term, which we write straight into Verilog-A:
// if the network factored to I = P1·V + P2·V²
I(p,n) < P1V(p,n) P2pow(V(p,n), 2);
Two nice consequences fall out. Piecewise behavior the net learned (e.g. „switch only above a threshold voltage“) becomes simple if/else blocks. And because the student uses smooth tanh activations, the resulting equations are smooth and differentiable, which is exactly what SPICE’s Newton–Raphson solver needs to converge (its Jacobian, ∂I/∂V, stays well-defined). Here’s a condensed version of the real memristor module Ψ-HDL generates, where a learned internal state x controls the resistance:
module memristor_psi(p, n);
inout p, n; electrical p, n;
parameter real R_on 1e3; // low-resistance (ON) state
parameter real R_off 1e5; // high-resistance (OFF) state
parameter real alpha 2.0; // nonlinearity exponent
parameter real tau 1e-3; // switching time constant
parameter real V_set 0.8, V_reset 0.6;
real x, R, dxdt, window;
analog begin
@(initial_step) x 0.1;
R R_on (R_off R_on) pow(1 x, alpha); // state-dependent resistance
window x (1 x); // keeps x inside [0,1]
if (V(p,n) > V_set) dxdt window (V(p,n) V_set) tau;
else if (V(p,n) < V_reset) dxdt window (V(p,n) abs(V_reset)) tau;
else dxdt 0;
x idt(dxdt, 0.1);
I(p,n) < V(p,n) R; // Ohm's law with the learned R(x)
end
endmodule
That’s about 45 lines, human-readable, and it compiles in real simulators (ngspice, Cadence Spectre).
Four test drives
The paper validates Ψ-HDL on four deliberately different problems, and watching what structure it discovers in each is the fun part:
-
Burgers‘ equation (a shock-wave PDE) hides an odd / sign-flip symmetry. Ψ-HDL discovers a
±Winput pattern and reduces 4,920 weights to 12 cluster centers (97.6%), while being ~2.3× more accurate than a standard PINN and converging ~2.3× faster. -
Laplace’s equation (our old butterfly) hides an even / swap symmetry. Exchange the two coordinates and nothing changes. Ψ-HDL compresses 3,320 weights to 8 centers (99.8%) and lands a striking ~54× accuracy improvement. Same network shape as Burgers, opposite internal structure: discovered automatically.
-
A neuromorphic XOR circuit (spiking neurons) maps to a minimal 6-memristor crossbar, hitting 100% logic accuracy in a tiny 120 µm² of silicon.
-
A real memristor, and this is the most interesting. Unlike the PDEs, we don’t hand it an equation; it learns device physics from I-V measurements alone. It compresses 3,360 weights to 13 centers (99.6%), and those clusters turn out to map onto physical operating regimes (sub-threshold, SET, RESET, and distinct transport modes). The structure isn’t arbitrary: it’s the device’s physics, read off the weights.
The honest catch (please don’t skip this)
Here is where the Ψ-HDL paper is unusually honest, and where the real lesson lives. On a clean synthetic memristor dataset, plain polynomial regression and lookup tables actually beat Ψ-HDL on raw accuracy: their error is roughly 8×10⁻⁶ and 5×10⁻⁶ amps, versus Ψ-HDL’s compressed model around 4×10⁻⁴. So why on earth use Ψ-HDL?
Because raw interpolation accuracy is the wrong scoreboard. Ψ-HDL is playing a multi-objective game: small and interpretable and physics-respecting and safe to extrapolate and circuit-ready. A lookup table is fast but bloated (2,500 grid points), physics-blind, and brittle past its grid edges. A polynomial is tiny but has no physical guardrails at all. The decisive evidence is the physics ablation. Turn the physics penalty off (λ = 0) and the network interpolates the training data beautifully, then fails catastrophically the moment it leaves familiar ground: 450+ predictions of physically impossible states (a conduction state below 0 or above 1) and test error ~70× worse. Turn the physics back on (λ = 0.1) and the violations drop to a handful. That is the whole point. A deployed compact model spends its life at operating points it never saw in training. Physics isn’t decoration there, it’s the thing that keeps the model from going insane. Ψ-HDL trades a little interpolation accuracy for a model that stays trustworthy exactly where it matters.
PART II. Ψ-xLSTM: GIVING THE BRAIN FAST REFLEXES
The blind spot: spectral bias
Now the second, deeper limitation. Plain neural networks have a built-in laziness: they learn smooth, low-frequency shapes quickly, and high-frequency wiggles only very slowly — or never. This is called spectral bias. Picture a camera with a slow shutter. It captures a calm scene perfectly, but anything moving fast comes out as a smear. For a device, „fast motion“ means sharp switching transients, ringing, and high-frequency harmonics. A plain PINN nails the steady I-V curve (the DC behavior) and then fails above ~80 kHz. For a memristor that switches in microseconds, that smear is disqualifying.
The cure that’s too expensive: xLSTM
There’s a known fix for spectral bias: the xLSTM, an upgraded recurrent network. It adds two tricks to the classic LSTM: exponential gating (sharper than the old squashed sigmoid gates) and a matrix memory (the cell’s memory becomes a whole d×d matrix instead of a single number). Together they let it track sharp, multi-scale, fast dynamics. It’s the fast-shutter camera. But the matrix memory costs O(N²): you store and update a d×d matrix at every time step. For offline science that’s fine. For SPICE, which must evaluate a model in nanoseconds inside an iterative solver, millions of times, it’s a non-starter. So the xLSTM solves the math problem (high frequency) and creates a hardware problem (too heavy). That’s the new gap: the capacity to model fast transients exists, but it can’t be deployed, and nobody had extended Ψ-NN/Ψ-HDL’s structure discovery to recurrent, time-based networks.
Ψ-xLSTM’s plan
Same Ψ playbook, recurrent backbone: train a heavy xLSTM-PINN teacher, distill it into a lean student, discover the temporal structure, and synthesize Verilog-A.
Two things change versus Ψ-HDL: how we distill (Step 2 must now preserve time), and what we discover (Step 3 finds time constants and low-rank memory, not mirrors and swaps).
RRAD: copy the dance, not just the poses
Standard distillation copies the teacher’s outputs. For a feedforward net, that’s enough. For a recurrent net, it’s a trap, and fixing it is the central innovation here. Picture copying a dancer by matching only their poses at each instant. You might hit every pose and still move between them at the wrong speed: jerky, lagging, wrong. To copy a dance, you must match the velocity of each motion, not just the snapshots. For a device, the „velocity“ of the output is the switching slope, ∂ŷ/∂t, and that slope is the high-frequency content. So RRAD (Recurrent Relation-Aware Distillation) trains the student to match two things: hidden-state trajectories and temporal derivatives ∂ŷ/∂t.
L_RRAD = α · (hidden-state mismatch)² + β · (temporal-slope mismatch)²
with α = β = 0.5: equal weight to where the dance is and how fast it moves.
The ablation proves it. Copying only hidden states (β = 0) gets you partway. Adding the slope term cuts the student’s error by about 60%, and beats training the small net from scratch by 2.4×. For recurrent architectures, the rate of change isn’t a detail, it’s the whole signal you’re trying to preserve.
What to discover: stopwatches and a small orchestra
Ψ-NN hunted for mirrors, flips, and swaps. A recurrent net hides different structure, and Ψ-xLSTM finds two kinds.
a) Time constants: the hidden stopwatches
Every decaying physical process has a characteristic fade time τ: how long an echo lasts, how long a hot pan takes to cool. In a device, state decays like e^(−t/τ). The xLSTM’s exponential forget gates secretly encode these decay rates, but an oversized net learns dozens of slightly different ones.
Cluster them (exactly as Ψ-NN clustered weights) and they collapse into a few real time constants. For the memristor, three:
The beautiful confirmation: τ1 = 0.34 ms lands within the expected order of magnitude (the VTEAM model’s rate constant predicts a characteristic time around 0.1 ms). The network didn’t fit arbitrary curves, it rediscovered the device’s actual physical clocks. Replacing the dense gate matrix with K shared „integrators“ (one per τ) is most of the compression.
b) Low-rank memory: the small orchestra
The matrix memory is a big d×d (here 64×64), and expensive. The hypothesis: the real physics lives in a low-rank subspace. Picture a 64-piece orchestra that, listened to closely, is really playing only four independent melodies, everyone else is just doubling someone.
SVD (singular value decomposition) finds those few dominant melodies. For the memristor, rank 4 captures 92% of the variance. Project the memory onto those four modes and the cost drops from O(d²) to O(r²). The four modes plausibly map to four real processes: electronic conduction, ionic drift, thermal diffusion, and trap-state occupation.
Mapping to hardware: the ddt() operator
A time constant maps perfectly onto a circuit-simulator primitive: the time-derivative operator ddt(). Each discovered τ becomes one first-order differential equation, and the simulator’s own adaptive time-stepping resolves the fast transients for free.
// three discovered relaxation modes -> three differential equations
parameter real tau_fast 0.34e-3; // ionic drift
parameter real tau_medium 1.2e-3; // thermal relaxation
parameter real tau_slow 8.7e-3; // trap states
real s_fast, s_medium, s_slow;
analog begin
ddt(s_fast) < (V(p,n) s_fast) tau_fast;
ddt(s_medium) < (V(p,n) s_medium) tau_medium;
ddt(s_slow) < (V(p,n) s_slow) tau_slow;
I(p,n) < w_fasts_fast w_mediums_medium w_slows_slow;
end
The whole heavy xLSTM, 46,409 parameters of recurrent state updates, has become a few lines of differential equations.
What it buys, and the honest caveats
The wins. The clustering student keeps the teacher’s accuracy (test error statistically tied with both the teacher and the baseline PINN) while using 63.5% fewer parameters and running 7.6× faster and, unlike the baseline, it stays flat across the entire 50–150 kHz band instead of collapsing above 80 kHz. The end-to-end PyTorch → Verilog-A → SPICE pipeline reproduces the model to 0.40 mA error (under 0.05% of the signal range). A more aggressive low-rank student reaches 84.1% compression at a modest 2.23× accuracy cost.
The caveats the paper is careful about (and so should we be). It’s the recurrence, not the magic gate: A head-to-head of sigmoid vs exponential gating found both work about equally well. The high-frequency win comes from the recurrent state-space formulation as a whole, not from exponential gating alone. Best model depends on the device: across memristor / MOSFET / BJT, no single architecture dominates; sometimes a simpler Fourier-feature PINN wins. The paper frames the extra devices as regime stress-tests, not a victory lap. Real data is messier: on three public experimental datasets it transferred well to two and struggled on the third (different normalization and measurement protocol). Heterogeneous lab data remains an open challenge. The headline speedup is a projection: the 7.6× is a measured Python lower bound; the >100× in compiled SPICE is an estimate (from eliminating matrix overhead), not yet confirmed on fabricated hardware. Scope limits: It’s proven up to ~50K parameters; the mirror-type structures lean on tanh’s odd symmetry; GHz-range switching and transformer-scale models are future work. And, exactly like Ψ-NN, it discovers the structure of a known kind of physics; it does not invent the governing equation from nothing.
The whole journey on one page
And the same story as a recipe:
# Ψ-NN — discover and hard-wire the physics structure teacher = train_PINN(equation, data) student = distill(teacher, tidy=True) # weights clump R, μ = extract_structure(student) # blueprint: +1 / −1 / 0 psi_nn = reconstruct(R, μ) # the structured brain # Ψ-HDL — translate that brain into circuit code verilog = to_verilog_a(psi_nn) # ŷ = Σ Pk·Φk(x) -> named params + equations # runs in SPICE / Cadence Spectre # Ψ-xLSTM — swap the MLP for a recurrent xLSTM to catch fast transients teacher = train_xLSTM_PINN(high_freq_data) # fast-shutter, but heavy O(N²) student = RRAD_distill(teacher) # copy outputs + slopes ∂ŷ/∂t + trajectories tau = cluster(forget_gates) # a few physical time constants U, V = svd(matrix_memory, rank=r) # low-rank: the few real "melodies" verilog = to_verilog_a(tau, U, V) # tau -> ddt() operators
The one-sentence story: Ψ-NN finds the physics, Ψ-HDL ships it to a circuit simulator, and Ψ-xLSTM makes it fast enough to catch the moments that matter.
Where this is going
The honest edges of each paper are exactly where the research is headed. Beyond memristors: applying the same discover-then-deploy idea to transistor modeling (augmenting the venerable BSIM models, or radiation-hardened devices), where conventional models fall short. RF and microwave: modeling complex impedance Z(f) = R(f) + jX(f) and enforcing causality through the Kramers–Kronig relations instead of time-domain I-V curves. Multi-physics: coupling electrical behavior with heat (electrothermal effects in power devices) or motion (MEMS resonators) by adding inputs and coupled physics-loss terms. Aging and reliability: treating slow degradation (interface-trap buildup) as just another internal state variable, the way the memristor treats its conduction state. Transfer learning: reusing a discovered structure (the relation matrix R) across related device families, relearning only the cluster centers: the „cookie cutter“ idea from the Ψ-NN tutorial, applied to real silicon. Symbolic regression post-processing: turning the discovered clusters into true closed-form equations, fully closing the loop between learned models and human-written compact models. Higher frequencies: pushing from kHz transients toward GHz and sub-nanosecond switching, where new physics (and finer measurement data) come into play.
Your next step: pick one case study, the memristor is the richest, and trace a single number end to end. Follow the 0.34 ms time constant from a cluster of forget-gate weights, to a physical ionic-drift timescale, to one ddt() line in the generated Verilog-A. Once you can follow one parameter across all three transformations, discovered, interpreted, deployed,
Neueste Kommentare