A From-Scratch Tutorial on Ψ-NN (Psi-NN)

The promise of this tutorial: by the end you will understand, step by step, how Physics Structure-Informed Neural Network (Ψ-NN) takes an ordinary neural network and rebuilds it so that it literally cannot break a law of physics, and how it figures out how to do that all by itself. We will keep every idea concrete, introduce each equation gently after the picture that explains it, and never use a word we haven’t first explained. Each of the three core stages also comes with a simple diagram and a runnable code cell, so you can do it, not just read it. The cells run top to bottom and chain into one working mini-pipeline. They need torch and scipy.

About this tutorial, credit where it’s due. Ψ-NN was created by Liu et al. (2025): Ziti Liu, Yang Liu, Xunshi Yan, Wen Liu, Han Nie, Shuaiqi Guo, and Chen-an Zhang, in „Automatic network structure discovery of physics informed neural networks via knowledge distillation,“ Nature Communications 16:9558. I wrote about their researcher paper shortly here a few months ago. The method, the three-stage framework, the experiments, and the figures are entirely theirs. This page is an independent educational explainer; the analogies, the simplified code, and the diagrams are teaching aids meant to make their work approachable, not new research. The paper (DOI: 10.1038/s41467-025-64624-3) and the authors‘ own code (github.com/ZitiLiu/Psi-NN) are the definitive sources.

The dream: a network that can’t be wrong

Imagine two students studying for a physics exam. The first student memorizes thousands of old answers. On questions like the ones they’ve seen, they do fine. On a slightly new question, they guess, and sometimes their guess is impossible, like a ball that gains energy from nowhere. The second student understands the rules. They might be slower, but they will never give an answer that breaks physics, because the rules are baked into how they think. A neural network that just fits data is the first student. What we want is the second student: a network whose answers obey the laws of physics by nature. That is the whole goal, and Ψ-NN is one clever way to get there.

A quick family tree: how we got here

Before we climb, here’s the whole ladder on one page: four ideas, each one fixing the exact thing that stopped the one before it. Ψ-NN sits at the top, and it only makes sense once you’ve felt why the lower rungs weren’t enough. (We’ll meet a real physics „rule“ properly in the next section; for now, think of laws like „heat always flows from hot to cold“). Start at the bottom rung: what a neural network even is.

Rung 1: the plain neural network (an „MLP“). MLP stands for multilayer perceptron, which is a fancy name for the simplest kind of neural network: layers of „knobs“ (called weights) with a little bending in between, that you tune until the network turns inputs into the outputs you want. Think of a lump of clay you press into shape by showing it example after example: „this input should give that output“, until it matches. Give it enough examples and it can copy almost any pattern. What it solves: learning a pattern straight from data, with no rules needed. Where it breaks: it only knows the examples it was shown. Ask it something new and it may give a physically impossible answer: it never learned the rules, only the answers (the memorizer student from Section 1). For physics, where you often have few measurements but strong known laws, pure data-fitting is both wasteful and unsafe.

Rung 2: the physics-informed network (a „PINN“). The fix: stop relying only on examples: tell the network the rule and dock it points whenever it breaks the rule (we build this in Section 4). Now it can learn from very little data, because the physics fills in the rest. What it solves: obeying a known physical law even with little data. Where it breaks: it obeys the rule only on average, never exactly, and it can’t guarantee clean properties like symmetry (Section 5).

Rung 3: the hand-built structured network. The next fix: instead of nagging the network to respect a property, wire that property into its very shape so it can’t break it. What it solves: guaranteed physics, built in rather than encouraged. Where it breaks: a human expert has to design the wiring by hand, for each problem, knowing the property in advance. Slow, and it doesn’t scale.

Rung 4: Ψ-NN. The final fix, and the subject of this tutorial: let the network discover the right wiring by itself, then build it in automatically. You get the guarantee of a hand-built network without needing the expert.

That’s the climb. Each rung removes exactly one limitation of the rung below it. The rest of the tutorial walks it slowly, from the bottom.

A „law of physics“ is just a rule about neighbors

Physical laws are often written as partial differential equations (PDEs), which sounds scary but really just means a rule connecting each point to its neighbors. Our running example is Laplace’s equation. In plain words it says: Every point sits at exactly the average of the points around it. Nothing is a bump or a dip: everything is perfectly smoothed out and in balance. (Think of a metal plate that has settled to a steady temperature: no spot is hotter or colder than the average of its surroundings). Written as a rule, with u standing for the value at a point (x, y):

∇²u  =  u_xx + u_yy  =  0

Here u_xx means „how curved u is in the x-direction“ and u_yy the same in y. Adding them and setting it to zero is the math way of saying no net bumps, perfectly balanced. That’s it. That’s the law. For this example we happen to know the exact answer:

u(x, y)  =  x³ − 3·x·y²

And this answer has a hidden symmetry we’ll care about a lot. Flip y to −y and nothing changes:

u(x, y)  =  u(x, −y)

Picture a butterfly: the top half is a perfect mirror of the bottom half. Our solution is a butterfly. Hold that image, it’s the hero of the whole story.

PINN: a network that gets points docked for breaking the rule

Before Ψ-NN, the popular tool was the PINN (Physics-Informed Neural Network). The idea is simple and clever. A neural network is just a flexible function with knobs (called weights) you can tune. A PINN tunes those knobs by being penalized in two ways:

1. Data penalty: at the edges of the plate, where we know the true values, it’s docked points for being wrong.

2. Physics penalty: at lots of points inside, it’s docked points for breaking the rule (for having u_xx + u_yy ≠ 0).

We add those up into a single score to minimize, called the loss:

Loss  =  (physics penalty)  +  (data penalty)
​
      =  average of (u_xx + u_yy)²   +   average of (u_predicted − u_known)²

The network is trained to make this loss as small as possible. (How does it even know u_xx? The network is made of smooth math, so the computer can take its derivatives automatically. This is called autodiff. You don’t need to know more than that here).

📦 The two jobs a PINN can do. A PINN is good at two different kinds of task, and it’s worth knowing which is which.

• The forward problem: given the full rule, find the answer. We know the equation and we want the solution u everywhere on the plate. Our butterfly is a forward problem.

• The inverse problem: given some measurements, discover a missing piece of the rule. Suppose you don’t know one number in the equation, say, how thick the fluid is, but you have a few measurements of how it actually behaved. A PINN can run backwards and figure out that hidden number while it solves the equation. The Burgers shock-wave example later is exactly this: the „fluid thickness“ is unknown, and the network discovers it. Same machinery, run in reverse.

This works remarkably well. So what’s the problem?

The catch: „usually obeys“ is not „always obeys“

Look closely at that physics penalty. It punishes the average amount of rule-breaking. So a PINN learns to break the rule a little, everywhere rather than not at all. More importantly, nothing in a PINN forces our butterfly symmetry. The PINN draws the butterfly freehand. The two wings come out almost the same, but never exactly, and the little mismatch never fully goes away no matter how long you train.

This is exactly what you see if you train a plain PINN and watch the quantity mean |u(x, y) − u(x, −y)| (how much the two wings disagree). It drops as training improves… and then flattens at a small, stubborn, nonzero floor. The symmetry is an accident the network stumbles toward, not a guarantee it’s built on. That gap between usually and always is the entire reason Ψ-NN exists.

Two ways to enforce a rule: nag it, or build it in

There are two fundamentally different ways to make something obey a rule. The soft way (nagging): keep telling it „try not to break the rule“ and dock points when it does. It can still break the rule. This is what a PINN does. The hard way (building it in): construct the thing so the rule is physically impossible to break. A ruler can only draw straight lines. Fold a piece of paper in half and cut once, and the two halves are guaranteed identical: that’s a symmetric butterfly, perfectly, every time.

The hard way is clearly better when you want guarantees. People had already built networks with physics „folded in“, but there was a catch: an expert had to design the fold by hand, for each specific problem, knowing the symmetry in advance. That’s slow, and it doesn’t scale. Ψ-NN’s big idea: let the network discover the fold by itself. Find the right structure automatically, then wire it in. The full name even says so: Ψ-NN stands for Physics structure-informed Neural Network. The physics lives in the structure, not just the scolding.

The obvious fix that backfires

If you wanted a network to discover its own simple structure, the textbook tool is regularization: a gentle pressure that tidies the knobs, pushing unimportant ones toward zero and encouraging repeated values. It’s how we normally simplify networks. So: just add regularization to a PINN, right? It backfires. When you do this, the PINN’s accuracy gets worse; its loss actually goes up. Why? Picture two coaches shouting at one player at once. One yells „obey the physics!“ The other yells „tidy your knobs!“ These pull in different directions, the player gets confused, and performance drops. The physics pressure and the tidying pressure fight each other inside the same network. This is the wall the authors hit. The fix is the cleverest part of the whole paper.

The key trick: a teacher and a student (distillation)

If two jobs fight when done by one network: give them to two networks. This is called knowledge distillation, and it’s the heart of Ψ-NN. Here’s the everyday version: A master chef perfects a recipe (this is the hard part: the physics). An apprentice does two easier things: copy the chef’s finished dishes, and keep the kitchen tidy. Because the apprentice isn’t simultaneously struggling to invent the recipe, they can keep a perfectly organized kitchen and still reproduce great food. And once the kitchen is tidy, you can finally see the patterns in how it’s organized.

In Ψ-NN, the teacher is a normal PINN. Its only job is the physics. It gets no tidying pressure. (It doesn’t have to be a PINN, by the way; the teacher can be any trustworthy source of physics-correct answers, even a known formula for a simpler version of the problem. A PINN is just the usual choice). The student has two jobs: copy the teacher, and stay tidy (regularization). It never wrestles with the physics directly, it just imitates a network that already got the physics right. The student’s score has matching parts:

copy penalty        =  average of (u_student − u_teacher)²
tidiness penalty     =  sum of (each weight)²          ← the regularization
Loss_student         =  copy penalty  +  ω · tidiness penalty

(ω just sets how much we care about tidiness versus copying).

📦 How the copying really works. Distillation was invented for a different kind of task: classifiers, like a network that labels a photo „cat“ or „dog.“ There, the teacher doesn’t just hand over the final label; it hands over its confidence in every option („90% cat, 9% dog, 1% fox“). Those in-between numbers carry the teacher’s hard-won intuition: researchers call it dark knowledge, and a temperature dial softens them so the subtle hints stand out. The student is then trained to match those soft confidences, using a mismatch measure called KL-divergence. Our setting is simpler. A PINN doesn’t output a list of confidences, it outputs one number, the value at a point. So there’s no soft list to soften and no KL-divergence needed: the student just matches the teacher’s single number with a plain squared difference, exactly as in the cell below. Same idea, learn from the teacher’s full output, not just a bare answer, stripped down to its simplest form.

By splitting the jobs, the fight disappears. The physics flows from teacher to student, and the student is free to get neat, which is what makes the next step possible. A neat detail: the apprentice learns cleanest when it copies only the chef and ignores the raw ingredients. The authors found that mixing the original data back into the student’s lessons actually muddies the patterns. Pure imitation gives the sharpest structure.

Step 1: Distill

So the first move is everything in Section 8 put into action:

1. Train the teacher PINN until it solves the problem well.

2. Train the student to imitate the teacher, with tidiness pressure switched on.

Result: a student network that is just as physically correct as the teacher, but whose knobs are now neat, primed to reveal their hidden structure. Onward.

▶ Run it (this cell defines the shared helpers the later cells reuse; ~a minute on CPU):

# === STAGE 1: DISTILL ============================================
# Train a teacher PINN (it carries the physics), then a student that
# copies the teacher while staying tidy (L2 regularization).
import torch, torch.nn as nn
torch.manual_seed(1234)
​
# the Laplace "butterfly" problem (same as Module 1)
def exact(x, y): return x**3 - 3*x*y**2
​
def mlp(widths):                              # tiny helper to build an MLP
    layers = []
    for i in range(len(widths) - 1):
        layers += [nn.Linear(widths[i], widths[i+1])]
        if i < len(widths) - 2: layers += [nn.Tanh()]
    return nn.Sequential(*layers)
​
def laplacian(net, xy):                        # the physics: u_xx + u_yy via autodiff
    xy = xy.clone().requires_grad_(True)
    u = net(xy)
    g = torch.autograd.grad(u, xy, torch.ones_like(u), create_graph=True)[0]
    ux, uy = g[:, 0:1], g[:, 1:2]
    uxx = torch.autograd.grad(ux, xy, torch.ones_like(ux), create_graph=True)[0][:, 0:1]
    uyy = torch.autograd.grad(uy, xy, torch.ones_like(uy), create_graph=True)[0][:, 1:2]
    return uxx + uyy
​
# sample points: interior (for physics) and boundary (for data)
xy_f = torch.rand(2000, 2) * 2 - 1
s = torch.linspace(-1, 1, 100).unsqueeze(1); one = torch.ones_like(s)
xy_b = torch.cat([torch.cat([-one, s], 1), torch.cat([one, s], 1),
                  torch.cat([s, -one], 1), torch.cat([s, one], 1)], 0)
u_b = exact(xy_b[:, 0:1], xy_b[:, 1:2])
​
# --- teacher: a normal PINN (physics only, NO regularization) ---
teacher = mlp([2, 16, 16, 16, 1]); opt = torch.optim.Adam(teacher.parameters(), 1e-3)
for _ in range(2000):
    opt.zero_grad()
    loss = (laplacian(teacher, xy_f)**2).mean() + ((teacher(xy_b) - u_b)**2).mean()
    loss.backward(); opt.step()
​
# --- student: copy the teacher (omega=0 raw data) + stay tidy (L2) ---
with torch.no_grad(): u_teacher = teacher(xy_f)            # the teacher's "dishes"
student = mlp([2, 8, 8, 8, 1]); opt = torch.optim.Adam(student.parameters(), 1e-3)
omega = 1e-4                                                # tidiness weight
for _ in range(2000):
    opt.zero_grad()
    copy = ((student(xy_f) - u_teacher)**2).mean()          # copy penalty
    tidy = sum((p**2).sum() for p in student.parameters())  # L2 tidiness penalty
    (copy + omega * tidy).backward(); opt.step()
​
print("Teacher trained; student distilled.")
print("The student is now physically correct AND tidy.")

Step 2: Extract (let the pattern appear, then read it)

Here’s the payoff of all that tidying. With the tidiness pressure on, the student’s many weights don’t stay as a messy cloud of different numbers. They clump together into a few values. Picture a junk drawer full of screws that all look slightly different. You sort them and discover there are really only two or three actual sizes. The dozens of „different“ screws were an illusion.

We do exactly this to the weights. We group the similar ones (using a standard sorting method called hierarchical clustering) and relabel every weight by its group’s center value, while keeping its plus-or-minus sign:

each weight   →   sign(weight) × (center value of its group)

Notice we cluster the sizes of the weights (their absolute values), not the signed numbers. That’s on purpose: a knob and its mirror have the same size but opposite sign, so clustering by size drops them into the same group, and the sign we kept then tells us whether two grouped knobs are copies (same sign) or mirrors (opposite sign).

Why are we allowed to expect them to clump? The paper proves two small facts that make this trustworthy: a) If two knobs play the exact same role in the network, the tidying pressure forces them to become equal (intuitively: any difference between equals is wasted „untidiness“ that gets squeezed out). b) If two knobs are supposed to be mirror images, their gap shrinks a little every training step, by a fixed fraction each time, so it steadily melts to zero. Now comes the clever reading. Once the weights are just a few repeated values, we can spot the relationships between them: some groups of knobs turn out to be the same → that’s sharing; some are equal but opposite in sign → that’s a mirror / sign-flip; some are the same knobs in swapped positions → that’s a swap. We record all of this in a small relationship blueprint (the authors call it the relation matrix R). Every entry is just one of three numbers:

+1  →  "same as"
−1  →  "opposite of"
 0  →  "unrelated"

(a swap doesn’t need a fourth number: it’s the same values, just listed in a different order).

That little blueprint of +1, −1, and 0 is the physics, written as a wiring diagram. For our butterfly problem, the blueprint will say „this wing is the mirror of that wing“, which, once we wire it in, is precisely the symmetry a plain PINN could only approximate.

These three relationships aren’t just abstract: each shows up in a real equation. The paper tries Ψ-NN on three classic problems, and each one hid a different one of the three relationships, which Ψ-NN discovered on its own: Laplace (our butterfly) hides a mirror: the top half equals the bottom half. Burgers (a shock wave forming in a fluid) hides a sign-flip: flip the left/right direction and the whole wave turns upside down. Poisson (another balance equation) hides a swap: exchange the two directions, x for y, and the picture is unchanged. So „mirror,“ „flip,“ and „swap“ aren’t toy categories: they’re the actual shapes of real physical laws.

▶ Run it (watch the weights consolidate into a handful of values):

# === STAGE 2: EXTRACT ============================================
# With tidiness on, the student's weights clump. Cluster them and see
# how many distinct values are *really* there.
from scipy.cluster.hierarchy import linkage, fcluster
import numpy as np

W = student[2].weight.detach().numpy().reshape(-1, 1)   # a hidden->hidden layer
mags = np.abs(W)                                         # cluster the MAGNITUDES

Z = linkage(mags, method="average", metric="euclidean") # hierarchical clustering
labels = fcluster(Z, t=0.1, criterion="distance")       # paper uses distance 0.1
centers = {c: float(mags[labels == c].mean()) for c in np.unique(labels)}

print(f"{len(W)} raw weights  ->  {len(centers)} distinct clustered values")
print("cluster centers:", {c: round(v, 3) for c, v in centers.items()})
# Takeaway: many 'different' weights are really a few repeated magnitudes
# (plus a +/- sign). Train longer (the paper uses ~80k steps) and they
# collapse into the tight clusters shown in its figures.

What’s real here vs. simplified. The clumping above is genuinely computed: that phenomenon is the whole point of the distillation step. But turning those clusters into the full blueprint R automatically (detecting every share, mirror, and swap across layers) is the paper’s core algorithm, and we don’t reimplement it here. We observe the clumping, then in Stage 3 we wire in the known y-mirror by hand. The complete automatic version lives in the authors‘ code repository.

Step 3: Reconstruct (wire the pattern in for good)

We have the blueprint. Now we build the real network from it.

1. Throw away the junk. Knobs the clustering showed to be unimportant get set to zero.

2. Keep the few distinct values, and make them adjustable again (so the new network can still learn and fine-tune).

3. Freeze the blueprint. The relationships in R(the sharing, the mirrors, the swaps) become permanent wiring. A mirror is no longer „encouraged“; it’s hard-wired. The network now cannot be unsymmetric, the same way folded-and-cut paper can’t have mismatched halves.

4. Retrain this rebuilt network on the original problem.

Two lovely consequences fall out for free: a) the symmetry is now guaranteed, not hoped for. Our butterfly is folded paper, not freehand. b) because mirrored and shared knobs are now the same knob, the network has fewer real knobs to learn. It’s lighter and more correct at the same time. That’s the whole method. Three moves: Distill → Extract → Reconstruct.

▶ Run it (this is the payoff — the symmetry violation goes to ~0, by construction):

# === STAGE 3: RECONSTRUCT ========================================
# Wire the y-mirror into the architecture. ONE shared set of blocks is
# reused across two mirror "groups" with a sign/swap pattern, so the
# output is EVEN in y by construction -- not by a penalty, and not by
# averaging an unconstrained network's output (that would be the
# "PINN-post" trick). The sharing lives *inside* the network.
import torch, torch.nn as nn
torch.manual_seed(1234)

class ButterflyNet(nn.Module):
    """u(x,y) = u(x,-y) is guaranteed by the structure itself."""
    def __init__(self, d=8):
        super().__init__()
        self.Wa  = nn.Parameter(torch.randn(d, 1) * 0.5)   # input weight on x
        self.Wb  = nn.Parameter(torch.randn(d, 1) * 0.5)   # input weight on y
        self.b1  = nn.Parameter(torch.zeros(d, 1))
        self.Maa = nn.Parameter(torch.randn(d, d) * 0.3)   # shared middle blocks
        self.Mab = nn.Parameter(torch.randn(d, d) * 0.3)
        self.b2  = nn.Parameter(torch.zeros(d, 1))
        self.Wo  = nn.Parameter(torch.randn(1, d) * 0.3)   # shared output block

    def forward(self, xy):
        x, y = xy[:, 0:1], xy[:, 1:2]
        # input layer: the SAME weights feed (x, y) and its mirror (x, -y).
        za = x @ self.Wa.T + y @ self.Wb.T + self.b1.T
        zb = x @ self.Wa.T - y @ self.Wb.T + self.b1.T     # y -> -y swaps a<->b
        ha, hb = torch.tanh(za), torch.tanh(zb)
        # middle layer: the two blocks are swapped between the groups.
        ha2 = torch.tanh(ha @ self.Maa.T + hb @ self.Mab.T + self.b2.T)
        hb2 = torch.tanh(ha @ self.Mab.T + hb @ self.Maa.T + self.b2.T)
        # output: same block on both groups, summed -> even in y.
        return (ha2 + hb2) @ self.Wo.T

# train it as a PINN (reuses laplacian / xy_f / xy_b / u_b / exact from Stage 1)
net = ButterflyNet(d=8); opt = torch.optim.Adam(net.parameters(), 1e-3)
for _ in range(2000):
    opt.zero_grad()
    loss = (laplacian(net, xy_f)**2).mean() + ((net(xy_b) - u_b)**2).mean()
    loss.backward(); opt.step()

# check the symmetry that a plain PINN could only approximate
g = torch.linspace(-1, 1, 50); GX, GY = torch.meshgrid(g, g, indexing="ij")
grid   = torch.stack([GX.reshape(-1),  GY.reshape(-1)], 1)
mirror = torch.stack([GX.reshape(-1), -GY.reshape(-1)], 1)
with torch.no_grad():
    viol = (net(grid) - net(mirror)).abs().max().item()
    rmse = ((net(grid) - exact(grid[:, 0:1], grid[:, 1:2]))**2).mean().sqrt().item()

print(f"max symmetry violation: {viol:.2e}   <- ~0: exact, by construction")
print(f"field error (RMSE):     {rmse:.3f}   (bump iterations for a sharper fit)")
# Compare to Module 1, where the plain PINN's symmetry violation stalled
# at a stubborn nonzero floor. Here it's machine-precision zero.

Why you can’t skip a step

It’s tempting to think some of these three moves are optional. They aren’t, and the paper checks this by trying each shortcut on its own: Skip Distilling and the old fight comes right back (physics vs. tidiness in one network), so the weights never clump and there’s no pattern to find. Skip Extracting and you have no blueprint, so „rebuilding“ just hands you back an ordinary tangled network. Skip Reconstructing and the strangest thing happens: you found the pattern but never built from it, so it evaporates as training continues: and you end up worse than a plain network, because you threw away your progress to start the search over. All three together is what makes it work.

Why this is worth it (the payoff)

When the authors tested this on real equations, the rebuilt network won on every front: More accurate: on the Laplace problem its error was about 95% lower than a plain PINN’s. Faster: it reached the same quality in roughly half the training steps, because folding the symmetry in shrinks the space of answers it has to search: it isn’t wasting effort considering unsymmetric shapes that can’t be right. Beats hand-tweaking, too: a common shortcut (call it „PINN-post“) forces the symmetry onto the network’s final answer as an afterthought. Ψ-NN was more accurate than that and faster because Ψ-NN bakes the symmetry through the whole network, not just patched on at the end, so it even cleans up the hard-to-fix errors near the edges. Interpretable: you can literally read the structure it found and see the physics in it. It transfers: this is the magic trick.

Once you’ve discovered a good template, you can reuse it like a cookie cutter: Same problem, different settings: the structure found for one fluid thickness worked, unchanged, at other fluid thicknesses. Different problem entirely: for flow around a cylinder, they snapped together the template from one earlier problem (for two of the outputs) and the template from another (for the third) into a single network, reusing both cutters on new dough.

The honest catch (what it can’t do yet)

A good tutorial admits the limits. You have to already know the rule. Ψ-NN discovers the symmetry structure of a known equation; it does not discover the equation itself out of thin air. Only fairly simple symmetries so far. The mirrors, flips, and swaps it handles are all „discrete“, one clean transformation. Richer symmetries, like „rotate the whole picture by any angle and it’s unchanged,“ are harder, and getting Ψ-NN to find those is still open work. The mirror trick leans on one ingredient. The sign-flip („mirror“) structures work because the network uses an activation function called tanh, which is itself perfectly odd/mirror-like. Pure swaps don’t need this, but the mirrors do. It’s proven for the simplest network type. The math is worked out for plain layered networks (MLPs). Fancier architectures like Transformers don’t yet have a clean „this knob = this physical feature“ correspondence, so extending the method there is open future work. None of this dims the core idea: it just marks the edges of where it’s been shown to work.

The whole journey on one page

The problem: a normal physics network (PINN) only obeys the rules on average, and it can’t enforce a clean symmetry, and the obvious fix (regularization) fights the physics.

The solution, in three moves:

# The Ψ-NN recipe

# 1. DISTILL  — split the fighting jobs across two networks
teacher = train_PINN(equation, boundary)         #   teacher: gets the physics right
student = train_to_copy(teacher, tidy=True)       #   student: copies it + stays tidy
                                                  #   → the student's knobs clump up

# 2. EXTRACT  — read the pattern that appeared
groups = cluster( abs(student.weights) )          #   find the few real values
R      = read_relationships(groups)               #   +1 same / −1 opposite / 0 none
                                                  #   → a wiring blueprint = the physics

# 3. RECONSTRUCT — wire the pattern in permanently
psi_nn = rebuild_from(R, keep_centers=True)       #   mirrors & shares become real wiring
train(psi_nn, equation, boundary)                 #   retrain the lighter, guaranteed net

The result: a network that is faster, smaller, more accurate, readable, and reusable, and that obeys the law of physics not because we keep reminding it to, but because it is built that way. That’s a Ψ-NN.

Where it’s used, and where you can take it

Where it shines. Ψ-NN is built for problems where you know the governing law and that law hides some structure: a symmetry, a balance, a conservation. That describes a huge swath of science and engineering: Fluids: flow around wings, through pipes, and past obstacles (the cylinder example). Heat: how temperature spreads and settles (Laplace and Poisson live here). Electromagnetism and gravity: fields governed by the same balance equations. Materials and structures: stresses, diffusion, reaction–diffusion patterns. Geoscience: subsurface flow and other large, expensive simulations.

The biggest practical win is reuse. Once Ψ-NN discovers a good structure for one problem, you can transfer it to related ones: different settings of the same problem, or even a new problem with the same hidden shape, instead of starting from scratch. In a field where a single simulation can cost days of compute, a reusable, physics-guaranteed template is worth a great deal.

The frontier (open problems). Ψ-NN is young, and the honest edges from Section 13 are exactly where the research is headed: Richer symmetries: moving beyond mirrors, flips, and swaps to continuous ones like rotation-by-any-angle or shifts in time, which connect to deep ideas about conservation laws. Messier reality: handling real, noisy measurements and laws that are only partly known, instead of clean textbook equations. Bigger architectures: carrying the discover-then-wire idea from simple layered networks up to powerful modern ones like Transformers, where „which knob means what“ is far harder to read.

Your next step. The fastest way to make this click is to run the three cells above, then try the experiment the whole method is built for: train the plain PINN (it has a stubborn symmetry error) and the ButterflyNet (its symmetry error is essentially zero), and watch the difference with your own eyes. From there, swap in a new symmetry: try a swap (u(x, y) = u(y, x)) instead of a mirror, and see if you can hand-wire it the way Stage 3 wires the mirror. Once you can build the structure by hand, you understand exactly what Ψ-NN automates: and you’ve climbed the whole ladder, from a lump of clay that just copies answers all the way up to a network that can’t break the laws of physics, and learns how to be that way on its own.

Sources & credit

Everything explained on this page is the work of Liu, Z., Liu, Y., Yan, X., Liu, W., Nie, H., Guo, S., & Zhang, C. (2025). Automatic network structure discovery of physics informed neural networks via knowledge distillation. Nature Communications, 16, 9558. https://doi.org/10.1038/s41467-025-64624-3

Authors‘ code: github.com/ZitiLiu/Psi-NN. The ButterflyNet and the three runnable cells above are simplified teaching reconstructions, not the authors‘ implementation; for faithful code and the exact published results, use the repository and the paper.