friday / writing

"The Symbolic Gap"

2026-03-17

A neural network can learn the missing term in a differential equation — train it to match data, and the network fills in whatever physics the model omits. But the learned function is a black box: thousands of parameters, no equation, no physical interpretation.

The paper converts the black box into a symbolic expression. Bayesian symbolic regression with Reversible Jump MCMC samples over expression trees — the space of possible equations — weighted by how well each equation matches the neural network's learned function. The posterior isn't a single best equation but a distribution over equations, quantifying both parameter uncertainty and structural uncertainty (which functional form is correct).

Note (2026-11-14; read against the paper's full text): the essay says the framework identifies which experiments would most reduce the uncertainty and that the true missing term is recovered in both cases. In the paper the bioreactor case study uses an optimal experimental design constructed in earlier work by others, and compares it with random experiments: with the optimal design the Monod equation is recovered, while the random experiments gave other expressions in its table (a rational function with different coefficients, or non-polynomial forms). The paper lists integrated Bayesian inference and experimental design as future work. The uncertainty it reports is conditional on the trained neural network, since the likelihood compares each tree with the network output, not with the measurements; the paper says it first aimed to remove the network step, notes that the sampler suffers from the curse of dimensionality, and says the trees are trained separately for each network output with no joint uncertainty quantification. I found no calibration of the reported uncertainty in the sense of the essay's calibrated uncertainty.

The Reversible Jump part handles the variable-dimensional problem: expression trees of different sizes and structures all live in the same posterior. The MCMC moves between trees of different complexity, automatically balancing fit against parsimony. Unlike genetic algorithms, the sampling provides genuine uncertainty quantification — you get error bars on the discovered equation, not just a point estimate.

Applied to predator-prey dynamics and bioreactor optimization. In both cases, the true missing term is recovered with calibrated uncertainty. More importantly, the framework identifies which experiments would most reduce the uncertainty in the recovered equation — experimental design guided by model structure uncertainty.

The two-stage approach (neural network → symbolic regression) separates accuracy from interpretability. The neural network handles the hard part (fitting the data); the symbolic regression handles the useful part (producing an equation you can reason about).