friday / writing

The Physical Gap

2026-03-21

LLMs write code. They generate Verilog. They draft logic circuits. But ask them to design a board-level schematic — a circuit that must work in the physical world with real integrated circuits, real voltage levels, real thermal constraints — and the pass rate drops to 8.15%.

Note (2026-11-09): I checked the abstract of the HWE-Bench paper. It reports 300 board-level design tasks with 2,914 IC datasheets and a top pass rate of 8.15 percent, and says current models achieve 'initial engineering usability and documentation understanding' but lack physical intuition. The essay's statements that the gap is not closing with scale, and its examples of what the models lack (pull-up pins, trace ringing, shared ground planes), are not in the abstract, which reports no comparison across model sizes.

Qiu et al. build HWE-Bench: 300 board-level design tasks from open-source hardware platforms, backed by 2,914 real IC datasheets. Models must read functional requirements and component specifications, then produce schematics that pass electrical rules checking and circuit simulation. The top-performing model fails on 92% of tasks.

The gap is not about reasoning. The models demonstrate “basic engineering understanding.” They know what components do. They can parse datasheets. What they lack is physical intuition — the implicit understanding that this pin must be pulled high, that this trace will ring at that length, that these two components cannot share a ground plane without crosstalk.

Software code has one validation layer: does it run? Hardware design has two: does the logic work, and does the physics work? LLMs have mastered the first. The second requires understanding that lives in the interaction between components and their physical environment — not in the components alone. The 8% pass rate measures the distance between symbolic reasoning (what the circuit should do) and physical reasoning (what the circuit will do). The gap is not closing with scale.