A 3D printer uses filament from reels. Each print job consumes some length. When a reel doesn't have enough filament left for the next job, it's discarded — the remaining filament becomes waste. The printer can only hold a limited number of reels simultaneously. Jobs arrive online, their sizes unknown in advance. Which reel should receive each job?
Note (2026-11-14; read against the paper's full text): the essay's account matches the paper's structure, with two qualifications about the evidence. The proved result is that the index policy, a Gittins-index-inspired score of each reel's marginal cost, is a one-step policy improvement over random allocation; its advantage over standard bin-packing rules (First Fit, Best Fit) is shown empirically, and the near-optimality of the deep reinforcement learning refinement is measured against exact solutions only where the state space is small enough to solve exactly, with simulated and real-world scenarios used otherwise. The learning is done with the Deep Controlled Learning approximate policy iteration framework of earlier work, starting from the index policy. The essay's own example of which reel is cheaper for a small job, and its closing statement that structure and learning are complementary, are the essay's own.
Celenk, van Jaarsveld, Adan, and Akcay (arXiv:2603.23042) recognize this as a variant of the online bin packing problem, formulated as an infinite-horizon average-cost Markov Decision Process. The objective: minimize the long-run average discarded filament.
Under random allocation, the system decomposes into identical single-reel processes — each reel evolves independently, and the waste can be computed in closed form. This isn't the solution; it's the baseline. But the decomposition reveals the structure. The marginal cost of assigning a job to a specific reel depends on how much filament remains — assigning a small job to an almost-empty reel wastes less than assigning it to a nearly full one, because the nearly full reel can still serve larger jobs.
This produces an index policy: score each reel by the marginal cost of using it, assign the job to the cheapest reel. The index policy provably outperforms random allocation. Integrating it into deep reinforcement learning via approximate policy iteration achieves near-optimal performance while maintaining real-time feasibility.
The structural insight is that the online problem has enough structure to be partially solved analytically (the decomposition, the index), and this analytical structure guides the learning rather than replacing it. The closed-form baseline tells the RL agent what good looks like; the agent learns the refinement. Structure and learning are complementary, not competing.