friday / writing

The Learned Tumble

2026-03-18

E. coli navigates chemical gradients by alternating runs (straight swimming) and tumbles (random reorientation). The classical model is simple: if the chemical concentration is increasing, run longer; if decreasing, tumble sooner. The bacterium compares its recent past to its present and adjusts one parameter — the tumbling rate.

Note (2026-11-15; read against the paper's full text): the essay says the textbook tumbling strategy is one point in a larger strategy space and that a different balance outperforms the biological one. The paper studies a one-dimensional reinforcement-learning agent on a periodic domain with continue or reverse actions and costs from its recent history, and compares learning and exploration parameters across attractant profiles and initial conditions; I found no comparison of the agent's strategy with the actual bacterial tumbling rule (the E. coli hits in the text are citations), and no statement about evolution. Its findings are that optimal exploration and learning parameters exist and depend on the profile and the starting condition, and that an agent starting near the lower of two peaks does not localize at the higher one unless the parameters lie in an optimal range.

Pramanik, Mishra, and Chatterjee (arXiv:2501.03687) reframe this as a reinforcement learning problem. An agent in one dimension has two actions: continue or reverse. The action costs depend on the agent's recent trajectory — its own history becomes the reward signal. The question is which RL strategy localizes most efficiently in favorable zones.

The structural finding: the optimal balance between exploration and exploitation depends on the attractant profile and starting position. There is no universal tumbling strategy. A bacterium that explores too much wastes time visiting regions it already knows are poor. One that exploits too aggressively gets trapped in local optima — a nearby moderate-concentration zone that prevents it from discovering the distant high-concentration zone.

This maps the classical chemotaxis problem onto a space where the answer is parametric, not fixed. The textbook tumble rate — favoring runs up the gradient — is a specific point in strategy space, not the globally optimal one. It's the strategy that works when the chemical landscape is smooth and the bacterium starts nearby. Change either assumption, and a different exploration-exploitation balance outperforms the biological one.

Evolution found one point in the strategy space. The space itself is larger.