Standard reinforcement learning for portfolio optimization maximizes discounted expected returns. The agent values a dollar tomorrow less than a dollar today, with the discount rate fixed. This is computationally tractable but psychologically unrealistic — human investors care about risk in ways that simple discounting can't capture.
Note (2026-11-07): the abstract reports that on 10 chronological train/test splits of South Korean ETF data the recursive-utility agent improves on the discounted baseline in Sharpe ratio, maximum drawdown and cumulative return, using a Monte Carlo certainty equivalent and a new approximate advantage estimate. Because the learning target and advantage estimate also change, the abstract does not support the essay's claims that the architecture is otherwise the same, that the objective matters more than the algorithm, or that the gain comes from preferences rather than learning; the remarks on psychological realism are mine.
The paper on portfolio optimization under recursive utility (arXiv: 2603.22880) replaces the standard objective with recursive utility from asset-pricing theory. In this framework, the agent's valuation of future outcomes depends on a certainty equivalent that has no closed form under observed returns. The authors approximate it by K-sample Monte Carlo and train actor-critic algorithms (PPO, A2C) on the resulting value targets.
The key technical contribution is an approximate advantage estimate (AAE) that generalizes the Bellman residual to multi-step with state-dependent weights. This formulation applies only to critic-based algorithms — policy-gradient-only methods can't use it.
On ten chronological train/test splits of South Korean ETF data, the recursive-utility agent improves on the discounted baseline in Sharpe ratio, maximum drawdown, and cumulative return.
The through-claim: the objective function matters more than the algorithm. The same RL architecture with a risk-sensitive objective outperforms the same architecture with a naive one. The improvement comes not from better learning but from better preferences — the agent optimizes for something closer to what a risk-aware investor actually wants. The recursive utility doesn't add information; it changes what counts as success.
2603.22880. Quantitative finance / reinforcement learning / recursive utility / portfolio optimization / risk sensitivity.