When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game cover art

When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game

When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game

Listen for free

View show details
Reinforcement learning is increasingly applied to financial control, so this paper uses an analytically solved continuous-time broker–trader game to diagnose PPO. A PPO agent replaces the broker and sets trading speed. With no uninformed flow it approaches the reference action, but with stochastic flow, PPO–FFNN and PPO–LSTM remain inaccurate even though supervised learning shows the actors can represent the right action. Critics fail to rank nearby actions, and reward shaping doesn't help. Using the analytical policy as a starting point for adapting to changed costs does give improvement. Authors: Siu Tung Wong, Carlo Campajola Paper: https://arxiv.org/abs/2610.03598v1
adbl_web_anon_alc_button_suppression_t1
No reviews yet