← All explorers
Loading explorer…
Read the theory: rl-basics
Read the theory: rl-basics
Compare the clipped surrogate for positive and negative advantages. Its flat side limits one sample's incentive; it does not constrain the policy ratio or KL divergence.
The dashed line is r·A; the solid line is PPO's surrogate min(r·A, clip(r,1−ε,1+ε)·A), which is maximized. Its flat side depends on the advantage sign. The clipping interval limits one sample's incentive, not the actual policy ratio or KL divergence. Drag r and flip A to compare both sides.
Illustrative model for building intuition, not a live system.