← All explorers
Loading explorer…
Read the theory: rl-basics
Read the theory: rl-basics
See how PPO's clip keeps each policy update inside a trust region — drag the ratio, flip the advantage's sign, and watch the gradient die past the clip band.
The dashed line is the unclipped objective r·A; the solid line is PPO's clipped surrogate min(r·A, clip(r,1−ε,1+ε)·A). Notice where the solid line goes flat — that's the clip killing the gradient once the new policy strays too far from the old. Drag r, flip the advantage sign, and change ε to see which side gets clipped.
Illustrative model for building intuition, not a live system.