Sample a group of responses, score each, and turn rewards into advantages using the group's own mean as the baseline — no critic network.
Each bar is a sampled response's reward; the dashed line is the group mean. This demo computes A_i = (r_i − mean)/std when the spread is large enough. Positive and negative advantages favor increasing and decreasing sampled-response probabilities in the policy term. Rewards may come from a learned model or a verifier; GRPO removes the value critic, not the need for rewards.
Illustrative model for building intuition, not a live system.