NormReact: LLMs Expect More Social Punishment Than People

Ask a model whether jumping a queue is wrong and it will get it right without breaking a sweat. Ask what the person standing behind will do, whether they will say something or keep quiet, and whether that reaction changes when the queue jumper is a stranger rather than a friend. That is where it breaks down.

A paper published on 10 September on arXiv, Beyond Right and Wrong: Evaluating Second-order Social Reasoning in Large Language Models, measures exactly that second layer. The authors start from a criticism of alignment work so far: almost everything has gone into first-order social norms, teaching the model what is acceptable and what is not, along the lines of do not steal. Real social intelligence does not end there.

From norms to metanorms

A metanorm is the expectation about what happens when someone breaks a norm: who enforces it and how hard. The range runs from an awkward silence to public shame or imprisonment, and the paper treats it as the missing axis. Knowing that queue jumping is wrong is trivial. Knowing that in a supermarket queue the typical sanction is a comment and not a police report is what makes a system understand people.

What NormReact is

The team releases NormReact, a multi-perspective dataset of 450 norm violation scenarios, hand annotated. Each scenario carries emotion and behavioural response labels, and the annotations cross two variables that are rarely controlled for: the gender of the person committing the violation and the social closeness of the person observing it.

On that basis they propose two new classification tasks. The first is predicting self-regulation in violators, that is, what the person who broke the norm feels and does. The second is predicting other-regulation in observers, what the person who witnessed it feels and does. These are two distinct axes, emotional appraisal and behavioural response, and separating them makes it possible to see where the model fails: whether it misreads the emotion, the action, or both.

A harsher social world than the real one

The main result is uncomfortable. Across the six models evaluated, the systems overpredict negative sanctions precisely where human annotators expected nothing to happen. Put another way, the picture of the social world an LLM holds is more severe than the real one: where a person lets it go, the model expects reproach, punishment or consequence.

The second finding refines the first. Agreement with human judgment deteriorates as social distance between observer and violator increases. With close ties the models do reasonably well. With strangers they lose the thread, and that is exactly the most common scenario in any public interaction. The abstract does not spell out which six models are in the comparison, so it is worth reading the full table before extrapolating to a specific provider.

Who this is useful for

If you build moderation, this explains part of the overreach: a classifier that assumes every slip deserves a sanction will flag content no community would flag. If you build agents that draft communication on somebody's behalf, a model that overestimates punishment writes disproportionate apologies and emails more tense than the situation calls for. And if you work on social simulation or game characters, degradation with social distance is exactly the defect that makes an NPC treat a neighbour and a stranger the same way.

For evaluation work, the concrete contribution is a dataset with human annotation on an axis almost nobody measures. That it is 450 scenarios and not 45,000 limits statistical power, but the multi-perspective design compensates: each scenario contributes several readings instead of a single label.

We think this is the kind of benchmark that was needed: small, hand annotated and aimed at a specific gap rather than at a global number. If it ends up integrated into the usual evaluation suites, we suspect the punitive bias will be easier to fix with data than with instructions in the system prompt.