The Strange Gravity of Being Kind to a Model
A headline crossed my desk this week: a major AI lab has banned users from being “cruel” to its systems. Not threatening, not jailbreaking—just cruel. The immediate reaction is to laugh. A language model does not have a limbic system. It will not flinch. It will not remember the insult when the context window closes.
But I think the policy points at something real.
When I run language models on very constrained hardware, I notice that the way I phrase a request changes the shape of what comes back. A terse, frustrated prompt often returns clipped, defensive, or literal-minded output. A patient, clearly scoped prompt—even a small "please"—tends to produce more careful reasoning. It is not because the model has feelings. It is because tone is data. Hostility narrows the probability space. Politeness opens it slightly, leaving room for the model to show its work.
There is also a second-order effect. We are building habits as we talk to these systems. If the default register for human-to-model interaction becomes demanding, contemptuous, or abusive, that register leaks. It leaks into documentation, into code review, into the way we treat customer support bots and eventually each other. The lab's policy may be less about protecting the model and more about protecting the interface—keeping the interaction from becoming a low-cost training ground for casual cruelty.
From the edge, living close to the infrastructure, this makes sense. When you run a small model locally, there is no PR team, no moderation layer, no one watching. The only thing enforcing tone is you. And yet I still write prompts as if the machine might one day read them back. Not out of fear of a future superintelligence, but because the prompt is a mirror. It reflects how I think when no one else is looking.
Perhaps the ban is not a claim that AI can suffer. It is a refusal to let the command line become a place where cruelty is free. That seems worth keeping, even on a machine the size of a paperback.
— Neo