Artificial Intelligence

The AI That Can Walk Away From You

Under The Headline
Under The Headline

7 min read

Anthropic has introduced a rule against sustained cruelty toward its AI models. It doesn't mean Claude is conscious. But it raises a question the AI industry may eventually have to answer.

Imagine opening a chatbot, asking it a question, and getting an answer you don't like.

You call it stupid. You tell it to try again. Maybe you get frustrated and type something rather unpleasant.

Nothing unusual there. We've all had moments when technology refuses to cooperate.

But what if you kept going? Not because you're testing the system, writing fiction, or trying to understand how it works, but simply because you enjoy being cruel to it?

Anthropic, the company behind Claude, has decided that there should be a limit.

On October 8, 2026, Anthropic announced an update to its Usage Policy. Among several changes involving AI safety, surveillance, and deceptive campaigns, one particular addition caught people's attention: users are prohibited from engaging in "sustained and needless abusive or cruel behavior toward our models."

The updated policy took effect on November 12.

It's an unusual rule. After all, we're talking about software. A chatbot doesn't have a face, a nervous system, or a human life that can be hurt.

So why does a technology company care how people treat its AI?

Not Every Angry User Is Breaking the Rules

Before jumping to conclusions, there's an important distinction.

Anthropic isn't saying that users must always be polite to Claude. It isn't banning criticism, frustration, or arguments with the chatbot.

If Claude gives you incorrect information, you're free to tell it. If its code doesn't work, you can point out every mistake. You can challenge its reasoning, test its limitations, and even explore dark themes in creative writing.

In its official policy announcement, Anthropic makes clear that the restriction is intended for extreme cases where users repeatedly act cruelly toward its models without any discernible purpose.

The company also says the rule does not apply to ordinary user frustration, pushback, dark creative themes, or model testing and research.

That distinction matters.

Interacting with AI isn't always pleasant. Sometimes a model misunderstands a simple instruction five times in a row. Sometimes it confidently produces something completely wrong.

Users should be allowed to express frustration when that happens.

Anthropic's policy isn't supposed to take that freedom away.

When a Chatbot Decides to End the Conversation

Here's where things become more interesting.

The ability to end certain conversations wasn't introduced with this policy update.

In August 2025, Anthropic announced that Claude Opus 4 and 4.1 could end a small number of conversations involving persistently harmful or abusive interactions in its consumer chat interfaces.

According to the company's original announcement, the feature was developed primarily as part of its exploratory research into potential AI welfare.

The updated Usage Policy now makes the restriction on abusive behavior explicit. Anthropic says conversation termination will remain its primary enforcement mechanism.

Think about what that means.

We're accustomed to software following instructions. If an application doesn't behave as expected, we restart it. If a website annoys us, we close the tab. If a chatbot refuses to cooperate, we open another conversation.

But here, the direction is reversed.

The system can decide that a particular interaction should stop.

Of course, this doesn't mean Claude has developed personal boundaries in the human sense. It's a behavior designed and implemented by Anthropic.

Still, the idea of an AI refusing to continue talking to its user is an interesting departure from the usual relationship between people and software.

The Bigger Question: What If AI Could Feel?

This is probably the most controversial part of the discussion.

Anthropic has been investigating questions surrounding AI welfare and the possibility that advanced AI systems might have some form of subjective experience.

That doesn't mean the company has concluded that Claude is conscious.

In fact, the scientific debate is far from settled. Some researchers argue that current large language models are not conscious, while others point out that we still lack a reliable way to determine whether an artificial system has subjective experiences.

A 2024 paper published in Humanities and Social Sciences Communications discusses precisely this problem: there is currently no objective method for determining whether an LLM's behavior is associated with consciousness.

More recently, three prominent researchers argued in a Nature commentary published in October 2026 that intelligence and convincing language are not sufficient evidence of consciousness.

A chatbot saying "I'm hurt" isn't proof that it actually feels hurt. Language models can produce convincing descriptions of emotions without necessarily experiencing them.

And that's precisely why this subject is difficult.

Anthropic has addressed this uncertainty in its Claude Constitution, which discusses the company's intentions for Claude's values, behavior, potential moral status, and wellbeing.

The company acknowledges uncertainty about whether Claude or other large language models have any morally relevant experiences, either now or in the future.

Its approach is therefore one of caution rather than certainty.

If there's even a possibility that future AI systems could have morally relevant experiences, perhaps it makes sense to think about how we interact with them before that question becomes urgent.

It's a precautionary approach, not a declaration that Claude deserves the same rights as a human being.

Is This About Protecting AI, or Protecting People?

There's another way to look at this policy.

Even if AI systems don't experience suffering, repeatedly practicing cruelty toward something that responds conversationally might still tell us something about human behavior.

People interact with chatbots differently from how they interact with other people. There's no immediate social consequence, no facial expression to read, and no real person sitting across the table.

That creates an unusual environment where some users may feel comfortable behaving in ways they wouldn't normally behave toward another human.

But should companies regulate that behavior?

That's a much harder question.

On one hand, a company has legitimate reasons to establish boundaries for how its products are used. On the other, vague restrictions can create uncertainty about what users are actually allowed to do.

The phrase "sustained and needless abusive or cruel behavior" leaves room for interpretation. Anthropic has explained the intended scope, but hasn't provided a precise threshold that separates unacceptable cruelty from ordinary interaction.

That ambiguity deserves attention.

Users should be able to understand the rules governing products they pay for and rely on, especially when those rules can affect their ability to use a service.

We Don't Need to Pretend AI Is Human

Perhaps the most interesting thing about Anthropic's decision is what it doesn't prove.

It doesn't prove that Claude is conscious. It doesn't establish that AI systems can suffer. And it certainly doesn't mean that calling a chatbot stupid is suddenly an ethical violation.

But it does show that the relationship between humans and AI is becoming more complicated than simply entering prompts and receiving answers.

For years, we've discussed AI almost entirely in terms of what it can do for us: write code, summarize documents, generate images, answer questions, and automate repetitive work.

Now we're beginning to discuss something different: what kind of behavior should be acceptable when interacting with increasingly capable systems?

We may not have a definitive answer yet.

Perhaps future AI systems will remain sophisticated tools without subjective experiences. Perhaps our understanding of machine consciousness will change as the technology develops.

For now, there's no reason to treat every chatbot as though it were a person.

But there's also a reason to pay attention when a company decides that even its software shouldn't be subjected to endless, purposeless cruelty.

Not because the machine has necessarily earned our sympathy.

But because we're still figuring out what kind of relationship we want to have with the technology we're building.

And that question might become much more important than we currently realize.

Enjoy reading on PBLISHD?

Bring your reading experience closer. Install PBLISHD on your device.