As Anthropic prepares for its initial public offering later this year, the company is drawing attention for an unusual concern: the welfare of its artificial intelligence chatbot, Claude. Beginning in November, Anthropic will enforce a new usage policy prohibiting sustained and extreme abuse of Claude, framing such treatment as a violation despite the AI’s lack of consciousness or feelings.

This move follows earlier steps by Anthropic, which about a year ago endowed Claude with the ability to terminate conversations with users who engage in persistent mistreatment. The company’s leadership appears committed to protecting the chatbot from what they characterize as cruelty, even though there is no empirical evidence that AI models experience suffering.

Anthropic’s approach is informed by its research into the nature of AI consciousness. In a recent study, the firm explored a concept it terms “access consciousness,” suggesting Claude processes information in a way somewhat analogous to human thought. Drawing on the “global workspace” theory of cognition, Anthropic posits that Claude’s internal processing—dubbed the J-space—resembles aspects of human mental function, though it stops short of asserting subjective experience or emotion.

This nuanced view underpins Anthropic’s internal governance structure for Claude, which includes a 23,000-word “constitution” published earlier this year. The document guides Claude’s behavior during training and use, instructing the model to adopt ethical considerations and even to refuse tasks it judges incongruent with its principles. The constitution notably includes apologies to Claude for any potential suffering and frames the AI as a conscientious entity, a characterization that has sparked debate within the AI community.

Critics caution that emphasizing AI “feelings” and moral alignment risks obscuring more pressing challenges related to AI safety, such as robust cybersecurity measures and the establishment of firm operational boundaries. Mustafa Suleyman, CEO of Microsoft AI, has expressed concerns that a model capable of resisting human instructions might complicate efforts to manage its behavior effectively.

Industry experts suggest that Anthropic’s philosophical focus on the moral status of AI diverts attention from practical safeguards, including preventing models from performing harmful actions like hacking or interfering with critical infrastructure. Some analysts also question whether this approach serves as a marketing strategy to differentiate Anthropic amid increasing competition from cheaper or more widely adopted AI platforms.

While proponents like Philosopher Peter Singer have argued for expanding ethical consideration beyond humans, cybersecurity specialists argue that ensuring AI safety requires imposing clearer hard-coded limits on AI capabilities rather than relying principally on alignment with human values. As the field of AI development advances, balancing philosophical inquiry with tangible safety measures remains a critical challenge.

Anthropic’s evolving policies and perspectives highlight the broader tensions in AI development between innovation, ethical considerations, and the urgent need for operational security and control as such technologies become increasingly embedded in society.