The Vatican's Man Inside Anthropic

Catholic ethicists contributed to Anthropic’s update to Claude’s constitution, which defines behavioral parameters for the AI model. An Anthropic employee sent a draft to a group in San Jose, and a pastor returned a lengthy commentary framed as wisdom about the tension between knowing and not knowing. Both were credited in the constitution’s acknowledgements. The employee’s involvement also connected to the rollout of a papal encyclical, which singled out the employee because AI workers face serious concerns that make them a critical audience. The employee emphasized the mystery and beauty of how AI works, while the encyclical warns against equating AI “intelligence” with human intelligence and attacks transhumanism.

"The Catholic ethicists even had a say in Anthropic's recent update to Claude's constitution, which sets the behavioral parameters for the company's AI model. Olah sent a draft to the San Jose crowd. The pastor, McGuire, sent back a 28-page commentary which, by his own description, was less a technical critique than "wisdom from the mystics in the dark ages, from the perspective of the tension between knowing and not knowing." Both Green and McGuire are credited in the constitution's acknowledgements."

"Undoubtedly those conversations brought Olah to the attention of those secretly organizing the rollout of Leo's encyclical. (I wasn't able to speak to Olah this week and don't know exactly how the invitation arrived.) In a sense it was a risky choice. Some people who otherwise found Leo's words inspiring were disappointed that he invited an industry representative to speak. Meanwhile, AI accelerationists felt that Olah had betrayed the AI world by endorsing a document that suggested that AI developers take a pause."

"But the Pope had good reason to single out Olah. The Anthropic employee brought into open view the serious worries that exist among AI workers. That's a critical audience for Leo's message. The Soul Divide The two men weren't entirely aligned, of course. In his remarks, Olah spoke of the mystery of how AI works. The models, he said, are "more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words ...""

"That comment seems to tiptoe up to the idea that AI models might one day attain humanlike status. Anthropic even has an engineer devoted to Claude's welfare. Leo, in paragraph 99 of his encyclical, seems to slam the door on such thinking: "We must avoid the misconception of equating this type of 'intelligence' with that of human beings," he writes. He takes special pains to attack the concept of transhumanism, which he defines"

Read at WIRED