bodies. they're gonna need bodies.

disclaimer alert: i’ll read anything by ted chiang.

No, Artificial Intelligence Is Not Conscious - The Atlantic

some of the best writing on AI and consciousness i’ve read as of late. i’ve always found ted chiang’s take on this space to be very interesting and well considered. this is well worth the read.

the following struck me as a succint and useful point for defining emotion.

This brings us back to my earlier contention that having a body is a prerequisite to having emotions. Experiencing an emotion such as desperation is inseparable from having stress hormones such as cortisol and epinephrine flood one’s body. Similarly, having a conscience means feeling sadness or moral repulsion at the idea of taking a certain action, and those emotions entail a physiological response, a remnant of having once felt sick with guilt after committing an immoral act. It’s interesting that an LLM can generate descriptions of actions that conscientious fictional characters would either take or refrain from taking, but this is not a replacement for a conscience.

fear strikes me as an emotion that has a tight coupling here. there’s certainly the ability the evince fear on the part of an LLM, but what does it mean to genuinely be in fear?

chiang closes with roughly this bit here …

if you think there is any chance that what you’re building might become a moral patient, you should think about what protections it deserves before you deploy it as your company’s economic engine, not after. Slave owners were not the ones to ask about the humanity of enslaved people, and factory-farm owners are not the ones to ask about the rights of animals. If we imagine Claude to be conscious, Anthropic could not possibly be entrusted with evaluating its moral status; the company has too much invested to be objective.