And the most extreme case on the other end of the spectrum from proofs is the entirely unverifiable “is this model conscious?”
What it has to tell you on that when you sample from it has ~no probative value at all
internal and not subject to verification by anything, so you can very easily drift off into that LLM psychosis people talk about. Or just develop confidently wrong opinions.
The more automatically verifiable the output, the better the case for AI
