Most people, including really accomplished people, don't have an accurate mental model of how LLMs operate (and why would they?)
You see this in widespread beliefs that AI is just copying from known sources, or that it only produces average answers, or that it can't generate effectively novel ideas