Adding another link. I had a similar experience with a student. The AI output looked so exciting and right that it didn't even occur to them checking whether any of it made sense. spoiler: it didn't tagide.com/blog/llm/the...
Recommend read! Struggling to implement code/bring your thoughts on paper is what builds topic-level expertise. Great tool for everyone already on the other side of the learning journey, but what about trainees? Would I have wanted it for myself when I started out? Not entirely sure.
From the article: "So what will happen when everyone decides to ignore the insides, and just trusts the black box? What will happen when people don’t know what questions to ask because they never learned the insides of anything?"