Recent debates on the use of generative AI in research, teaching, and learning have increasingly framed AI assistance through the lenses of transparency and intellectual humility. Within this framing, the use of AI tools is often presented as a legitimate response to uncertainty or gaps in one’s knowledge, provided that such use is disclosed and undertaken in good faith. Admitting “I’m not sure” and seeking assistance is rightly seen as a professional virtue rather than a failing.

While this position is intuitively appealing, it risks obscuring a more fundamental issue: the situated appropriateness of AI use. The question is not merely whether AI assistance is transparently acknowledged or motivated by humility, but whether its use is appropriate given the user’s level of intellectual and professional readiness. AI tools are not universally benign aids. In some contexts, their use may undermine rather than support sound judgement.

At the heart of this issue lies a simple but often overlooked paradox. To verify, evaluate, or improve AI-generated outputs in a given domain, one must already possess sufficient expertise in that domain. This is not a controversial claim but a matter of common sense. Current AI systems, including the most advanced generative models, are well known to produce errors, omissions, and fabrications. Their outputs therefore require meaningful human oversight. That oversight cannot be performed adequately by someone who lacks the relevant knowledge and experience.

This paradox becomes especially clear when considering the use of AI tools for academic writing and research. It is frequently suggested that researchers who are less confident in a topic or in academic writing can rely on AI to identify literature, generate summaries, or improve language, provided they “check the sources” or “take responsibility for the final output.” Yet this raises an obvious question: if a researcher does not already have sufficient expertise in the topic or the language, how can they reliably judge the accuracy, completeness, or quality of what the AI produces? If they are able to evaluate the output confidently, one might reasonably ask what substantive problem the AI is solving in the first place.

The same paradox becomes even more visible in teaching and learning contexts. Advocates of generative AI in teaching sometimes argue that traditional skills such as writing essays or literature reviews are becoming less important, and that students should instead be taught to analyse, critique, and improve AI-generated texts. However, this proposal collapses under scrutiny. If students do not already know how to write a coherent essay or literature review themselves, they are not in a position to judge whether an AI-generated version is any good, let alone to revise and improve it meaningfully. One cannot critically evaluate a practice one has never mastered.