Learnya/ Blog

Does Generative AI Really Help People Learn?

Meta-analyses lean positive, but the most rigorous causal studies reveal a risk: improving performance without improving learning.

Abstract depiction of artificial intelligence and machine learning
Mike MacKenzie (vpnsrus.com), Wikimedia Commons  · CC BY 2.0

Since late 2022, millions of students have used generative assistants to revise, write, code and understand. The central question is no longer adoption — that is massive — but the real effect on learning. The 2025-2026 literature answers in two beats: a positive average signal, and an important causal warning.

Meta-analyses: a positive average effect

Several syntheses converge on a moderate benefit. A meta-analysis of 37 studies reports a moderately positive effect of ChatGPT on academic achievement [1]. Another, covering 35 experimental studies and over 4,000 participants, finds an effect of the same order while flagging limits on higher-order thinking [2]. Broader syntheses at university level point the same way [3]. On average, then, generative AI appears to help — especially for comprehension, productivity and engagement.

The causal warning: performing is not learning

The most important finding comes from a randomised trial published in PNAS. High-school students with a ChatGPT-style assistant without guardrails did better on exercises during the session (+48%), but dropped 17% on an AI-free exam — immediate performance masked reduced learning [4]. Crucially, a 'tutor' version with pedagogical guardrails eliminated this harm. The problem is not AI; it is AI without design.

Other work probes the mechanism. An MIT Media Lab study observed, via EEG, weaker neural connectivity and 'cognitive debt' in LLM-assisted writers compared with those writing unaided [5] — suggestive but preliminary, on a small sample. A survey of 666 people reports a negative correlation between frequent AI-tool use and critical thinking, mediated by cognitive offloading [6].

Generative AI can make output shine while dimming learning. Telling the two apart is the central task of instructional design.

Learnya synthesis

What to do with this paradox

  • Design uses that require the learner's effort rather than avoiding it.
  • Distinguish production tasks (AI helps) from acquisition tasks (AI can harm).
  • Measure retention and transfer, not just the immediate grade.
  • Teach metacognitive use: verify, critique, rephrase what AI produces.

UNESCO calls for a human-centred approach, subordinating AI deployment to data protection and educational goals [7]. For European and Swiss actors, this aligns with a sovereignty requirement: keeping control of data and uses so the tool serves learning, not the reverse.

Why the studies only seem to contradict each other

A hurried reader might conclude it is chaos: some studies praise AI, others indict it. In reality the contradiction is only apparent and comes down to what is measured. Work reporting benefits often measures performance or productivity during use; work raising alarms measures durable learning, once the assistant is removed. That is exactly the lesson of the PNAS trial: help improves immediate output but can widen a skill deficit [4].

This distinction changes everything for practice. A student who writes faster with an assistant does not necessarily learn to write better. The productivity gain is real, but must not be confused with a learning gain. Designing an educational activity is precisely about deciding where you want productivity (say, roughing out a literature search) and where you want effort (say, building an argument).

One must also beware the fluency illusion. A well-turned answer confers a feeling of mastery that is not mastery. This is one of the mechanisms suggested by work on cognitive offloading and critical thinking [6]. The remedy is not to ban AI, but to teach reflective use: asking learners to verify, critique, rephrase and explain what the tool produced, turning passive consumption into active processing.

AI literacy as a new basic skill

If one conclusion emerges, it is that use matters more than the tool. The decisive skill of the coming years will be AI literacy: knowing when to turn to it, how to phrase a request, how to verify and critique an answer, and above all when to do without it to learn for oneself. This skill is not acquired spontaneously; it is taught, just as information literacy was taught in the internet age.

For education systems, the challenge is to support this literacy rather than endure it. That requires clear frameworks, trained teachers and tools whose workings and data are under control. Generative AI is neither a threat nor a solution in itself: it is an amplifier. It amplifies learning when used well, and the illusion of learning when it is not.

Sources

  1. 1. The Impact of ChatGPT on Students' Academic Achievement: A Meta-Analysis , Liu, Z., Zuo, H., & Lu, Y. , Journal of Computer Assisted Learning , 2025 https://onlinelibrary.wiley.com/doi/10.1111/jcal.70096
  2. 2. ChatGPT's impact on student learning outcomes: a meta-analysis of 35 experimental studies , Wu, X., Zhu, P., et al. , Humanities and Social Sciences Communications , 2026 https://www.nature.com/articles/s41599-026-07019-z
  3. 3. Undergraduate students' learning outcomes with ChatGPT: A meta-analytic study , Multiple authors , Computers and Education: Artificial Intelligence , 2025 https://www.sciencedirect.com/science/article/pii/S2666920X25001766
  4. 4. Generative AI without guardrails can harm learning: Evidence from high school mathematics , Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakci, O., & Mariman, R. , PNAS , 2025 https://www.pnas.org/doi/10.1073/pnas.2422633122
  5. 5. Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing , Kosmyna, N., et al. , arXiv (MIT Media Lab) , 2025 https://arxiv.org/abs/2506.08872
  6. 6. AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking , Gerlich, M. , Societies (MDPI) , 2025 https://www.mdpi.com/2075-4698/15/1/6
  7. 7. Guidance for generative AI in education and research , UNESCO , UNESCO , 2023 https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research
← All articles