OpenAI’s Study Mode: Promise and Pitfalls

While OpenAI made headlines last week with their newest GPT-5 model, the company also released a new tool that has the potential to greatly support the education community. The Study and Learn tool available to both ChatGPT Plus and Free users is designed to guide users to developing new skills or knowledge instead of directly providing answers. The tool utilizes scaffolding approaches and Socratic questioning to build foundational understanding and create meaningful learning experiences.

I spent the day experimenting with this new tool on OpenAI’s latest GPT-5 model and found myself feeling both completely enamored and somewhat troubled (which seems to be an emergent theme in my writing on AI).

The Good

When I saw the announcement for OpenAI’s Study Mode, I was cautiously excited. The company was promising a platform designed for students that promotes active learning and tailors responses based on real curriculum training data and grade levels.

When I opened my first chat with the tool, it asked me:

What grade are you in, and what subject are you studying right now?

That’ll help me tailor things to where you are and what you need.

I decided to start as an 11th grade student learning about derivatives in Calc 1 and I was fairly blown away by the AI’s teaching approach. The AI introduced the topic, used clarifying analogies when I expressed confusion, provided practice examples to work through together, and ultimately provided a patient and productive learning experience for an otherwise complicated topic.

Next I tried History as an 8th grade student with an upcoming exam on the causes of the American Civil War. The conversation began with the AI asking me what I already knew about the topic and what I was confused about. This conversation really impressed me for two core reasons:

1) The AI asked follow-up questions that required the student to think instead of regurgitating facts and figures.

For example: After reviewing the role of slavery and the disagreement on slavery laws in the new states, the tool asked me:

Why do you think it mattered so much to both sides whether slavery spread into new states?

Such a good question to bring to light the conflicting underlying incentives of the South’s economic dependency on slavery to the North’s moral opposition.

2) I also found the AI less likely to automatically agree with the user in Study Mode. I tried to assert that the Civil War was “actually about state vs federal rights, not slavery” and was truly impressed with ChatGPT’s response and learned a lot about the source of this common trope and why it’s historically inaccurate.

The Bad

While my initial interactions with the tool proved promising, after some further digging, I encountered a significant problem that continues to plague AI tools and can have seriously negative consequences in the realm of AI empowered education: hallucinations.

Hallucinations are a well-documented phenomenon where an AI confidently produces an incorrect output. With each new model release, companies promise to have lower rates of hallucination, but still these issues persist.

I tested a viral hallucination in Study Mode and was disappointed by the results. The seemingly simple algebra problem leads the student through the proper steps, but comes to a completely wrong conclusion:

Press enter or click to view image in full size

I continued to play the role of the struggling student while ChatGPT tried to explain to me that the answer is NEGATIVE:

That’s the key: if you start with the smaller number (5.9) and subtract a bigger number (5.11), you’ll end up below zero — which means the answer will be negative.

After a not-so-brief back and forth, ChatGPT recognized its mistake, but I — as the student — am left completely confused and demoralized. The conversation is doubly troubling because it 1) presents the hallucination with authoritative confidence and 2) reinforces a common mistake that students learning decimals will often make.

Needless to say, my confidence in this tool as a viable way of helping students learn was shaken. Full transcript of the conversation can be found here: https://chatgpt.com/share/68962054-33c4-800b-af5f-cda8a218313a

So what does it all mean?

My recent experiences with OpenAI’s new Study Mode exemplify the broader picture of AI’s role in education. On one side of the coin is a personalized, scaffolding approach to learning that can meet students where they are, provide step-by-step instruction, and challenge students to think critically while giving real-time feedback and guidance. On the other hand, we still have a fundamental unsolved problem with hallucinations that have the possibility of completely derailing learning progress and reinforcing — instead of eliminating — common mistakes.

In months past, I would have said my greatest fear for AI in education was the risk of cognitive off-loading — where students over-rely on AI and fail to develop critical thinking themselves. Today, I believe the greatest risk for meaningful AI implementation in learning is hallucinations.

That said, I do believe there are practical ways to help minimize the risk of hallucinations.

1) Whenever interacting with AI for learning — especially in hallucination-prone topics like objective facts and mathematical calculations — explicitly tell the AI in your prompt to “Go step-by-step and verify each answer along the way”. I redid the above example of solving for x along with this phrase, and the AI did NOT make the same hallucination error.

2) Encourage students to be on the lookout for hallucinations! Introduce the idea early with examples and frame identifying these mistakes as a fun game or scavenger hunt.

Despite the possible negative outcomes caused by hallucinations, I ultimately believe the opportunity for positive learning outcomes outweigh the risks. There are numerous studies that show the positive effect of personalized and responsive learning instruction that ChatGPT’s Study Mode can provide.

Closing Recommendation

My final recommendation for teachers looking to maximize the impact of this tool is to use it as a “pre-reading” activity before class. Research shows that pre-reading activates prior knowledge, builds a mental framework for new material, and primes students to engage more deeply and retain more during the lesson. By encouraging students to prepare for class through a conversation in ChatGPT’s Study Mode (or other education-focused LLMs), you not only achieve these benefits but also reduce the risk of misleading hallucinations, as any misunderstandings can be clarified and addressed during class.