Grokking: Can AI Solve New Problems or is it all Memorization?

Send us Fan Mail

This episode dives into one of the most debated questions in AI today: Are large language models actually reasoning, or are they just incredibly sophisticated parrots?

Our discussion traces two schools of thought. On one side, new ideas like grokking and adaptive reward systems suggest that AI may soon cross the threshold into true problem-solving—discovering novel solutions it was never explicitly trained on. On the other side, researchers argue that LLMs mainly excel at pattern recognition and context-driven extrapolation, not genuine abstraction. When patterns shift or problems fall outside their training distribution, even the best models tend to break.

The episode explores the fragility of current AI reasoning and introduces a promising direction: Neuro-Symbolic AI—a hybrid approach, which combines the statistical power of LLMs with structured logical systems.

Support the show

If you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!