If you think far back, you’ll notice that for as long as technology has been applied to learning, the central challenge has remained the same, for at least the past few decades. Abstract concepts resist verbal explanation, and the tools available to “visualize” them have always demanded both a high level of professional skill and a significant investment in time and capital. The widespread access to generative AI is set to change all of that, and we received the first hint with the introduction of Claude’s interactive visuals.
Perhaps for the first time in the history of technology, a model can take a complex concept and render it into a demonstrable audiovisual clip from a text prompt alone. I am, of course, talking about Gemini Omni. Here is why I think Google has completely undersold its applications in education, and why it’s a massive leap in the design and delivery of learning.
Visual learning has always worked, but…
Producing it never came cheap
Cognitive science has always supported the view that, since the brain processes verbal and visual information through separate channels, engaging both simultaneously results in better information retention. You may have noticed comments on YouTube explainers that frequently state, “This ten-minute clip did a better job at explaining this concept than my years of schooling ever did.” So, the question is, if audiovisual media almost always results in a better learning experience, why isn’t it more common in university and school curriculums?
The problem has historically been the production gap between knowing that visual learning works, and being able to deliver the same at scale. For example, a ten-minute explainer that you watch on a YouTube channel requires a subject-matter expert, a designer, animation software, narration, and someone with the expertise to put all of it together in a way that’s conducive to learning. While this was accessible to well-resourced institutions, it was unavailable, largely, to everyone else.
I tried Gemini Omni, and it’s so good it feels straight out of science fiction
There’s a new king of video generation in town, and it’s no joke
Gemini Omni is the first sign of the production gap closing
A text prompt shouldn’t be able to produce this, and yet…
Perhaps the first time I realized that the production gap was effectively closing was the first time I tested Gemini Omni. The model combines the knowledge capabilities of a large language model with those of a video producer. For context, Gemini Pro models are estimated to have been trained on somewhere between five and fifteen trillion tokens of multimodal data, which is an overwhelmingly ginormous breadth of information to draw from.
Google describes the model as one that “combines an intuitive understanding of physics” with “Gemini’s knowledge of history, science and cultural context to bridge photorealism and meaningful storytelling”. This means that the model already carries a working understanding of virtually any subject matter before the prompting even begins.
Not the one to take any claims at face value, I tested the limits of that claim with a non-specialist’s level of understanding. I prompted the model to generate explanatory clips covering Mendel’s laws of inheritance, Newton’s universal law of gravitation, and even differential calculus. While I wouldn’t go as far as to state that the results were perfect, it would be a mistake to evaluate a model that can visualize complex information in a matter of minutes based on relatively minor errors that may as well just be imperfections in generation. Any subject-matter expert with a clearer brief and deeper domain expertise could use the same model to produce something considerably better, without needing expensive tools or skill that requires years of practice to perfect. Now, if that isn’t democratizing audiovisual learning at large, then what is?
Google built something transformative for educators
And decided to market it for TikTok instead
Despite its obvious capabilities, the marketing behind Gemini Omni is almost shockingly disingenuous to the actual, transformative use-cases it can enable. If you were to go on Gemini Omni’s landing page right now, you’d find a tagline that reads, “Speak it. See it. Share it”, and alongside it, curated templates and a few AI avatar features. The flagship marketing video is a clip of a person dancing in a field, half real, and half-transformed into a cartoon monster a second later. This happens to be the demonstration for a model that was capable of making abstract concept in physics accessible to a twelve-year-old in my own tests. It’s exactly the kind of missed opportunity that comes with real consequences for how the technology gets adopted and who ends up benefiting from it.
Despite most of the marketing around Gemini Omni being geared towards content creators and the social media algorithm, it’s important to realize what the technology represents for learning and development. For perhaps the first time, the ability to produce rich audiovisual explanation is no longer reserved for institutions with large budgets, professional animators, and to those with years of technical expertise, and that, itself, is an exciting prospect.
Here’s hoping the algorithm doesn’t win this one
It’s not difficult to imagine the possible use-cases, even if it has only been a few weeks since the model was released to the public. Teachers explaining complex concepts with custom visuals, researchers communicating their findings more effectively, and public service announcements and infographics tailored to audiences that may otherwise struggle with traditional forms of communication all seem much more attainable. I do, however, sincerely hope that these applications don’t end up being overshadowed by the relentless pursuit of “viral content”.
