Google Brain and the Birth of Modern AI: From Pre-Training to Visual AGI

Artificial intelligence did not begin with ChatGPT. It did not begin with Claude, Gemini, coding agents, image generators or the current race toward AGI.

The academic field of AI goes back decades. But the AI that now dominates the world — large language models, multimodal systems, coding assistants, frontier labs and agentic tools — has a more specific origin story. That story is not only about algorithms. It is about people, institutions, papers, infrastructure and research cultures that allowed strange ideas to become foundational ones.

It is about Andrew Dai and Quoc V. Le exploring pre-training and fine-tuning before that pattern became the industry's default. It is about Ilya Sutskever, Oriol Vinyals and Quoc Le helping establish sequence-to-sequence learning. It is about the transformer paper, where Google researchers introduced the architecture that would later power modern large language models. It is also about Jeff Dean, Geoffrey Hinton, Google-scale infrastructure, and a research culture that gave unusual people enough freedom to try ideas before anyone knew which ones would matter.

And increasingly, it is about what comes next: visual reasoning, physical-world intelligence, and the question of whether language models alone are enough to reach AGI. The short version: modern AI was born from a convergence of four forces — language modelling as a foundation for understanding; transformer architectures that could scale; web-scale data and compute; and a rare concentration of research talent, especially at Google Brain. But the next phase may require a fifth force: models that can understand the visual and physical world natively.

Much of this framing comes from an interview with Andrew Dai, a former Google Brain and DeepMind researcher, on the Inside the Silicon Mind podcast. The factual claims below have been checked against the original research papers and reporting; the sources are listed at the end.

Why Google Brain matters

Google Brain was not the only place that mattered in modern AI. DeepMind, FAIR, OpenAI, academic labs and many independent researchers all shaped the field. Any serious history has to say that clearly. But Google Brain occupies a special place because it combined three things that rarely coexist: open-ended research freedom, world-class machine-learning infrastructure, and extreme talent density.

That combination is why Andrew Dai, reflecting on his years inside Google Brain, described it as something like the Bell Labs of the AI era. The comparison is not that Google Brain invented everything — Bell Labs did not invent everything either. The comparison is about concentration: a place where foundational research, engineering ambition and future products were close enough to influence one another.

In Dai's telling, the culture mattered as much as the technology. Researchers could think freely. They could test ideas without immediate product pressure. They could discuss research in micro-kitchens and over lunch. They could be wrong in front of smart colleagues without being punished for it. That is not a soft detail; it is central to the story. In frontier research, most ideas fail. A culture that punishes failure produces cautious work. A culture that combines high standards with permission to be wrong creates the conditions for breakthroughs.

The 2015 insight: pre-training before it became obvious

One of the key moments in the birth of modern AI came in 2015, when Andrew M. Dai and Quoc V. Le published work on semi-supervised sequence learning. The basic idea now sounds familiar: train a model on a broad task using unlabelled data, then fine-tune it for a more specific supervised task. At the time, that was not yet the obvious mainstream path.

Dai and Le used recurrent neural networks, specifically LSTMs, because transformers did not yet exist. They trained models to predict what comes next in a sequence — essentially a language modelling objective — then fine-tuned them on downstream tasks such as sentiment analysis. The important lesson was not just that the method worked. It was why it worked.

A language model is not merely learning to guess words. To predict language well, it must absorb grammar, style, sentiment, facts, relationships and hidden structure. It has to compress patterns from the world as expressed in text. This became one of the central intuitions behind modern AI: language modelling can become a route to general-purpose representation learning. That idea later became normal. But in 2015, it was still a research discovery.

Sequence-to-sequence: the earlier foundation

The 2015 pre-training work did not appear from nowhere. A year earlier, Ilya Sutskever, Oriol Vinyals and Quoc V. Le published sequence-to-sequence learning work using neural networks. That paper showed that neural networks could map one sequence to another: one sentence into another sentence, one language into another language, one structured input into one structured output.

That mattered for translation. But it also mattered more broadly, because it helped establish a pattern that now sits beneath many AI systems: take an input, encode it, transform it and generate a useful output. Translation, summarisation, question answering, chat, code generation and reasoning traces can many of them be understood as descendants of this sequence-to-sequence mindset. The modern AI era did not come from one paper. It came from a chain of papers, experiments and intuitions that began to reinforce one another — a lineage we trace in our AI breakthroughs timeline.

The transformer: the missing machine

The transformer changed the scale of the game. The 2017 paper Attention Is All You Need proposed a new architecture based on attention mechanisms, dispensing with recurrence and convolution. This made sequence modelling more parallelisable and much easier to scale.

That mattered because pre-training alone was not enough. LSTMs could demonstrate the principle, but transformers made it industrial. The formula became simple enough to describe but extremely hard to execute: train a large transformer on massive data using a language modelling objective, then adapt it through fine-tuning, instruction tuning, reinforcement learning or other post-training methods. That formula now sits behind much of the AI boom.

GPT-1 showed the power of generative pre-training for language understanding. GPT-2 showed that language models trained on web text could begin to perform tasks from naturally occurring demonstrations. Scaling-law work then showed that performance could improve predictably with more parameters, data and compute. Later work such as Chinchilla refined the relationship between model size and training data. By the time ChatGPT arrived, the public saw a sudden product. But the deeper technical path had been forming for years.

The triangle: objective, architecture, data

Andrew Dai's framing is useful because it reduces the birth of modern LLMs to a triangle: the transformer, the language modelling objective and web-scale data. The transformer gave the machinery. The objective gave the training signal. The web gave the fuel. Compute made the triangle move faster.

But even this technical triangle is incomplete. It still needed people with the taste to notice which ideas mattered, the infrastructure to test them, and the culture to let them grow. That is where Google Brain's role becomes more than a footnote.

Jeff Dean and the infrastructure of ambition

Jeff Dean's importance in this story is partly technical and partly cultural. Modern AI is not only clever mathematics. It is systems engineering at extreme scale. It requires distributed training, huge datasets, accelerators, infrastructure, experiment velocity and the ability to turn research ideas into trainable systems.

A lab without infrastructure can write papers. A lab with infrastructure can test whether ideas scale. Google Brain had that advantage: it combined open-ended machine-learning research with Google-scale computing infrastructure. That meant researchers could explore ideas that would have been impossible in a smaller academic setting. This is why infrastructure is not separate from intelligence. In modern AI, infrastructure shapes what researchers are able to imagine.

Geoffrey Hinton and the brain-inspired philosophy

Geoffrey Hinton's influence was different. His contribution was not only a particular paper or technique. It was a philosophy: the human brain remains the only working example of general intelligence we have, so artificial systems should learn from that example.

Not by copying biology literally. Artificial neurons are much simpler than real neurons, and backpropagation is not obviously what the human brain does. But the broader belief was powerful: build general learning systems, expose them to the right data, and let optimisation discover useful internal representations. That view helped move AI away from hand-coded rules and toward learned representations. Earlier AI often tried to encode intelligence explicitly. Modern AI tries to grow useful behaviour through training. That is one of the great shifts of the field.

The Google Brain diaspora

One of the most important consequences of Google Brain was not a single model. It was the people who left. Many names that now define frontier AI passed through Google Brain, DeepMind or the wider Google research ecosystem.

• Dario Amodei worked at Google Brain before OpenAI and later became co-founder and CEO of Anthropic.
• Ilya Sutskever worked at Google, co-founded OpenAI and later founded Safe Superintelligence.
• David Ha led Google Brain's research team in Japan and became co-founder and CEO of Sakana AI.
• Liam Fedus worked at Google Brain and OpenAI before co-founding the materials-science startup Periodic Labs with former Google Brain researcher Ekin Doğuş Çubuk.
• Sara Hooker worked at Google Brain and later led the non-profit research lab Cohere For AI.
• Andrew Dai spent close to 14 years at Google, more than a decade of it across Google Brain and DeepMind, before co-founding Elorian — a visual-reasoning lab that raised about $55 million, with backing reported to include Nvidia and Jeff Dean.

This is why people compare the Google Brain network to the PayPal Mafia. The analogy is imperfect but useful. The point is not that everyone came from one company. The point is that dense early networks can shape an industry for decades. Google Brain produced ideas. It also produced founders.

Why the frontier labs were not all built inside Google

This raises an obvious question. If Google Brain had the talent, culture and infrastructure, why were OpenAI, Anthropic, Safe Superintelligence, Sakana, Periodic Labs, Elorian and other frontier efforts not simply built inside Google?

The answer is partly organisational. Large companies are excellent places to learn, gather resources and work with extraordinary people. But they also bring product pressure, politics, promotion ladders, internal competition and strategic constraints. For some researchers, leaving was not a rejection of Google Brain. It was the next stage of the Google Brain effect. The lab had trained people to think bigger. Eventually, some of those people wanted more ownership, more focus and fewer institutional constraints. The ideas moved because the people moved.

Psychological safety and osmosis

Two cultural ideas from Dai's interview deserve special attention: psychological safety and osmosis. Psychological safety means researchers could show unfinished ideas, admit uncertainty, challenge each other and be wrong without destroying their status.

Osmosis means people learned by being around elite colleagues. They absorbed research taste. They learned when to abandon a project and when to keep pushing. They heard how senior researchers framed problems. They saw ambition modelled at close range. This is difficult to reproduce in a fully remote or highly bureaucratic environment. A scheduled video call can transfer information. It is less good at creating accidental collisions between ideas. Many important research conversations happen before anyone knows they are important. That is one reason the birth of modern AI was social as well as technical.

Was Google Brain the only birthplace of modern AI?

No. A stronger history should not overclaim. Google Brain was central, but it was not alone. DeepMind contributed reinforcement learning, scientific AI, AlphaGo, AlphaFold, Gemini and major work on scaling and multimodality. OpenAI pushed the GPT line into public consciousness. FAIR contributed important open research and model work. Academic labs developed many of the underlying methods, benchmarks and theory.

So the better claim is not "Google Brain invented modern AI." The better claim is: "Google Brain was one of the densest birthplaces of modern AI, because it combined pre-training insights, transformer-era research, infrastructure and a talent network that later seeded frontier labs." That is both more accurate and more credible.

Are we already at AGI?

Dai's answer is essentially no. His argument is not that language models are unimpressive — quite the opposite. On text, code and some abstract tasks, modern AI systems are already extremely powerful. But much of the real economy is not text.

Enterprises work with floor plans, electrical diagrams, wiring systems, CAD models, engines, construction sites, medical images, farms, robots, infrastructure and data centres. These are visual, spatial and physical problems. A model that writes good code but cannot reliably understand what two things a wire connects is not yet general in the way industry needs.

This is the "baby vision" problem. Current multimodal systems can describe images, but they still struggle with precise visual reasoning: counting, spatial relationships, object permanence, diagrams, occlusion, rotation, physical constraints and design intent. Recent benchmarks make the gap concrete — work such as BabyVision and VisuLogic finds that leading multimodal models score far below human baselines on basic visual tasks. That matters because the next wave of AI may depend less on producing better text and more on understanding the world behind the text.

Where are we going? The next phase of AI

The first phase of modern AI was language-centric. That made sense. Language is abundant, digitised, compressible and deeply connected to human knowledge. The web gave researchers an enormous training set. Transformers gave them a scalable architecture. Language modelling gave them a simple objective.

The next phase may be physical and visual. That means AI systems that do not merely caption images, but reason through visual information directly — systems that can understand diagrams, spatial layouts, physical constraints, mechanical assemblies, engineering drawings, robotics tasks and real-world cause and effect. This is where companies like Elorian are aiming: native visual reasoning. The shift can be summarised like this:

• First wave: models that read and write
• Second wave: models that see and describe
• Next wave: models that see, reason, design and act

This connects to a broader trend around vision-language-action models in robotics: systems that take visual observations and language instructions, then generate actions in the physical world. That does not mean robots arrive everywhere tomorrow. It means the research frontier is moving from "AI as text machine" toward "AI as world model."

Why visual reasoning could matter more than another chatbot

Another chatbot may be useful. But visual and physical reasoning could open much larger domains. If models can reason visually, they become more useful in architecture, engineering, manufacturing, robotics, construction, aerospace, agriculture, medicine, data-centre design and scientific discovery.

These are not just "nice to have" use cases. They are where much of the physical economy lives. A model that can read a wiring diagram, understand a machine part, identify a physical bottleneck, reason over a floor plan or help design infrastructure is operating in a different category from a chatbot. That is why the "next birth" of AI may not look like a bigger text model. It may look like a model that finally understands visual structure and physical constraint.

The better frame: AI keeps being born

The phrase "the birth of AI" can be misleading if it implies one moment. There was the original birth of AI as an academic field in the twentieth century. There was the birth of deep learning as the dominant paradigm. There was the birth of modern LLMs through pre-training, transformers, web data and scale. Now there may be another birth: visual and physical-world AI.

This framing avoids a fake single-origin story. It lets us say something more accurate: AI has had several births. The one we are living through now began with language. The next one may begin with vision. That is a better thesis than simply saying "Google Brain created AI."

FAQ

The human story behind the machine

The public usually sees the product: ChatGPT, Gemini, Claude, Midjourney, Copilot, agents. The deeper story is harder to see. It is papers, micro-kitchen conversations, failed experiments, compute infrastructure, unusual hiring, psychological safety and a group of researchers learning from one another by osmosis.

Andrew Dai and Quoc Le saw that language modelling could become a foundation for understanding. Ilya Sutskever, Oriol Vinyals and Quoc Le helped establish sequence-to-sequence learning. The transformer authors gave the field the architecture it needed to scale. Geoffrey Hinton supplied a brain-inspired philosophy of learned representation. Jeff Dean helped create the infrastructure and systems culture that let research scale. People such as Dario Amodei, Sara Hooker, David Ha, Liam Fedus, Ilya Sutskever and Andrew Dai carried pieces of that culture into the next generation of AI labs.

Even Isaac Asimov belongs at the edge of the story, through Dai's recommendation of the Foundation series. Asimov wrote about long arcs of history, hidden causes and small groups of people shaping enormous futures. The AI story has some of that shape. Modern AI was not born from one invention. It was born from a convergence of people, ideas, architecture, data, compute and culture.

And if the next phase is visual and physical-world intelligence, the question is not only which model wins. The deeper question is: where is the next Google Brain? Where is the next place that combines talent density, infrastructure, freedom, psychological safety and patience for hard problems? That is where the next AI may be born.

Sources and further reading

• Andrew Dai interview — Inside the Silicon Mind: The Next Phase of AI: Visual AGI
• Andrew Dai interview — YouTube
• Andrew M. Dai and Quoc V. Le — Semi-supervised Sequence Learning (NeurIPS version)
• Ilya Sutskever, Oriol Vinyals, Quoc V. Le — Sequence to Sequence Learning with Neural Networks
• Vaswani et al. — Attention Is All You Need
• OpenAI — Improving Language Understanding by Generative Pre-Training (GPT-1)
• OpenAI — Language Models are Unsupervised Multitask Learners (GPT-2)
• Kaplan et al. — Scaling Laws for Neural Language Models
• Hoffmann et al. — Training Compute-Optimal Large Language Models (Chinchilla)
• Chen et al. — BabyVision: Visual Reasoning Beyond Language
• Xu et al. — VisuLogic: A Benchmark for Evaluating Visual Reasoning in MLLMs
• Bloomberg — Ex-Google DeepMind researchers debut Elorian, focused on visual AI
• TechCrunch — OpenAI and Google Brain researchers launch Periodic Labs
• Elorian · Sakana AI · Safe Superintelligence