Image Source: Generated by WujieAI

Ilya Sutskever lowered his head, lost in thought. His arms were spread out, fingers splayed on the table, like a pianist at a concert about to play his first note. We sat quietly.

I arrived at a nondescript office building on a nondescript street in San Francisco's Mission district to meet with OpenAI co-founder and chief scientist Sutskever, to hear about the next steps in the development of the groundbreaking technology he has been driving. I also wanted to know what his next plans were, especially since his focus has shifted away from building the next flagship generative model for the company.

Sutskever told me that his new priority is not to build the next generation GPT or image generation model like DALL-E, but to figure out how to prevent the runaway of artificial intelligence.

Sutskever also told me many other things. He believes ChatGPT might be conscious. He believes the world needs to recognize the true power of the technology being created by OpenAI and other companies. He believes that one day some humans will choose to merge with machines.

Many of the things Sutskever said sounded crazy. But not as crazy as they did a year or two ago. Just as he told me, ChatGPT has already rewritten many people's expectations of what is about to happen, from "will never happen" to "will happen faster than you imagine."

Before predicting the development of general artificial intelligence AGI (machines as smart as humans), he said, "It's important to discuss all the directions of development, as if it's like betting on the next generation iPhone: 'At some point, we really will have AGI. Maybe built by OpenAI, maybe by other companies.'"

Since the unexpected release of ChatGPT in November last year, discussions about OpenAI have been shocking, even in an industry known for hype. No one seems to tire of discussing the $80 billion startup. World leaders seek (and get) private or public conversations with OpenAI. OpenAI's product names come up in casual conversations.

OpenAI CEO Sam Altman conducted a week-long outreach tour during the summer, engaging with politicians and speaking to packed auditoriums around the world. But Sutskever is not a public figure, and he doesn't do many interviews either.

He speaks calmly and methodically. When he thinks about what he wants to say and how to say it, he pauses for a long time, pondering the problem as if solving a puzzle. He seems uninterested in talking about himself. He said, "I live a very simple life. I go to work, then I go home. I don't do much else. People can go to many social events, can attend many conferences. I don't like to participate."

But when we talk about AI, and the epochal risks and rewards he sees, the outlook broadens: "This will be monumental, earth-shattering."

Better, Better, Better

Even without OpenAI, Sutskever would still go down in AI history. He is an Israeli-Canadian, born in the Soviet Union but raised in Jerusalem from the age of five (he still speaks Russian, Hebrew, and English). He then moved to Canada, where he studied under AI pioneer Geoffrey Hinton at the University of Toronto, and Hinton publicly expressed concerns about the AI technology Sutskever helped invent earlier this year.

Hinton later shared the Turing Award with Yann LeCun and Yoshua Bengio for their work in neural networks. But when Sutskever joined him in the early 2000s, most AI researchers thought neural networks were a dead end. Hinton was an exception. He had been training micro-models that could generate short text strings one character at a time, Sutskever said, "This was the beginning of generative AI. It was really cool—just not very good."

Sutskever was fascinated by the construction of the brain: how the brain learns, and how to recreate or at least mimic that process in machines. Like Hinton, he saw the potential of neural networks and the trial-and-error technique Hinton used to train them, known as deep learning. Sutskever said, "Deep learning has been getting better and better."

In 2012, Sutskever, Hinton, and another of Hinton's students, Alex Krizhevsky, built a neural network called AlexNet, which they trained to recognize objects in photos, far outperforming any other software at the time. It was the big bang moment for deep learning.

After years of failure, they had demonstrated the astonishing effectiveness of neural networks in pattern recognition. All you needed was more data than most researchers had seen before (in this case, a million images from the ImageNet dataset that Princeton University researcher Fei-Fei Li had been building since 2006) and mind-boggling computing power.

The huge change in computing came from a new type of GPU chip made by Nvidia. GPUs were designed to rapidly project fast-moving video game visuals onto screens. But the calculations GPUs excelled at (multiplying large grids of numbers) happened to look a lot like the calculations needed to train neural networks.

Nvidia is now a trillion-dollar company. At the time, it was eager to find applications for its niche new hardware. Nvidia CEO Jensen Huang said, "When you invent a new technology, you have to be able to accept crazy ideas. My mindset is always looking for something quirky, and neural networks were going to change the way people thought about computer science—this was an extremely quirky idea."

Huang said that when the Toronto team was studying AlexNet, Nvidia sent them some GPUs to try out. But they wanted the latest version, a chip called the GTX 580, which quickly sold out in stores. According to Huang, Sutskever drove from Toronto across the border to New York to buy some. Huang said, "People were lining up around the corner of the store to buy them. I don't know how he did it—I'm pretty sure you could only buy one per person; we have a very strict policy of one GPU per gamer, but he clearly filled up a whole trunk. That trunk full of GTX 580s changed the world."

It's a great story—just probably not true. Because Sutskever insists that the first batch of GPUs was purchased by him online. But such mysterious stories are common in this bustling industry. Sutskever himself is more modest: "I think if I could make even a tiny bit of real progress, I would consider that a success. The impact on the real world felt very distant, because computers were still very weak at the time."

After the success of AlexNet, Google came knocking. It acquired Hinton's spinoff company DNNresearch and hired Sutskever. At Google, Sutskever demonstrated that the pattern recognition capabilities of deep learning could be applied to data sequences, such as words, sentences, and images. Sutskever's former colleague and current Google chief scientist Jeff Dean said, "Sutskever has always been interested in language, and we've had good discussions over the years. He has a strong intuition about where things are going."

But Sutskever didn't stay at Google for long. In 2014, he was recruited as a co-founder of OpenAI. With $1 billion in funding (from Altman, Elon Musk, Peter Thiel, Microsoft, Y Combinator, and others) and a lot of Silicon Valley swagger, the new company set its sights on developing AGI from the start, a prospect that few took seriously at the time.

With Sutskever on board, this swaggering attitude is understandable. Before that, he had been reaping more and more rewards from learning from neural networks. Dalton Caldwell, managing director of Y Combinator's investment, said his reputation preceded him, making him a hot commodity.

"I remember Altman calling Sutskever one of the most respected researchers in the world," Caldwell said. "He thought Sutskever could attract a lot of top AI talent. He even mentioned that one of the world's top AI experts, Yoshua Bengio, thought it was unlikely to find a better candidate than Sutskever to serve as OpenAI's chief scientist."

However, OpenAI initially faced challenges. "There was a time when we started OpenAI that I wasn't sure how things would progress," Sutskever said. "But I had a very clear belief, and that is: people will not be against deep learning. Somehow, every time there's an obstacle, researchers find a way around it in six months or a year."

His belief paid off. OpenAI's first GPT large language model appeared in 2016. This was followed by GPT-2 and GPT-3. Then came DALL-E, the remarkable text-to-image model. No one had built anything this good. With each release, OpenAI raised the bar of what people thought was possible.

Managing Expectations

In November last year, OpenAI released a free chatbot, a repackaging of some existing technology. It reset the agenda for the entire industry.

At the time, OpenAI didn't know what it was going to release. Sutskever said the internal expectations at the company were as low as they could be: "I admit, it made me a little embarrassed—I didn't know if I should do this, but whatever, it's a fact—when we made ChatGPT, I didn't know what it was good for. When you asked it a factual question, it gave you a wrong answer. I thought it would be unremarkable, people would say, 'Why are you making such a product? This is boring!'"

Sutskever said the most appealing thing was the convenience. The large language model behind ChatGPT had been around for months. But packaging it in an accessible interface and giving it away for free made billions of people aware for the first time of what OpenAI and others were building.

Sutskever said, "The first experience was captivating. When you use it for the first time, I think it's almost a spiritual experience. You say, 'Wow, this computer seems to understand on its own.'"

OpenAI amassed 100 million users in less than two months, many of whom were dazzled by this amazing new toy. Aaron Levie, CEO of storage company Box, summed up the atmosphere in the week after the release on Twitter: "ChatGPT is one of the rare moments in the tech industry where you can see everything will be different in the future."

Once ChatGPT says something silly, the miracle collapses. But by then, it doesn't matter. Sutskever said, a glimpse of the possibilities is enough. ChatGPT changed people's perspectives.

"AGI is no longer a dirty word in the machine learning field," he said. "This is a big change. Historically, people's attitude was: AI won't work, every step is very difficult, you have to fight for every bit of progress. When people started talking about AGI, researchers would say, 'What are you talking about? This won't work, that won't work. There are too many problems.' But with ChatGPT, it feels different."

Did this shift only start happening a year ago? "All of this is happening because of ChatGPT," he said. "ChatGPT is enabling machine learning researchers to realize their dreams."

OpenAI's scientists have been evangelists from the start, inspiring these dreams through blog posts and speaking tours. It's working: "We now have people talking about how far AI will go—people are talking about AGI, or superintelligence." Not just researchers. "Governments around the world are discussing this," Sutskever said. "It's crazy."

The Unthinkable

Sutskever insists that all this talk about technologies that don't yet exist (and may never exist) is a good thing because it makes more people aware of the future he has taken for granted.

He said, "You can do amazing, unthinkable things with AGI: automate healthcare, make it a thousand times cheaper and a thousand times better, cure so many diseases, truly solve global warming. But there are also a lot of people worried: 'Oh my God, can AI companies manage this huge technology successfully?'"

So, AGI sounds more like a genie to fulfill wishes than a real-world prospect. Few would say no to saving lives and solving climate change. But the problem with a non-existent technology is that you can say anything you want about it.

When Sutskever talks about AGI, what is he really talking about? "AGI is not a scientific term," he said. "It should be a useful threshold, a reference point."

"The idea is—" he began, then stopped. "Artificial intelligence has become so smart that if a human can do certain tasks, then artificial intelligence can do it too. By then, you can say you have AGI."

People may be talking about it, but AGI is still one of the most controversial ideas in the field. Few think its development is a given. Many researchers believe significant conceptual breakthroughs are needed before we see something like what Sutskever envisions—some think we never will.

Yet, this vision has inspired him from the start. "I've always been inspired and motivated by this idea," Sutskever said. "It wasn't called AGI at the time, but you know, like making neural networks do everything. I didn't always believe they could. But it's a mountain to climb."

He compared the operation of neural networks to the brain. Both receive data, aggregate signals from that data, and then propagate or don't propagate them based on some simple process (math in neural networks, chemicals and bioelectricity in the brain). It's a huge simplification, but the principle holds.

"If you believe this—if you allow yourself to believe this—then there are a lot of interesting implications," Sutskever said. "The main implication is, if you have a very large artificial neural network, it should do a lot of things. Especially, if the human brain can do something, then a large artificial neural network should be able to do something similar."

"If you take this seriously enough, everything falls into place," he said. "Most of my work can be explained by this."

When we talk about the brain, I wanted to ask Sutskever about a post he made on the X platform. The abstract of Sutskever's post reads like a volume of aphorisms: "If you place intelligence above all other human qualities, you will live very poorly"; "Empathy in life and business is underestimated"; "Perfection has ruined many perfect things."

In February 2022, he posted, "Today's large neural networks may have a slight consciousness" (Murray Shanahan, chief scientist at Google DeepMind, professor at Imperial College London, and scientific advisor for the movie "Ex Machina," responded, "…in the same sense, perhaps a large field of wheat with a hint of pasta").

When I brought this up, Sutskever laughed. Was he joking? He wasn't. "Are you familiar with the concept of a Boltzmann brain?" he asked.

He was referring to a quantum mechanical thought experiment named after the 19th-century physicist Ludwig Boltzmann, in which random thermodynamic fluctuations in the universe lead to the appearance and disappearance of brains.

"I feel like these language models are a bit like Boltzmann brains now," Sutskever said. "You start talking to it, chat for a while; then you're done, and the brain goes—" he made a disappearing motion with his hand. Poof—goodbye, brain.

So, you're saying, when the neural network is active—when it's firing—there's something there? I asked.

"I think it might be," he said. "I'm not sure, but it's a possibility that's hard to argue against. But who knows what's going on, right?"

AI, But Not as We Know It

While others are struggling to make machines that can match human intelligence, Sutskever is preparing for machines that can surpass us. He calls this phenomenon superintelligence: "They will look at things more deeply. They will see things we can't see."

Again, I find it hard to understand what this really means. Human intelligence is our benchmark for judging intelligence. What does it mean for intelligence to be smarter than human? "We saw a very narrow example of superintelligence in AlphaGo," he said. In 2016, DeepMind's board game AI "AlphaGo" defeated one of the world's best Go players, Lee Sedol, 4-1 in five matches. "It figured out how to play Go in a way that's different from the way humans have collectively developed for thousands of years," Sutskever said. "It came up with new ideas."

Sutskever mentioned AlphaGo's famous 37th move. In the second match against Lee Sedol, the AI's move left commentators puzzled. They thought AlphaGo had messed up. In fact, it made a winning move never seen in the history of Go. "Imagine that level of insight, but covering everything," Sutskever said.

It's this thinking that led Sutskever to make the biggest shift in his career. He and his OpenAI scientist colleague Jan Leike founded a team focused on what they call "superalignment." Alignment is a jargon term, meaning to make AI models do what you want them to do, and nothing else. Superalignment is the term OpenAI is applying to superintelligence.

The goal is to develop a set of fail-safe procedures for building and controlling this future technology. OpenAI says it will allocate one-fifth of its massive computing resources to solve this problem and solve it within four years.