We’ll Never Win! Google’s AI can Beat Atari Games

Last month, a team of Google scientists unveiled a deep-learning algorithm called DQN—a general purpose, self-educating computer program that is killing it at Atari.

(Are you thinking what I’m thinking? Thought so.)

DQN is running on a high-end desktop computer with two notable advances:

• Q-learning is a positive-reinforcement learning method by which the program constantly tries to achieve the best reward (the “Q” factor). In the case of Atari, Q = a high game score.
• An artificial neural network, inspired by the human brain, tells DQN what information is and is not important to reaching its goal.

Taken together, those two advances allow DQN to absorb information and respond accordingly.

Volodymyr Mnih heads the Google team that has given birth to DQN, and while they shy from labelling it “artificial intelligence,” Mnih has said that this algorithm could be the first step to real AI.

Google and AI

If you’ve ever clicked in a Google search bar, and have seen your query suggested before you could finish typing, it should come as no surprise that Google is very deeply invested in artificial intelligence research.

First came the DeepMind acquisition at the start of 2014, and the almost immediate development of an ethics board to govern its use and preemptively save the human race. Last September Google welcomed a quantum computing team from UCSB, and a month later they “acqui-hired” two teams of AI researchers from Oxford. To date, there are 406 publications on Google’s research site that are labelled Artificial Intelligence and Machine Learning.

It’s Learning …

They are definitely making advancements. It’s amazing that a general-purpose algorithm, with no prior knowledge of—or programming for—the Space Invader rules of play, can learn to master the game. Apparently, it’s also very good at Breakout, Boxing, and Star Gunner, along with 49 other Atari 2600 games:

• In 43 of the 49 games tested, DQN achieved better scores than previous machine learning algorithms.
• In half of the games tested, DQN achieved at least 75% of the skill level of a human game player.

(Suddenly feeling the need to brush up your skills? You can play a few Atari games online at IGN. You’re welcome.)

While more complicated games are still beyond DQN’s scope, the fact remains that Google’s algorithms are actively learning.

In certain games, DQN even came up with surprisingly far-sighted strategies that allowed it to achieve the maximum attainable score. For example, in Breakout, it learned to first dig a tunnel at one end of the brick wall so the ball could bounce around the back and knock out bricks from behind. – Dharshan Kumaran and Demis Hassabis, Google Research Blog

Next, developers want to advance the program so it can learn to master 3D games, which Demis Hassabis believes could happen within the next five years.

AI Search Engines?

The technology is not being primarily developed for search engines.

In the future I think what we’re most psyched about is using this type of AI to help do science and help with things like climate science, disease, all these areas which have huge complexity in terms of the data that the human scientists are having to deal with. – Demis Hassabis

But it would also be naive to think that Google will not use every algorithm advancement at its fingertips to improve a user’s search experience.

“This system [DQG] that we’ve developed is just a demonstration of the power of the general algorithms.” – Koray Kavukcuoglu

Just a demonstration? The possibilities are just starting to unfold.

It’s too soon to tell how this may (or may not) impact SEO. I can see it now: “Google Announces Next Update: The Self-Aware Algorithm.”

Nate Dame

CEO and Founder

Nate is the founder and CEO of Profound Strategy, a results-oriented SEO consultancy trusted by forward-thinking companies, including a few of the world's largest B2B and technology brands. Profound Strategy builds holistic SEO strategies, supports internal teams, and offers full-service execution to create an organic search presence that generates significant revenue.

LinkedIn Twitter

What's Next?

Profound Strategy is on a mission to help growth-minded marketers turn SEO back into a source of predictable, reliable, scalable business results.

Start winning in organic search and turn SEO into your most efficient marketing channel. Subscribe to updates and join the 6,000+ marketing executives and founders that are changing the way they do SEO:

And dig deeper with some of our best content, such as The CMO’s Guide to Modern SEO, Technical SEO: A Decision Maker’s Guide, and A Modern Framework for SEO Work that Matters.