The Startup Training Robots With Video Games

A visit to General Intuition, which just raised $320M at a $2.3B valuation and sees gaming data as a faster path to physical world impact.

General Intuition’s Pim de Witte, without his Dutch gear. Credit: General Intuition

In a New York office earlier this week, General Intuition CEO Pim de Witte receives me wearing a Netherlands jersey — it’s the World Cup, after all — to demonstrate his company’s pet robot.

Known to its data analyst handler as Clippord, the robot’s about the size of a big golden retriever or husky, and utilitarian looking. De Witte asks me not to take photos on aesthetic grounds.

Trust our word for it that this robodog eventually clomps around the office successfully, with just the occasional bump into the side of a column, although the team doesn’t trust it to make it past a glass meeting room wall just yet.

If you’ve seen robotics demonstrations before — quadruped clankers are a fixture at tech conferences these days — it’s nothing state-of-the-art.

But it’s how General Intuition, which announced a $320 million funding round led by Khosla Ventures yesterday at a $2.3 billion valuation, is “zero-shotting” Clippord’s basic navigation that has de Witte fired up.

“‘I’ve never seen a clown walk on the road. And when a clown walks on the road, it’s like, what the f***?’”

Clippord has no maps or specific data on the office layout as a reference; it’s approximating its own size, and how to handle obstacles, entirely from a corpus of video game usage data, topped off with 8 minutes of real-world data collected on the sidewalk below.

Only one sensor, a camera in the front where its eyes would be, is processing what it sees as pixels, sending that data back to a data center, predicting a next action, and returning that command 10 times per second, faster than a human’s reaction time.

Why does that matter?

According to de Witte: robotics companies are spending way too much time and money on their own training.

“Everybody is collecting way too much,” de Witte tells Upstarts. Better to turn to Fortnite.

The rest of this conversation is for paid Upstarts subscribers. Take 15% off an annual subscription here, for a limited time.