Fast-forward three years. His little mapping visualization startup Keyhole was acquired by a newly public Google for $35 million in stock options. “Other search engines thought of mapping as something you print out and take with you in your car–like Mapquest,” says Hanke at the American Geographical Society’s Geography 2050 conference in New York. “[Google] bought into this vision that we had of assembling all the world’s geographical information.”
According to them–two people who have redefined what it means to make a map–the next generation of maps are semantic. That’s a fancy way of saying that maps will have a complex, dynamic understanding of the world around them. Why is this important? Self-driving cars and augmented reality.

Self-driving cars are going to need really, really good maps. But the kinds of maps we have today–like Google maps–aren’t accurate enough. While autonomous vehicles will rely on their cameras and sensors to create a picture of the world, they also need to know how to make sense of that information. That’s where a semantic map comes in–a map that continues to learn about the physical world and refine its predictions of what objects are and how they will act through huge amounts of data. Without semantic maps, self-driving cars won’t ever be able to intelligently move through the world–at least not without crashing into something–a development that will arrive when self-driving cars do (between 5 and 30 years from now, depending on whom you ask).

Today, to build a map that dynamically reflects and understands the world, you need countless sensors recording it so you can constantly update the digital cartography–and so machine learning algorithms can look for patterns in all the data that’s generated. That requires capturing and storing lots of data–a estimated gigabyte per second for a self-driving car. Then, you need to have ubiquitous and powerful enough mobile computing to capture that data, make sense of it on the spot, and render it in a way that’s intelligible and useful. “You need that data to create the maps that are needed for AVs–that will make all the work we’ve done in mapping to date look small in comparison,” Hanke says.

These are the maps that self-driving cars will require. A car’s cameras and sensors can instantly pick up on objects around it, but it also needs to know where it’s going–and that system isn’t going to work unless there’s significantly higher fidelity in the maps these cars rely on. For instance, Google StreetView cars take an image every thee feet or so; but three feet can mean being in the wrong lane or missing a stop sign entirely. A new GPS that can map down to the foot is coming to smartphones in 2018–but autonomous vehicles may need even higher resolution.
Companies like TomTom, Here, and Carmera are already working on this problem. It will require machine learning algorithms, processing and storing vast amounts of data, and a network of sensors that can frequently update the map and power the AI models–a network that self-driving cars will readily supply. In essence, it’s the problem of creating a three-dimensional, live, super-accurate map of our constantly changing messy world. No biggie.

The idea of mapping our indoor spaces is rife with problems. First up: Who owns the data? Perhaps you own the mapping data of your home, but what about in commercial or institutional spaces? As McClendon noted, that’s dangerously close to a surveillance state. He proposed that as a rule, only certain data is uploaded to the cloud, and most of the mapping data stays on your device. “Then it’s not personal pictures, it’s the geometry of the world around you,” McClendon says. “That reduces the privacy risk, certainly for glasses.”

The vision Hanke and McClendon paint might fill you with inspiration or dread. After all, the closer we get to the high-fidelity semantic maps they’re talking about, the closer we get to total surveillance. But ultimately, that the push for self-driving cars and augmented reality relies on making that real-time, high-resolution, all-knowingly perfect map.