Drivers Snap Street-Level Imagery with Capture Projects from Mapillary

Mapillary, the street-level imagery platform that uses computer vision to work with and manage the world’s maps, this week announced a new tool that allows cities and mapmakers to take control over their map data collection in order to build better maps at scale. Capture Projects consists of a combination of a web and a mobile app, named Mapillary for Drivers, that gives cities and mapmakers the ability to task and manage an unlimited number of drivers to capture street-level imagery at any given point, increasing how quickly map data can be generated just using cameras.

According to company materials, map data is increasingly in demand as rapid advances in autonomous driving and urban development is escalating the need for a new way of mapping the world. Research shows that the global geospatial analytics market is set to surge by 357% between 2017 and 2027, growing from $38.6 billion to $174.6 billion as fresh map data is becoming increasingly important for industries ranging from transportation and automotive to GIS and maps.

In January, Mapillary announced another 42 object classes to the set of map features that we extract and place on the map. This means fresh data at scale for a lot of use cases across various industries, such as:

• Crosswalks, street lights, and benches for pedestrian mobility;
• Bike racks and bicycle traffic lights for cycling;
• Lane markings, traffic lights, and parking meters for transportation;
• Manholes, utility poles, and trash cans for public works;
• Traffic cones and construction barriers for road maintenance projects.

Jan Erik Solem, CEO and co-founder of Mapillary, spoke with GISCafe Voice about Capture Projects as well as the new features Mapillary has placed on the map.

How do you define “computer vision?”

Computer vision is a form of artificial intelligence that allows machines to see and, in the best case, understand their surroundings. Mapillary has built computer vision technology that detects objects like traffic signs, fire hydrants, utility poles, and many other things in street-level imagery, and positions and places these onto the global map. Computers are able to do the detection and positioning at a much faster pace and a more detailed level than any human could accomplish.

When you say “anyone can access map data,” do you mean anyone who is a mapmaker, or anyone who is a user of maps?

Both. Anyone can go on the Mapillary website and view the imagery and data, at no cost. Likewise, the data is available for anyone to purchase and use to improve their own maps. All imagery on Mapillary is available under CC-BY-SA and the imagery and derived map data is freely available for editing OpenStreetMap.

Mapillary is the only player to open up map data in this way, with clear commercial terms and usage rights. We know that this is a model that works—Mapillary now has 450 million images available on the platform, and our customers and users range from mapping companies like HERE and Mapbox, to automotive companies like Toyota and AID (Audi’s self-driving arm), and cities ranging from capitals like Amsterdam to small towns like Clovis in New Mexico, USA.

What is the learning curve for those generating data for Capture Projects?

Our goal is to make map data easy for everyone to access and we have designed Capture Projects to be as user-friendly as possible for both project administrators and drivers. Once the drivers upload the tasked imagery, there is no need for further action to generate data from the imagery—the computer vision technology does the rest. We will work with users to help them make the most of this tool, and welcome feedback to help us improve its features.

Was Mapillary for Drivers also available before Capture Projects? How do they relate to one another?

No. Mapillary for Drivers is a part of Capture Projects. Once project administrators have assigned tasks to drivers, Mapillary for Drivers is the mobile app where drivers can see what needs to be mapped in their task area.

How does Mapillary integrate or work in concert with solutions from other geospatial companies?

Mapillary integrations are available for Esri’s ArcGIS as a suite of tools that allow users to manage, build, and explore their geospatial data with ease. Integrations also exist for QGIS, OpenStreetMap, and HERE Map Creator, allowing community members to contribute details to those maps using Mapillary imagery.

Mapillary is an independent and neutral provider of street-level imagery and map data—our focus is on providing the best and most scalable means of updating maps all around the world.

What do HERE and MapBox contribute?

 We work closely with both HERE and Mapbox. HERE maps are in 100 million cars globally, they power Baidu’s maps (Baidu is China’s largest search engine), and a number of tech giants work with HERE for their map needs, including Amazon, Facebook, and Garmin. Mapillary helps HERE to fill out the gaps where they have insufficient map coverage themselves, and Mapillary data allows HERE to quickly assess and validate the data they already have. HERE also has a phenomenal global community of map editors, who have contributed a great deal of imagery to Mapillary.

Mapbox powers the maps of companies like Snap, The Weather Channel, and CNN. Mapbox’s map and navigation products are based on OpenStreetMap, and they use Mapillary to validate and enrich OpenStreetMap as a means of improving their own maps. When OpenStreetMap is better and more detailed, everybody wins. OpenStreetMap powers a number of maps around the world.

How do you sift through the contributed imagery that ultimately gets processed into images and fresh map data?

This is done through a completely automated process. All images contributed to Mapillary are automatically processed using computer vision. Before the images are published on the platform, sensitive information like faces and license plates are automatically blurred and the computer vision algorithms detect map data in the imagery. The data from the imagery is published on the platform at the same time as the images themselves.

Tell me about the ability to detect CCTV cameras?

Our computer vision technology is able to detect CCTV cameras in imagery though, and CCTVs are available as point features via our data subscription service. When retrieved, the estimated GPS position of each point is made available in a standard geospatial format (shapefile or GeoJSON).

Is the Capture Projects tool customized in house to capture certain objects, such as utility poles or traffic signs, as needed by the customers?

Capture Projects allows organizations to designate the area in which they require map coverage. When that imagery is uploaded to the Mapillary platform, it goes directly to our processing pipeline where we apply computer vision to automatically detect and position 43 different object classes, ranging from traffic signs and fire hydrants to benches and utility poles. All of those map features are then available within hours on the Mapillary platform. The strength of our approach is that anyone can take the pictures used to generate map data, and our system processes them—as our technology improves, you get even more out of the same images you once collected. Organizations no longer need to worry about missing data or having to go back out into the field if they need more data later on.

How is pricing set up?

There is no additional cost to use Capture Projects. However, a commercial license is needed to download the map data for commercial purposes. For these, we offer yearly subscriptions to map data based on the length of the road network requested, beginning at $10,000 per year for up to 1,000 km. By comparison, we know that contractors normally charge cities $4-6 per traffic sign when doing inventories—something that quickly racks up costs to tens thousands of dollars.

Mapillary imagery and data are available both under open licenses and under commercial licenses.  Commercial licenses are offered under a subscription model. These new map features will be a part of existing subscription plans, with customers gaining access to the new object classes in addition to traffic sign data. The map features are also available and free to OpenStreetMap editors.

Are the new features from Mapillary in an existing database and how do users add to the map features?

We currently have over 440 million images on the Mapillary platform and have extracted over 186 million map features from those images using computer vision. Our computer vision technology for detecting map features is automatically applied to every image that is uploaded to the Mapillary platform. This means that anyone can go out and capture images with their smartphone or another camera and have the data from those images put on the map almost instantly. So yes, the new features are derived from images in the existing database.

How do these features contribute to the success of autonomous cars?

AI-generated map features is an important step towards HD maps. For autonomous vehicles to safely navigate our streets, they require extremely detailed maps containing all of the information that these cars need to be aware of. In our fast-changing world, it is difficult to keep up with dynamic location data. Allowing computers to do this detection offers the speed and a level of detail that could never be obtained manually. With map features, Mapillary is now able to detect and localize 42 different kinds of object classes in images and literally put them on the map by using computer vision and GPS coordinates.

How do you manage and vet crowdsourced data coming into the program?

We have fully automated processes in place. For instance, when it comes to privacy, we’ve trained our algorithms to anonymize sensitive information like faces and license plates through blurring. The imagery and data are also crowd moderated. Anyone with a registered account can delete images, blur images, or alert us to an issue that requires handling.

How do users get started?

It’s a very easy process. Download the Mapillary app, sign up for a Mapillary account, and you’re ready to start capturing imagery and have it analyzed with computer vision technology. Once you’re signed up you can also create an Organization in Mapillary for Organizations, which is a workspace for teams to use and create data and street-level imagery together in their workflows. If you are interested in what type of data is already available in your area, you can explore the new map features directly in our web app.

How reliable do you think cars that can see technology will be?

There are already lower levels of autonomous vehicles on the streets today, but it will be several years before the technology is reliable enough for fully autonomous vehicles deployed broadly. At the core of scaling this technological development to broad deployment are HD maps with good coverage and data that is diverse enough to train autonomous systems to operate in all kinds of environments. Autonomous vehicles need to be trained on datasets that are not only rich in detail but also geographically and situationally diverse. That’s why we have developed the Mapillary Vistas Dataset—the world’s most diverse dataset for training autonomous driving systems. It is diverse because it is based on images from our global Mapillary community.

Tags: cloud, crowdsourcing, data, ESRI, geospatial, GIS, Google Maps, imagery, Infrastructure, intelligence, location, Mapillary, mapping, mobile, satellite imagery

Categories: analytics, Big Data, citizen science, cloud, crowd source, developers, drones, emergency response, Esri, field GIS, geospatial, GIS, government, lidar, mapping, photogrammetry, remote sensing, sensors, smartphones, spatial data

This entry was posted on Friday, February 15th, 2019 at 11:09 am. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.