U.S Department of Commerce’s NTIA Interactive Digital Broadband Map Released

Laura McNulty, National Government Sciences Manager from Esri manages Esri the National Health, Government and Sciences Team, that supports NTIA, FDC, and many other science and health based federal agencies.

Thomas Gibbs, National Government Sciences at Esri, is a team lead on the Department of Commerce with responsibilities from an account perspective over NTIA, a role he has held since the beginning of the project and program.

The U.S. Department of Commerce’s National Telecommunications and Information Administration (NTIA) released this week a new publicly available digital map that displays key indicators of broadband needs across the country based on Esri technology. This is the first interactive, public map that allows users to explore different datasets about where people do not have quality Internet access.

According to press materials, the public “Indicators of Broadband Need” tool released puts on one map, for the first time, data from both public and private sources. It contains data aggregated at the county, census tract, and census block level from the U.S. Census Bureau, the Federal Communications Commission (FCC), M-Lab, Ookla and Microsoft. Speed-test data provided by M-Lab and Ookla help to illustrate the reality that communities experience when going online, with many parts of the country reporting speeds that fall below the FCC’s current benchmark for fixed broadband service of 25 Mbps download, 3 Mbps upload. This is the first map that allows users to graphically compare and contrast these different data sources.

LM:  This announcement was triggered off the White House announcement of the Broadband map which is part of a broader program.

TG:  This effort began in 2018 under the Trump Administration when they wanted a better understanding of the gaps where broadband was or wasn’t available. A lot industry people looked toward federal communications. There’s a form called Form477 that they’re responsible for and they work primarily with the telecommunication providers. It’s a federal form that provides the basic information that FDC has been using since the early 2000s to determine where coverage is and where it doesn’t exist. There have been a lot of pundits from policy people to actually telco industry spokespeople saying that’s just not good enough. The detail of the map is not there, a lot of telco providers are doing this on a voluntary basis, so in 2018 Congress mandated the Department of Commerce under cabinet level to develop a better mechanism for filling in the gaps where the data doesn’t exist and come up with a better methodology.

NTIA, the National Telecommunications and Information Administration, operates at a policy level and cabinet level. They are the ones that were doing the funding of the grant programs that were out there supported by Department of Commerce. They have a collaborative spirit with FCC but as you can imagine, sometimes  they have their own missions and sometimes they have to follow those. FCC, being regulatory, kind of like EPA, are the sheriffs of the town, and they are also regulatory over the telecommunications path and inform broadband. But from a policy perspective, that’s where the Department of Commerce comes in and NTIA is underneath that. They formulated an office called the National Broadband Availability Map, called MBAM. The initial goal was to stand up a pilot number of states, about 8 states, just to get the feel of not only the FCC data but also the state data as well as commercial data. And that was the difference in NITA’s approach — aggregating all the different datasets that are either out there or commercially available leveraging GIS technology.

The ESRI platform is there to be able to put the pieces together and it looks like a big puzzle for the U.S. There are things that you know and that you think you know and other things you don’t know until you compare it with other data.

So that’s what sets NTIA apart is being an analytical platform that they can do better analysis beyond just a single data set. You just can’t make policy from that.

NTIA is partnered with the U.S. Census Bureau so population data and demographic data are part of the map. They partnered with USDA, the Department of Agriculture, Economic Development Administration and they have written individual Memorandums of Understanding with the different states. Some states are going to be sharing data they don’t want other people to know, commercial data won’t want to be shared publicly but they’ve written special joint public agreements, so it’s really a system now that’s heavily authenticated just because of some of the proprietary data sets. But just recently with the White House announcement, they were able to share some of the public data and that gave a great way to show the power of this system when you start looking at it from a national perspective.

• In helping in the analytical data processing NTIA utilized Artificial Intelligence in deploying the Esri’s Big Data Toolkit (https://www.esri.com/en-us/arcgis/products/spatial-analytics-data-science/capabilities/real-time-big-data-analytics) to process the large datasets:  (NTIA’s – National Broadband Availability Mapping Program (NBAM) supports over 252 data services and layers across 45 data partners having processed over several TeraBytes of data (4TB currently).  NTIA is one of the first federal agencies to utilize this unique geospatial AI Big Data Toolkit to analyze such large data sets derived from the program. Eg.  Ookla SpeedTest data massive files (1-2TB) that provide the necessary granularity for more detailed broadband analysis of underserved and unserved communities.
• NTIA NBAM is hosted in a secure Esri cloud infrastructure leveraging both the Esri ArcGIS Online Mapping Platform and the Esri Managed Cloud Services FedRAMP environment.   NBAM efforts were non-stop and seamless enhancing the systems and application tools thorough the pandemic in early 2020 and 2021 due to remote access through the secured cloud infrastructure.

LM: The bottom line is we’ve been trying to help NTIA find a way to uncover where the real broadband deficits are across the country,  and starting a small pilot as Thomas mentioned and growing that out, and looking at how you bring all this information together to tell the story.

To really start to visualize is where the beauty of GIS comes in to tell a very complex story in a way that is easy to understand for just about anyone, making data that exists in a normal form, shaping it better as usable and accessible by a broad range of people including citizens. If I’m a researcher I can analyze it but if I’m a citizen I can go and look and see what kind of services are available in my area so it can come down to the individual households as well. That’s the only thing I would add to Thomas’s mechanics of how we do this and the technology that was used. Bottom line, Esri is bringing leadership from a broadband perspective and then applying that technical expertise to bring value to rural and non-rural communities and see if they can access broadband and if the answer is no, then they can raise their hand and say we’re here and need some help.

Red areas of the map are where there is going to be a need, and you can drill down to a census block level to look at it. And the other distinction between this is they’re not only looking at unserved areas but underserved. What they mean by that is taking speed test data, and just because you have been told you have access to broadband in your area, is it adequate or not? Your neighbor may be better than you, so I won’t say it’s down to a household level just because of privacy but there are some special aggregations they are doing behind the scenes to get a little more granularity than you can just do in GIS.  You’re not going to make an assumption. The FCC is mandated that one house in a census block has it then it’s served. There are a lot of houses and households that can be in a census block, especially out in the rural areas, so in here FCC needs to get down, they want better data. Everyone would like to get more granular data but when you start piecing these pieces together from a policy perspective to understand then better decisions can be made.

Certainly there are gaps in areas, the map shows on broad scale, internet availability.

There are a lot of federal dollars going out there, even with the new infrastructure bill there will be close to a million dollars are going to be a part of that, so you want to make sure those funds are being directed in the right way.

AECCafe Voice: Each place that doesn’t have broadband is a place that can’t really do business very well.

LM: This is a huge factor to look at long term. Tele help, education, economic development.

TG: Uncovering the digital divide, the geospatial part visually there are gaps that are so evident. Being able to have a tool as an individual to contest what a vendor is saying about their ability to service a certain area, so everyone is participatory and intuitive. Someone who can’t read 4477 because it’s very complex can look at a map and say that’s my house, it says I have coverage but I don’t. Where the highest need is for dollars, where there are limited resources is the first place I go.

LM: The White House map was the first public exposure of this particular data but Esri has made data available through the Living Atlas in cooperation with NTIA. The application itself, the web maps that formulate that are now available for Esri community to take advantage of, and they might want to add their data on top of that. They’re doing partnerships with 36 states with the goal of partnering with all of them, constantly look at datasets to finalize that puzzle piece. This is the business model, it’s not just an application, it’s a program. They have the full power of our analytical program behind the scenes to do reports, they have don’t that for some of the have nots. We can make it available to them, communicate and work with state organizations s to provide this information to get out to individual communities.

AECCafe Voice: How frequently would it update?

TG: We currently have an almost monthly update, on a monthly cycle, the major updates from USC we would update at that same time. Commercial vendors once they do a data refresh, at least every month there’s a data refresh, not of the whole platform. Every six months almost all data has been refreshed.

AECCafe Voice: It’s available to mappers, but who has the right to change the map?

LM: For the Living Atlas, the authoritative data is under the purview of those who shared it. NTIA had registered the data and there’s a check-in check-out process within the Living Atlas. NTIA owns data, and if there are questions about the data they have a contact name in there. It’s almost a metadata profile with a version of the data and then a data stewardship where they can be able to mitigate it.

The Census Bureau links back to USDA data but all that is in a registered format so you would know if there are problems with it or what versions you’re using.

This gives you a way to chase it down. We have our own curation process, on the federal side, and we make sure federal data is authoritative in nature. It needs to be updated, and over time it improves.

Tags: ArcGIS, broadband, data, ESRI, geospatial, GIS, imagery, Infrastructure, location, mapping, maps, mobile mapping

Categories: ArcGIS, ArcGIS Online, cloud, data, geospatial, GIS, government, mapping, mobile

This entry was posted on Friday, July 2nd, 2021 at 5:40 pm. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.