GISWeekly examines select top news each week, picks out worthwhile reading from around the web, and special interest items you might not find elsewhere. This issue will feature Industry News, Top News of the Week, Acquisitions/Agreements/Alliances, Announcements, New Products, Around the Web, and Events Calendar.
GISWeekly welcomes letters and feedback from readers, so let us know what you think. Send your comments to me at Email Contact
Susan Smith, Managing Editor
Netezza Spatial Blazes into the GIS Market
By Susan Smith
Ever heard of Netezza? Last week at the Netezza User Conference in Florida, the company Netezza, known for its data warehouse appliance, announced Netezza Spatial, a software extension to their appliance which can deliver location-based information at the same blistering rates its business intelligence (BI) customers are accustomed to.
Click to Enlarge ]
A global re-insurance company uses location intelligence to enable clients to assess the risk associated with their insurance portfolios and identify sources of risk accumulation across those portfolios.
Data warehouse appliances are a somewhat new concept to the GIS market. What they do is the following: they integrate database software, storage and hardware that’s optimized for analytics, delivering performance that is 10, 20 to 100 times faster than general purpose databases at a price point that tends to be less than 50% for acquisition cost and administrative costs, according to Jonathan Shepherd, general manager of location-based solutions with Netezza. “With the announcement of Netezza Spatial, we’re extending the data types that are supported within the Netezza database to include location-based information, storing spatial data and providing the operators to query that data,” said Shepherd.
For about five years, Netezza has been selling to the business intelligence (BI) market. Spurred by customer demand, Netezza began to seek out a way to incorporate location-based data into their database. Employing Boston-based Intelligent Integration Systems, Inc. (IISi) was the first step. A year ago, in answer to a challenge set forth by Netezza, IISi created a geospatial toolkit for Netezza’s built-in database to manage data encoded using Open Geospatial Consortium (OGC) standards. At that point, Netezza acquired that technology from IISi.
Now, with Netezza Spatial, “We’re delivering to the GIS market location intelligence with the same value proposition we deliver to business intelligence, which is significant performance improvements for spatial analytics with low cost of ownership,” Shepherd said.
Maintaining all the data in one place without the requirement to index for performance is how Netezza Spatial lowers the cost of ownership. “Our customers routinely run databases that are larger than any other databases in the market, that’s tens of terabytes up to hundreds of terabytes with only one database,” said Shepherd.
Peter Batty, of Spatial Networking, has been consulting with Netezza on this spatial project for the past few months. Batty has a good idea of how the administrative costs of Netezza compare with those of other commercial databases.
“One of the interesting things about Netezza is there are really two key value propositions: one is a very big performance improvement and the second is lower design and administration requirements, and a big part of those is the way it’s architected,” said Batty. “It’s very parallelized, so basically they don’t use indexes at all. In a traditional complex analytical application, you have to design your item indexes and tune the system, etc. The great majority of that goes away, not completely in the Netezza system but certainly the part about indexes. Compared to an Oracle Spatial implementation, there’s a lot less work involved in the design and administration.”
Enter Safe Software, which has entered into a partnership with Netezza. “We’re at the beginning of the Netezza Spatial food chain,” noted Safe president Don Murray. “When people get a Netezza Spatial appliance they’re going to need to get data into it. As you know, that’s one of our areas of expertise, being able to take our 220 some odd spatial data sources and directly load them into Netezza Spatial. The other way we provide value to Netezza Spatial users is by being able to connect Netezza Spatial to a third party GIS like Autodesk, ESRI and others, so they can get the data in but then they can see the data in Netezza or get the data out of Netezza or get the results out of Netezza in a particular format that then they can use in subsequent systems.”
Another value from Safe Software is their plug-ins for Informatic, Power Center and IBM WebSphere DataStage. Informatic or Power Center users who are also using Netezza with the Safe Software plug-in into Power Center are able to directly leverage all the spatial capabilities of Netezza within the Power Center or IBM DataStage Environment as well.
PitneyBowes’ MapInfo location intelligence division, Idea Integration, Intergraph, and SRC are working with Netezza Spatial. In addition, thincSoft has integrated its thincVIew application with Netezza Spatial.
Batty said there are two key ways Netezza gets its high performance: one is its being very highly parallelized, so in a small Netezza system there are around 50 parallel processing units and a large one up to about 1,000. “To parallelize SQL queries in particular and spatial queries in general is a very complex action.” And in the second area, they have very specialized hardware and in particular, specialized disk readers that use a device called the Field Programmable Gate Array (FPGA). This takes SQL functionality down into hardware so they can eliminate rows and columns that the user is not interested in, which eliminates a big bottleneck during very large complex queries on large datasets.
“In other systems all that data has to be run off a disk before you can figure out if it’s of interest or not. This makes it hard for people to copy, because developing that specialized hardware is a much more complex effort than just developing a software solution,” he pointed out.
Would this supplant the need for Oracle Spatial and other like products?
Shepherd said, “For analytics, yes. Netezza sees customers moving to Netezza Spatial from two directions – one is we have 200 plus customers today using Netezza for BI and our mantra has been “all the data all the time.” We see customers expressing interest in this, from financial services customers to online marketers, merchandisers, telecommunications, retailers, government customers, so it’s across a wide range of vertical markets. Many of those folks are looking to augment their data with spatial data. There are other customers of ours who have spatial data in Oracle or DB2 that are operational systems who don’t get the performance they would like to run high end analytics. We believe those customers will be very interested in this announcement.”
Batty said that if you have an operational system in Oracle Spatial where you are simply querying a single record, then you won’t need Netezza. “The strength of Netezza is large complex queries. If someone is focused on analytics, they might replace their Oracle Spatial with Netezza Spatial completely but in a lot of scenarios, they have an operational system which would stay in Oracle Spatial or Microsoft SQL Server, and they can use Safe and other capabilities to export and import from those systems.”
The one public customer Netezza announced is New York-based Guy Carpenter, a re-insurance company who is using Safe FME and Netezza.
Guy Carpenter looked at the current hurricanes to see what exposure their customers (insurance companies) have to the oncoming hurricanes, and updated constantly as they received new live weather information. “Guy Carpenter was able to get performance improvements on the following type of queries: they had a bunch of impact polygons and point data that were insurance policies and they were able to take that down from about 45 minutes to five seconds using the Netezza architecture,” explained Murray. “If you start giving turnaround like that to users, they can start to run scenarios very quickly, whereas if it takes you 45 minutes to get a solution, then obviously you’re not going to be able to do that many times. If you can get down to 5 seconds for different scenarios then you can start to run 'what-ifs.' With some interesting analytics a whole raft of things become possible, especially when you start combining this with live data base feeds. There are lots of live database and web feeds becoming available all the time.