Open Data, Big Data……but what about Spatial Data?

A world of two and half billion people connected with various devices is creating an explosion of sensor and behavioral data.  For most businesses in Ireland the most challenging aspect of any decision making process has been finding, validating and massaging relevant data by normalising, analysing and harmonising it in a manner that helps solve business problems.  In the last two years there has been a heightened awareness of both the challenges and benefits of harnessing business-relevant information that is inherent in the vast array of open and big data that is available to us.  It is this explosion of data from sources such as sensor networks, crowd sourcing and social media channels that ensures that ‘data’ is king when it comes to the discipline of business analytics.   

Businesses now, more than ever before, need access to authoritative data content and relevant information, whether that content is contained in open data sources, big data sources or, in the case of a fast developing area of business analytics called location analytics, ‘spatial’ data sources.  Up to now, spatial data (to you and me this is simply data that can be viewed on a map) fell into that very specialist area of Geographical Information Systems (GIS).   Now, spatial data is pretty much available and accessible to anyone with an internet connection whether that is through a PC, laptop, tablet or smartphone.  Companies such as Esri, Google and Microsoft are providing large volumes of rich ready-to-use spatial data content, including high quality aerial imagery, street level data, census data, environmental data, health data, economic data and other business-relevant data such as demographic profiles, deprivation indexes and consumer spending. 

Location analytics makes use of this array of spatial data by combining it with your own business data in a way that will reveal trends and patterns that otherwise wouldn’t be visible with traditional business analytics tools and processes.  Location has always been a part of business analytics, often helping answer questions such as, where should I locate my next store or branch? Where are my existing customers? and where are my target prospects? However this explosion of location based data collected through smartphones and other devices has meant that it is now fast becoming a source of competitive advantage for the business sector. 

Location analytics is about dynamic, interactive mapping; sophisticated spatial analytics and rich complementary spatial data. Making use of this resource is now no longer the domain of the GIS specialist.  For example, Esri is now working with major technology suppliers such as Microsoft, IBM and SAP to integrate mapping with common Business Intelligence (BI) technologies.   In this way Microsoft Office users can now harness the benefits of location analytics and spatial data inside Excel and PowerPoint by using products such as Esri Maps for Office (similarly for users of MS Dynamics and MS Sharepoint); Cognos (IBM) users can benefit from location based analysis through Esri Maps for Cognos and later this year Esri Maps for SAP will expose the power of location analytics to SAP users.  This means that the powerful business benefits associated with the efficient and effective use of spatial data is now accessible to traditional business analysts from within traditional BI tools.  

As we hear more and more from the open data and big data movements it is easy to get caught up in the hype around both of these data sources. But let’s not forget about spatial data, which in itself is both open and big in the context of its impact and value proposition to the business community.

Paul Synnott (Article for Business Ireland & Public Sector Times, Q2, 2013)

Image

Esri Ireland Guest Blog “Geographic Data Capture using Unmanned Aerial Vehicles” – By Paudie Barry, MD, Baseline Surveys

Welcome to our very first guest blog kindly submitted by Paudie Barry, Managing Director of Baseline Surveys Ltd.

Paudie writes about the different types of unmanned aerial systems and their many potential applications, from corridor mapping to search and rescue missions.

Unmanned aerial vehicles, otherwise known as drones, have been put to extensive use for military purposes over the last 10 years. Now this technology has filtered down into civilian use, as drones can be used for aerial mapping and data collection to great effect. My blog will give you an introduction & insight to a variety of drone types, their applications and the accuracy of data that can be derived from drone aerial ortho-photography.

TYPES OF DRONE
There are a multitude of mapping and inspection drones, weighing in at under 20kgs, on the market, but they can essentially be categorised as two main types: fixed-wing drones and rotary-wing drones. A fixed-wing drone needs a large area for take off and landing, but is far more efficient at flying. As a result, it can survey far greater areas and is much more suitable for surveying areas of over 10 acres.
Rotary wing drones are far more suitable for working in confined or urban areas. It is very suitable for surveying intricately detailed sites up to 10 acres. It is also capable of carrying out aerial structural/heat loss inspections and low-altitude oblique photography, as the operator can control the camera’s field of view.
Both fixed-wing drone and rotary-wing drone are capable of producing very accurate planimetric and height data, all within 5cm, throughout a 10-acre site. The orthophotography that is produced can be geo-referenced to ITM or ING75, with a pixel size of only 1.5cm. This allows for unprecedented accuracy and resolution.

APPLICATIONS
This means that drone aerial photogrammetry is as accurate as land surveying, but the data, because it is photographic, is far richer. It is also more complete and is much cheaper to produce than traditional aerial or ground surveying. Drone aerial photogrammetry is extremely useful for land registration, boundary issues and topographical surveying. It is ideal for road, roof or structural condition surveys; volumetric analysis; or planning enforcement issues.
Drone aerial photogrammetry can also be used for corridor mapping, coastal erosion mapping, construction spatial management and environmental monitoring – and that is only with a pocket-grade camera.
The list of capabilities expands dramatically when one adds an infrared or near-infrared sensors. With the use of thermal imaging cameras, drones can detect and pinpoint heat loss through roofs, and with an NIR sensor it can detect vegetative diseases and malnutrition in trees and crops.
Other applications include the identification of archaeological sites, disturbed ground, pollution plumes in rivers and streams, and structural defects in bridges. Drone aerial photogrammetry also has the potential to play an important role in helping to locate missing people as part of search and rescue missions; for large scale event management/concert planning; boundary planning; crime scene investigation or even land dispute litigation–and all at a fraction of the cost of the current price for such services.

Further information can be found here, which is a video of an Engineers Ireland CPD lecture delivered in January 2013.

It’s the mechanic that matters, not his toolbox!

When it comes to topping up the oil, changing a fuse, replacing a faulty light bulb and generally keeping the car clean and tidy, most of us don’t need to turn to a mechanic for these minor maintenance level activities.  However, when we get up in the morning and the car won’t start, when the check engine indicator light begins to flash or when we require a scheduled service, this is when most of us turn to the specialist services of the mechanic.  Why? Because we inherently know that we will most likely need knowledge, expertise and skills that are far beyond our own limited capabilities when it comes to engines.  It is true that the mechanic will draw on the contents of his/her toolbox to help diagnose the issue and it’s also true that the better equipped the toolbox the better the opportunity to quickly resolve the problem, but ultimately it’s the mechanic’s knowledge of how to efficiently and effectively apply the tools that will return the vehicle to its optimum operational performance. 

In this instance I use the metaphor of the mechanic to draw attention to the evolving role of the GIS professional, but I could easily have used the doctor, the engineer, the electrician, the dentist, the clinical physician, and so on; all of whom combine their superior knowledge, expertise and skills in their respective areas with their respective toolboxes, to help solve real world problems.  And just like today’s GIS professional they all supply information that helps us to make better decisions.

The modern day GIS professional has knowledge, expertise and skills in the effective and efficient application of geographical information that goes far beyond the toolbox.  For sure the toolbox is important, and, of course,  the better the toolbox the better the likely outcome, but even more important, in this ever changing and evolving industry, is the knowledge and understanding of how to use these tools to deliver business benefits and help customers make better business decisions.  In this new world where knowledge and expertise is king, it is no longer about the toolbox, it is now about the people; people who know best what tool to use for what problem and how to combine different tools to deliver the required outcomes and results.  The GIS professional is “the mechanic” in the world of GI enabled solutions and systems.

Another reason for using the mechanic metaphor is to do with the fact that no matter how functional the toolbox is and irrespective of how expert the mechanic is, if you do not put the correct fuel in the car, it will not run to its optimum performance and in some cases, it simply won’t go at all.  The same holds true for GIS, where relevant, fit-for-purpose location-based data is needed if the GIS professional is to combine his/her toolbox, knowledge, expertise and skills to achieve superior performance outcomes for his/her customer.

In the GIS industry today, the toolbox is now expected to have all the tools necessary to do the job, it is increasingly cheaper to acquire and it is a lot more accessible to a lot more people; and that’s good for everyone.  The challenge now is to ensure that there are enough knowledgeable, expert and highly skilled “mechanics” to help us all to deliver real business benefits to customers and stakeholders alike. After all, in this new world, it’s the mechanic that matters not his/her toolbox.

Paul Synnott

Hipsters, Hardhats, Neo-Geographers and Heavy Lifters….. Are you kidding me?

I am really beginning to wonder what sort of message the GI industry is trying to convey to the GI community here in Ireland. Why do we feel that we must come up with buzzwords that sound like they are cool and trendy but when you consider them in the context of the customers real world business issues they simply border on the ridiculous.

I am proud to be part of an industry sector that is making a fundamental difference to both public and private sector organisations and businesses, and one that continuously contributes to better decision making for the greater good of the future economic development in this country.  That industry sector, whether you like it or not, is GI, and I for one refuse to be labeled as either a hipster or a hardhat, in the same way as I refuse to be labeled a neo-geographer, a heavy lifter or any other hair-brain term that surfaces from time to time from intelligent people in organisations who should know better. 

To label GI professionals in this manner is simply disrespectful to the knowledge these people have built up over the years and to the expertise that they contribute to their respective organisations.  Furthermore it only serves to distract from, confuse and dilute the value proposition of what it is we (the industry) do for our customers.  In my experience customers and prospects want to engage with people who fundamentally understand how the effective and efficient use and application of geographic information can help them make better decisions for their respective businesses.  I would doubt very much that these same customers or prospects are remotely interested in engaging hipsters, hardhats or neo-geographers in this regard.

If, by using these terms, we are simply trying to convey the fact that the GI industry is changing and evolving with respect to spatial data (information & content), technology (software & hardware) and people (skills & expertise) then I fully support this proposition, but then let’s just say that and stop trying to be clever about it by making it sound like it is more than it actually is.

In this way, whether you do regard yourself as a hipster, a hardhat, a neo geographer or heavy lifter; whether you are in the open data, big data, INSPIRE, open source or COTS camps; or whether you are simply an end user or a supplier, it is incumbent upon us all to come together collectively and use what is best from all of these different views and opinions, for the greater good of the customer and the Irish economy.

The sooner we stop polluting the GI airways with nonsensical sales pitches, cleverer than clever marketing  and ego driven agenda’s, the sooner we can start to inspire our customers and prospects through the efficient and effective application of geographic information, in all its various manifestations, to help them meet their challenges and contribute to their successes.

So I put it to those of you who are interested, “Hipster” or “Hardhat”, a “neo-geographer” or “heavy lifter”, why is any of this even relevant, if, as an industry, we are focused on the needs of the customer?

Paul Synnott.

ArcGIS – what’s in a name?

“The species that survives is the one that is able best to adapt and adjust to the changing environment in which it finds itself”.Image

I often reflect on this quote by Charles Darwin (from Origin of Species in 1859) when observing the fast-changing and dynamic world that we live in. Everything moves on so fast, with the release of new “stuff” all the time.

For those of you (developers, individuals, organisations and governments) who are using as ArcGIS as a platform for mapping and geographic analysis, some of whom I see every day, the pace of change must be dizzying.

One dynamic is the constant updates and releases to the ArcGIS platform. And as one would expect Esri do have a convention to manage naming – this helps us to govern the product life cycle, but also to help you to resolve technical challenges. It also helps reflect the constant evolution of the ArcGIS platform as fixes are incorporated, enhancements are made, and new features and concepts are added.

In line with this, the ArcGIS naming convention is changing in parallel. You will hear us talk about upcoming updates using a month/year combination – for example the “March 2013 release” or the “June 2013 release”.

Once the software is released, then we are moving away from the major/minor release with service packs convention (e.g. ArcGIS 10.1 SP1) towards a numbering convention as follows: the March 2013 release will be ArcGIS 10.1.2 (focussed on enhancements and fixes to ArcGIS Online); the June 2013 release will be ArcGIS 10.2 (with significant changes across the ArcGIS platform); and so on (e.g. 10.2.1, then 11.0).

Of course there is an exception to all this – ArcGIS Online is always current, so new updates are referred to by their month/year of delivery.

If you are a customer of ours at Esri Ireland, I urge you to use the Esri customer care portal to keep on top of updates to the ArcGIS platform.

Another key take home point is that on December 31st 2013, as per the Esri product life cycle policy, we will see:

  • ArcGIS 10.1 move to “extended support” – meaning no new environments (e.g. operating systems, databases, web servers) will be certified against this version;
  • ArcGIS 10.0 move to “mature support” – as with extended support, but in addition there will be no further patches or hot fixes;
  • ArcGIS 9.3 and ArcGIS 9.3.1 move to “retired support” – as with mature support, but all phone, e-mail, fax and forum technical support stops, with the online support “frozen” (i.e. no longer updated).

I hope this helps you to understand the release naming so that you can plan your business around the evolution of the ArcGIS platform. If you need more information, why not leave me a comment or send us a note?

Michael Byrne

The Signs are Good?

In March 2011, I wrote about how I felt that geographic information underpinned the new (back then) Programme for Government. Back in April last year I wrote about how the Government had finally embedded digital mapping and GIS technologies into their plans for public service reform through four primary actions; contained in action items 27-30 .

Since then we have seen many public sector organisations really embrace the power of geographic information for many of their business process and organisational initiatives.

For example in the last year the Department of Environment’s MyPlan  was launched and has continued to grow in status.  We have seen the Department of Education embed geographic information into their on-going Education Surveys. And the Minister of State launched an economic study in into the value of geographic information and associated systems to the Irish Economy.

We even read how one member of the public, through Letters to Editor at the Irish Times asked why Geographic Information Systems were not being used to their fullest potential by Government (in the context of the National Children’s Hospital).

This month (February) Bord Gais has just begun a procurement process for the provision of Geographical Information Systems and Services to support our new Irish Water entity.  Last week we heard how the Revenue Commissioners are to use aerial photography to help with the issue of property taxation  and this week we now read how Local Authorities are to use electronic mapping to track down septic tank evaders.

Furthermore the Taoiseach agreed to the EUROGI (European Umbrella Body for Geographic Information) Eye on Earth  and imaGIne  conferences that are being held at the Dublin Convention Centre in March, to be categorised as EU Presidency Events, further emphasising the value that he sees in our industry.

So it certainly seems as though our Government is following through on its promise to “evaluate the potential for exploiting digital mapping and GIS technologies in ways that are affordable, sustainable and of relevance to the customer bases of their services” (Action Point 27) and that they are ensuring that “Public bodies will identify data sets they hold that contain location based data and will make these details available  to  other  public  bodies  where appropriate to reduce duplication and to facilitate  greater  area-based targeting of public services” (Action Point 28).

If you agree, I’d love to hear where you have recently experienced similar examples of Government embracing Geographic Information and associated solutions?

Paul Synnott

GI in Ireland – Our Challenges

Earlier this year, I wrote about the future of geographic information (“GI”) science and what it means for us all here in Ireland (http://esriireland.wordpress.com/2012/07/10/gi-science-the-next-10-years/). I discussed how the integration of virtual and physical realities is being driven by GI: “real” 3d/GI indoors; mobile; real time/dynamic communications founded in location; big data – both quality and speed; crowdsourcing; and linked data. I called for the co-ordination of information management, citing post codes as one example of the problem here, and the skills challenge facing our community – with a reliance on a small number of key individuals, and the need for greater teaching, research and industry collaboration. So how have we done over the last 4 or so months?

My belief is that we have done quite well, although I would still place our community in the foothills of growth, with greater heights yet to be reached towards the top of the mountain! The market in GI is moving in the right direction: contracts are being awarded within government (the mainstay of any GI community); businesses like Esri Irelands are growing; and various key initiatives are being undertaken. Bearing the latter point in mind, just look at the fledgling collaboration between the Department of Environment, Community and Local Government and Ordnance Survey Ireland to at long last breathe life into the delivery of the Republic of Irelands INSPIRE obligations, taking a leaf out of Northern Ireland’s book (see www.spatialni.gov.uk).

Locally here in the Republic of Ireland, the Department of Public Expenditure and Reform have released their pilot site www.irelandstat.gov.ie to highlight the achievements, actions, costs and international comparisons of our public sector reform programme. It is interesting though to see the focus here being on “how” initiatives (how people are managed, how people are organised, how expenditure is managed, how the political framework functions), but sadly there is a lack of “where”! For me GI offers options for presenting and visualizing data and therefore communicating the rationale behind evidence based decisions. GI offers a framework for sharing data, for collecting data, for cleansing data. GI offers tools for evaluating, modelling and measuring data. Yet, although “IrelandStat” is in pilot phase, there does not yet appear to be a GI approach to problem solving and results delivery here. When one thinks that much of the reform has trimmed around the edges and not yet really addressed the three pillars of public sector income/expenditure (a stable income from taxation; public sector cost reductions; social welfare cost reductions), it is clear to me that GI would at the very minimum help with communicating some of the unpalatable messages that are yet to come.

Very encouragingly the open data movement in Ireland is showing further signs of building momentum, despite the kildarestreet.com setback (http://www.broadsheet.ie/2012/09/20/killing-kildarestreet/). Interestingly a recent report in the Irish Times by Conor O’Carroll (http://www.irishtimes.com/newspaper/sciencetoday/2012/1025/1224325667800.html) highlights the path to open access to the outputs from public funded research – it is perhaps easy to see how such an approach could be extended to publicly funded data. The community in the open data space has rallied together and recently began what we hope will be an on-going monthly meeting of interested parties. There is a real sense of energy around the open data community in Ireland, and I would urge you to get involved!

Globally this movement continues to stir thoughtful and insightful debate: a city head of IT in Uruguay calls (http://blog.okfn.org/2012/11/01/towards-a-public-digital-infrastructure-why-do-governments-have-a-responsibility-to-go-open/) for public data (and also services, software tools, apps) to be treated as a national infrastructure, just like roads for example; San Francisco is starting to reap the rewards of treating open data as a key city infrastructure (http://radar.oreilly.com/2012/10/san-francisco-open-data-economy.html); one of Europe’s open data evangelists, Ton Zijlstra, is spending much of his time travelling the world to spread the community building message (http://www.slideshare.net/TonZijlstra/community-building-some-guidelines#btnNext) as a key to successful open data usage; and there are calls from within government in the UK to open up the Royal Mail postal address file as a core part of a national address dataset to generate new revenue sources for businesses (http://m.guardian.co.uk/ms/p/gnm/op/sQShafeIjaSmuj41C1_S3Xg/view.m?id=15&gid=technology/2012/nov/01/royal-mail-postcode-address-data&cat=technology).

So has my call for action changed in four months? Not in substance, but yes in terms of evidence:

I very much look forward to the release by OSi, probably in mid 2013, of an economic survey of the size and potential of the GI market in the Republic of Ireland, commissioned on GIS Day (November 14th 2012) by Minister of State Fergus O’Dowd (http://www.osi.ie/News/Latest-News/Minister-Launches-GIS-Study.aspx). It will be interesting to see if this stimulates some of the debate above and helps to ensure GI is a key part of national initiatives. As many active stakeholders in GI in Ireland already know, the constituent components for collecting, cleansing and sharing data are all there to be harnessed within our community of practice – I hope that in a few months time I shall be writing gleefully about the progress being made on the points made herein!

Michael Byrne, Esri Ireland