Anticipate, Advise, Adapt, Act: The Data Deep Dive

Disruptive Analytics – a platform for a better understanding, as described by Dave Bonn

It’s data, data, data all the way: “It’s tomorrow’s oil”; “Big data is the way forward”; “You can’t have enough data”; “Data analytics is the future”.

We see these or very similar expressions in every technical publication, we hear it at every conference these days, no matter what the domain under discussion is. Do we believe it? Does having data really make things that much better? Is it just about collecting more and more data? Do we know why we are collecting it? And perhaps more importantly, are we getting the best out of the data we have?

To look forward we need to consider what has been happening in the past. Centralised systems became commonplace with all sorts of interfaces to data sources, many with bespoke interface protocols and of differing update rates and widely varying data quality. As time has progressed, standardisation for same type devices has improved these aspects significantly but the standards have struggled to keep up with the myriad of new data sources that are being deployed on the transport network. Looking backwards, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable, both in real time and historically when trying to predict future operating scenarios.

In the past, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable

Quality street

There is an increase in the use of Edge Computing where data interpretation is taking place closer to the point of measurement. This can only lead to better more reliable actions being taken in a repeatable fashion. The central systems then need only be told of the actions taken. This distributed processing may reduce the amount of data transmitted back to a central location but doesn’t reduce the need for data collection local to that device. Balance this with the increase in crowdsourced data available, social media, connected vehicles etc, where vast quantities of data are available but it still needs to be sorted and filtered to extract the key information relevant to the transport operator at that time, at that location. It is the ability to get quality information that can support operational staff make the best decision that the industry is striving for.

Having sourced the data, cleansed it and packaged it into the format that better support the decision-making process, the central system can then undertake informed analysis on what action is best taken to meet current and future conditions. We see the Intelligent Congestion Management Programme (ICMP) in Sydney seeking to “detect in 5, react in 30”, or in other words detect current conditions and react in a way that prevents adverse network conditions in 30 minutes. This requires a significant change in approach to decision making away from reactive to proactive intervention strategies. Accessing historical data helps, using current data is critical but more so is the need to bring in augmented analytics which enables other impacting factors such as weather, time of day, day of the week, angle of the sun, air quality, presence of surface water, speed of traffic, volume of vehicles, if the road being used as a diversion route, and so on. The list of factors that impact driver behaviour is extensive.

In the past, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable

Think of augmented analytics as the closest our machines today can get to the human brain in terms of processing, balancing and reasoning through multiple data sources to confidently predict and plan for the most likely scenario. How does augmented analytics emulate the human brain to surpass less effective single thread data analytics, data cubes and data warehouses? By following the Anticipate – Advise – Adapt – Act sequence, augmented analytics parses which data sources present evidence of value to generate a Prescriptive Option. As mobility services automate, there will not always be a human brain to act on the emerging scenario. The time saved by augmented analytics to quickly prescribe the option to treat a risk or incident can mean the difference between commuter chaos and re-routing for trip optimisation.

The deepest of all possible data

To understand how “augmented analytics” can support transport managers, they must first identify the problem they are trying to solve. Too often this isn’t clearly understood at the outset of the problem-solving journey. In common with many other industries, the transport sector agrees that data analytics is a very good thing and that it has the potential to drastically improve decision making (if done properly). Many however recognise that data analytics is not exactly the easiest thing to pull off in the real world. For example, you may detect that traffic using a particular route has increased in the last year by 15 percent but the reason for that change may not be immediately obvious. That is why the use of deep data analytics is key to informing you of the reason for the change so you can react and decide whether you want to discourage the increase or apply the same reasoning to other routes.

Now that the reason that drove the change in driver behaviour is understood, transport managers have an insight that is actionable because it connects directly to an action you can take to address your traffic change challenge. Going forward, your decisions are now based on known factors and an understanding of their impact. This identification of the reason for change required the use of disparate data sets, data cleansing, analysis and the presentation of the results in a real-world action driven format. What augmented analytics does is to relieve a business’s dependence on data scientists by automating insight generation through the use of advanced machine learning and artificial intelligence algorithms done using natural query language.

An augmented analytics engine can automatically go through the available data, clean it, analyse it, and convert these insights into action steps for the transport network operators with little to no supervision from a technical person. It is data fusion without the programming. Augmented analytics can, therefore, make analytics accessible to all. By involving staff with an understanding of the domain but who are not necessarily data scientists, the power of data insight and how it can be best used is given back to those in the business who are tasked with making informed operational improvements: it’s a business pull model.

An augmented analytics engine can automatically go through the available data, clean it, analyse it, and convert these insights into action steps for the transport network. It is data fusion without the programming

Further questions remain: from where does this approach get its from? Does it need to go to a data broker who has collected the data already, does it create its own data broker store through data replication or does it need to measure its own data? What is available today is the ability for the augmented analytics engine to interface to data stored anywhere. Be it a legacy store or a real-time source these tools, subject to access constraints on the part of the source system, can access and use any and all available data sources. The concept of creating a “local copy” of the data for analysis purposes is no longer required. The role of “Data Broker”, the lack of which, for the transport domain, has prevented many organisations from accessing the data that would support them to gain a better understanding of what they were trying to manage, is now not an issue. Staff in these organisations can go directly to the source of the data they need without the use of an intermediate facilitator or Data Broker This is very much a disruptive approach in the transport data analytics environment and one that will take some organisations a period of time to get comfortable with but it is the future.

Open for business

A frequently encountered challenge that will continue to frustrate efforts to make best use of data collected is the constraints many organisations put in place to hinder collaborative sharing of data. There have been may initiatives from governments across the globe striving to make data “open” and this is to be encouraged. Bringing industry along with that same thinking is key. Too often, claims of commercial sensitivity and confidentiality is hampering the wider sharing of data. Yes, we must respect the commercial sensitivity of data sets within competing organisations but perhaps some are over-protective of some of their data and, if it was made “open”, better decision-making by other organisations such as network managers could be achieved.

If augmented analytics can be used to support mobility decision-making, the potential exists to improve government, corporate and private contributions in a wide range of aspects including environment, sustainability, heath, liveability, urban planning and employment opportunities. The use of augmented analytics enables the impact of decisions made to be fully understood taking into account all the variables that impact a decision. The people undertaking that analysis understand the domain and hence better understand the relative impact of the measured environment.

There are organisations that have an understanding of how to enable the augmented analytics approach to release business benefits. The ROI associated with using this new technology will always be an initial question however there is an increasing amount of evidence to show the benefits that can be achieved. Organisations such as Silicon Billabong offer support services on the use of the products available in the market today. Silicon Billabong’s approach is to support organisations identify the “low hanging fruit” in the data world ensuring that early benefits can be both demonstrated and realised for the benefit of the business. This gives senior management the confidence they are looking for to take this approach forward. It also gives those new to the tools the confidence to take on more challenging tasks. Using only the internal data that is currently available for the initial deployment is usually the first step before branching out into data sources in external locations. The rate of deployment of more complex solutions can then be driven by the business using domain-aware staff rather than waiting for data scientists to be hired.

While this disruptive technology still has some convincing to do within some organisations, there is little doubt that augmented analytics will give businesses who embrace the technology as part of their data plans a market edge on their competitors. This won’t be a quick fix to all data challenges but it enables transport operators to better understand the impact different factors have on driver behaviour, enabling the most effective interventions to achieve a desired outcome to be better understood.

Transportation Case Study 1

Operational Transport Management

Transportation Case Study 2

IoT – the Future for Connected Devices on the Transport Network

Transportation Case Study 3

Analytics to Support an Investment Business Case

Transportation Case Study 4

Post Project Justification of Benefits Realised

Dave Bonn is Managing Director of Bonn Business Solutions. For more information on how Silicon Billabong can help take you on the data discovery journey call Errol Kruger on +61 481 120 883 or email him at

Published by

Share now

Read next article