Niamh O’Brien, Technical Success Manager at Fivetran, addresses how analytics is informing digital data technology today
Niamh O’Brien, Technical Success Manager at Fivetran, addresses how analytics is informing digital data technology today. O’brien tells Technology Magazine how cloud evolution democratises choice and access to data analytics.
Predicting the future may sound like the stuff of magic and myth, but modern technology and data analytics capabilities are making business forecasting more accessible than ever. The key drivers of this transformation today are the expanding use of business intelligence (BI) and data science, including machine learning (ML) models.
While many businesses already rely on aspects of both to inform business decision-making, there is a plethora of opportunities to successfully utilise BI and ML that – despite providing lucrative results to businesses of all sizes – are often overlooked or deemed too technically complex to implement. This inflexibility, limitation in choice, and burden of manual data engineering work is fast becoming a thing of the past, thanks to the evolution of analytics-enabling technologies collectively known as the modern data stack.
The modern data stack comprises automated data integration and transformation, centralised cloud data warehousing or data lake, and business intelligence. Together, these form a series of steps that turn raw data from various different applications into actionable insight that can be the basis of benchmarking, operational improvements and even forecasting. The task of unearthing and presenting business trends is that of business intelligence.
The role of business intelligence and data science
BI can provide insight into a variety of issues such as variabilities in company processes or consistent delays to services, and it can also make inferences for the future based on the trajectory of growth, churn, profits and costs. Business leaders use BI to guide their next steps, but this valuable information comes with two caveats – namely that all BI processes are ‘backwards-looking’ (they can only make connections in the data that’s already there) and that the information will always be interpreted by a human rather than a machine. And herein lies the challenge. The explosion of available data, coupled with the advancement of analytics capabilities, means that today BI is a necessary but not sufficient condition for future-proofing organisations.
To achieve the next stage of digital maturity and scale at speed, businesses will need to increasingly invest in data science, which in turn, will require much more robust and well-governed data processes. This is a real step-change for organisations. While before, by relying on BI alone, they were able to “get away” with data quality issues – since any results irreconcilable with common sense could be filtered out by the humans looking at the data – they won’t have this leeway once they deploy machine-learning capabilities.
The power of ML is that it can take data much further than BI is able to – unearthing and predicting events beyond what can be reasonably expected based on past experience. For example, while a company can use BI to forecast spending based on current business rates and expenses, ML is able to analyse these trends in the context of wider (internal and external) factors and predict more accurate outcomes. However, just as with BI, machine learning relies on data it can trust. Rather than discard anomalies, as a human would do, ML algorithms “learn” data quality issues as patterns, which can lead to inaccurate or even dangerous predictions.
Going forward, the determinant of success for companies won’t be how fast they can leverage ML, but how well they can do so. And without stringent data governance processes in place, they will be taking two steps back for every one step forward.
Multi-cloud evolution
These processes may sound overly complex but the evolution of the cloud – and especially, the rise of multi-cloud environments – means that making BI and data science work in tandem is more achievable than ever before. This evolution is the result of companies’ growing realisation that they need different cloud services in order to carry out different types of analytics.
For example, while data warehouses – with their structured data – lend themselves perfectly for BI processes, they fall short when it comes to advanced data science. Conversely, data lakes – which are able to run complex analysis even on unstructured data such as images – are more suitable for advanced analytics, but can seem like ‘data swamps’ without any clear organisation. The evolution of the cloud means that today, companies can cherry-pick the functionality they need without getting locked into any single technology. The multi-cloud enables them to choose an analytics stack based on what is best-in-class and will bring the most business value to their specific use case.
In this sense, the evolution of the cloud is really an evolution in choice and access to data science. With the modern data stack supporting this multi-cloud strategy, businesses can open up BI and data science to users across the business. When we consider who uses the data, the most obvious answer may be data engineers, data scientists and data analysts, but in reality, every job function is tied to data. Democratising access to data in a governed manner is just the first step in the right direction – in time, the multi-cloud will empower wider data teams to rely on machine learning for a variety of use cases and drive business growth.
The future is BI + ML
It’s clear that for businesses that want to successfully integrate the valuable insights from BI while leveraging the predictive forecasting gleaned from data science, a multi-cloud strategy is the future. Using BI and ML processes simultaneously, business leaders will benefit from a comprehensive view over their data as well as the decision-making paths available to them.
Those that are rearing to start the next phase of their analytics journey, however, must evaluate their data governance strategy first. A common governance framework across clouds will be key to avoiding data duplication and any resulting compliance issues. Companies that take the initiative to invest in robust data pipelines and automation will benefit from a single version of the truth when it comes to data, which in turn will enable them to reach business-critical insight in the shortest time possible – ultimately maximising the value of their data, people and technology investments.
About author
An established thought leader and analytics expert, Niamh supports Fivetran’s mission to deliver analysis-ready data to enterprises. She gives practical advice to business decision-makers wanting to take the next step with their key technologies – setting the analytical foundations to the growing use of artificial intelligence (AI) alongside business intelligence (BI).
