Business is demanding faster time to insights to impact the data driven ways of interacting with customers and value chain. Analytics market is going to be changed by AI & Machine learning within how insights are generated, how they are consumed, how they are shared and how they are acted upon. From manual processes to AI infused machine learning augmented capabilities for authoring, and acting upon context, we are in the new horizon that impacts time to insights, accuracy and reduced bias.
Evolution:
We started with semantic layer based platforms, where we encapsulate the complexity of the underlying relational data with a single source of truth for enterprise reporting. We later had interactive analysis with visual based exploration with user experience. We now need to infuse AI and machine learning to automate the data science modelling and operationalizing model process. Natural language will be a dominant paradigm on how we consume and interact with analytics today.
The Need:
1. Be automatically served with what are important relationships in data, important drivers, correlations, segments, outliers
2. Explore those relationships with natural language both in terms of asking questions like using personal assistants such as Alexa, Siri.
3. Explain the data relationships in natural language along with data visualization.
4. Being able to have augmented analytics serve us with analytics.
5. Being able to interact with data through conversation and also being embedded in applications that we use.
Comparison between Modern Analytics Vs Augmented Analytics:
Our current traditional reporting solutions help us report on known data and known questions. The cornerstone of modern BI is visual based data discovery and exploration. This does help us to ask questions more quickly but the data is still known. We have some idea and assumptions on the relationships that we want to explore with the questions we have.
When we don’t really know the data, when it is in the data lake, we can explore data in more free form but data lakes still need to have some structure, & some model. The business still needs to know the questions that they want to ask. We need to have different approaches to move beyond. NLQ/NLG will help us get to more unknown type of questions, unknown answers, and unknown relationships.
Essentially when we build analytics model, we prepare the data, find patterns in data, share these findings/insights. We operationalize the insights. When we prepare the data, we manually select tables, analyse, join them together. We add different types of business logic, calculations, groups and hierarchies. We clean the data, enrich it in different ways by adding new data sources. Catalogue, tag and have lineage to the data. These processes are still manual.
In augmented data preparation, algorithms are used to make lot of those manual processes easier. They auto detect the variables, schemas. They profile the data and automatically look at different shapes of the data, outliers and make recommendations about cleaning, enriching, lineage, and metadata.
In finding patterns in data, today we have to manually explore the data using interactive visualizations. We have to manually feature engineer and build advanced analytics models. In augmented analytics, we bring in interaction concept like NLQ. Algorithms find all patterns in the data. Features are auto selected, models are auto selected, even increasingly code is auto generated
In order to share and operationalize findings, today we depend on the user to interpret the results. However, by adding natural language narrations, users with different analytical expertise interpret the same insights from the data.
As we begin to explore the unknown, we will need to consistently explore and find a way to solve complex analytical challenges.
Andrew Stephen, PMP, CSCP, ASQ CSSBB
