This episode features an interview with SpringML co-founder Prabhu Palanisamy discussing Analytics and AI ML in 2020.
Welcome to SpringML’s Podcast Vision 2020. Today’s episode features an interview with SpringML co-founder Prabhu Palanisamy discussing Analytics & AI/ML in 2020. Now here’s your host SpringML CMO Megan Dahlgren.
Megan Dahlgren – Thank you, Tammy. I’m delighted to kick off this conversation with Prabhu Palanisamy launching our 2020 SpringML podcast. Prabhu, looking ahead at analytics in 2020 do you have any advice for our Enterprise customers on how to be thinking about the industry in the market next year?
Prabhu Palanisamy – How we see the analytics market changing and what the users are looking for we observe that in three major areas.
Firstly, How do I use the insight to derive actions that will have a meaningful impact on my business operations? We live in an environment where we have data deluge and insights deluge. We have a lot of insights that are generated. We need to think about how it is going to be meaningful to my business operations. How do I filter out the noise from the signals that will impact my change to the team or business operations? That is an area we at SpringML focus heavily on. We analyze the performance, whether it is sales, service, or marketing-related to use cases that will provide not only the insights but the actions that the user can change.
The second trend is to observe how to make sense of the data. Preparing and processing data has predominantly been in back-office IT environments. SpringML is helping out enterprise users to prep and make sense of the data.
The third environment is the current buzzword Artificial Intelligence and Machine Learning. Enabling enterprises in the ML journey starts with having a solid analytics foundation. We at SpringML help enterprises not only process the data but do the right level of Analytics to enable the ML/AI in the right way. There are a lot of technologies and frameworks, and we do the fit to purpose a tailored environment that will allow enterprises to get started. So to me, those are three areas that we see as a trend that SpringML can potentially help enterprises in 2020 and beyond.
Megan Dahlgren – Well thank you for that introduction Prabhu, how would you recommend companies approach analytics, get started with it?
Prabhu Palanisamy – Number one is the incremental approach meaning customers have established business intelligence decision support systems and want to include the next generation of features to support ML or AI-based Analytics Insights. These are incremental proof of concepts you know the art of possible projects that will allow the enterprises to look at data differently. We recommend this approach to enterprises that have established BI and Analytics that support the function.
The second approach that we look at for enterprises is to move and improve. We migrate the existing environment to a cloud provider, get the immediate benefit of scale, cost efficiency, and processing power, and then develop the analytics pipeline from data processing to generating the insights.
The third approach that we are looking at for enterprises is re-architecting from the ground up. This is potentially suited for environments where customers have legacy tools, technologies, and a fragmented approach. So instead of trying to build the pipeline in the current infrastructure, we look at making the Next Generation modern data platform to support analysis.
Megan Dahlgren – So when you think about experimenting with AI & ML across these three different types of adoption journeys, the experiment, the move and improve and the legacy start fresh we build from scratch models, are there some best practices that you can highlight for each of these three different journeys when you’re thinking about AI & ML?
Prabhu Palanisamy – The biggest question from the adoption is time to value. The first approach that I mentioned was the incremental experimentation mindset. It is beneficial for enterprises to pick the right use case and see the use case in action within their environment. This builds a lot of confidence, comfort level, lessons learned for the teams to start building upon it. So the best practice some of the things that we request a customer on the proof of value is don’t overdo it meaning, let’s define a use case that we will be able to see value in a matter of 10 to 12 weeks. We don’t want to make it much more significant, much complex, or even overdoing it by bringing in all the technical jargon of ML, AI, models, etc. Pick a use case and define a use case that we should be able to find the right data set and map the questions that we want to answer within the data.
The second approach of move and improvement is of two parts the first best practice in terms of how we want to approach is taking advantage of the containerization, cloud provider scale, and cost-efficiency. This is basically like doing my foundation and getting ready to use Advanced Technologies. The best practice or the recommendation that we recommend to our customers is how do we put the migration plan in place that quickly allows the data and the associated applications to run in a cloud environment.
The third one is basically architecture. It’s better to start with a clean state of paper for environments that have a legacy as well as a fragmented decision support system. Define the analytics pipeline and build the common data pipeline insights, dashboard, visualizations instead of trying to stitch and make it work.
Megan Dahlgren – Wonderful Prabhu, this framework of these three different journeys, is a beneficial tool I think for the industry in the market to understand where they need to frame up their journey and move to adopt AI & ML Technologies. It sounds like I imagine some large companies could do a mix of all these three different approaches is that fair to say?
Prabhu Palanisamy – That’s correct Megan. Yep!
Megan Dahlgren – When you start to think about these different approaches to the adoption of AI ML Technologies. Are there specific tools or technology trends for each one of these different approaches to your journey from experimentation, move and improve and starting fresh that you can speak to you know a technology that comes to mind would be something like big query?
Prabhu Palanisamy – Yes, from the analytics and building next-generation modern data platform point of view we need a couple of tools. The number one is the ability to process and support large-scale data processing. So imagine having a data warehouse environment that scales and fit to customers’ data insights. That by itself reduces a lot of management and infrastructure overhead for organizations to support the complex BI environment. Our team uses Google BigQuery in building modern data management. It’s a petabyte-scale fully managed service that Google Cloud offers that solves the data processing, data storage & data management problems. We still need a few other items, one of them is the integration which is required to pull data from various data sources to bring into Google BigQuery. Integration is often a lot more time-intensive and laborious than building the BI itself at times. Mapping it again to the Google Cloud platform is helpful. We have services like Data Fusion, Data Flow, in our Salesforce ecosystem, we use Mulesoft. These are integration tools that allow to extract from various sources and pipe it into a data warehouse environment.
The last mile of analytics is the visualization and generating of insights. We are fortunate to work with the leaders in this segment both with Tableau on the Salesforce ecosystem and on the Google site data studio and Looker. They provide a comprehensive set of dashboarding and visualization features for users to build that last mile of analytics. Then you overlay it with ML features like BigQuery ML, Datalab or integration, python, a whole lot of data science frameworks that are natively supported in a BigQuery allows you to immediately take advantage of the ML features right within the BigQuery.
Megan Dahlgren – When you think about the different technology options that are out there you know it’s also from adoption perspective how you think about technology you’ve mentioned in the past that there can be drivers like business-driven drivers technology or IT-driven drivers package solutions versus building your own do you have any best practices or high-level advice for the industry about how to think about tools as you go on this journey?
Prabhu Palanisamy – Perfect! That’s a great question Megan, for example, let’s take marketing as a function within an enterprise. Marketing teams, in my opinion, deal with large data sets and are often required to answer some of the complex behavior questions like is this lead going to close? What’s the audience segmentation should I target my customers? How should I even structure my campaign to focus on the right customers that will buy or are also come as a repeat buyer? If you look at it on the peripheral side, these require a lot of data, high-quality data, and constant analysis to answer each of these specific questions. What would be my retention rate? What would be my average lifetime value of the customer? Who are the customers that are at the risk of not renewing a contract etc.?
These are all business-driven use cases and if we approach these use cases in the traditional way that’s where IT teams are set up to build BI environments, setting up of data warehouses, visualizations, running insights and providing the dashboard and reports. The demand from the marketing team is, I wanted now and in real-time.
If we take these use cases and try to implement it in an existing way, it’s not possible to address those requirements. So, for example, the marketing team is very clear on what they need and how quickly they need it. This requires that to how these implementations need to be approached. The combinations of using new technologies combined with the execution and implementation approach of organizations like us are going to meet those requirements.
Megan Dahlgren – You potentially walk us through an example of obviously we can’t name the customer but if you could share how this journey has taken place with some of our customers where we really look at, you have to do this differently and I would say in answering the question turning to the IT team to say hey guys this is how we really need to be thinking about this moving forward.
Prabhu Palanisamy – Exactly, so I will continue with the same example of marketing analytics that I gave. So one of our customers is a leading CPG organization, and they wanted to understand how their campaigns are performing in assigned territory, region, and how the audience is responding to those ad campaigns. You learn from the campaign performance, and it takes a while, like say 6 to 8 weeks for the team to get the data analyzed, find the performance of the campaign, reaction, the conversion of the audience, etc. What happens now is that results are the lessons learned are applied to the next campaign. So basically, given the time lag and the time taken to process these data sets.
The lessons learned are applied to our future campaigns. So what is challenging here is the environment is never going to be the same. The demand from businesses I want to know how my campaign is performing and how can I make changes to the current campaign immediately, which means closing the loop within 24 hours. If we look at it, this requires a fundamental shift into how BI or decision support systems are built. This means you get the data, store it, run it as a batch, find some insights, build a dashboard, and users logs into the dashboard and are able to see the historical performance. The shift has to change on every step of the processing, meaning a batch-based execution to switch to a windowing to a streaming related process involving a constant stream of data comes in.
The analytics, Machine Learning, BI needs to be performing at a scale that processes data as it comes in and similarly, the visualization needs to support volumes of data and at the same time meet that into the users workflow meaning ability to access the dashboard in Mobile or their CRM environment or their applications of choice. These three pieces require a fundamental shift from what is being done today to an environment that will meet the customer’s requirements.
Megan Dahlgren – Understood so if you were to basically give I don’t know a high-level maybe three tips for BI teams that want to get started doing this what would your recommendation be?
Prabhu Palanisamy – Number one is getting the data correct. I’m sure this is something every best practice or every recommendation has this one, but it’s supercritical important. Getting the data correct solves a majority of the issues and the related analysis making it easier as we deal with complexity — number two asking the right questions. Every organization will have questions that they want to answer on, for example, who are the customers that will churn away. That’s a million-dollar question for every enterprise. How we can effectively answer this question relies on the data that we can collect and use technology to analyze and come up with a meaningful result. So asking the right question, validating the right question to meet the data that I have is supercritical.
Number three straightforward. Consulting one on one crawl, walk, and run. Organizations that are heavy enterprise workflow-driven can start simply by getting the right data set, asking the right questions, and doing the analytics as a foundation and then gradually extending into AI & ML. There is nothing wrong with this approach. It will be a journey where there is a lot of excitement and eagerness to get started with ML, but having the right foundation of data and analytics is not only making it easier to adopt ML or AI. It will also help prioritize the proper use cases from AI and ML standpoint.
Megan Dahlgren – It sounds like that’s because when you get the data cracked and you ask the right questions that are really what drives the technology adoption not the other way around?
Prabhu Palanisamy – That’s exactly right Megan.
Megan Dahlgren – That’s wonderful this time with you has been really enriching for me and I hope for our listeners too and we look forward to our next interview.
Prabhu Palanisamy- Thanks to Megan for the opportunity.