What to Ask When Implementing Machine Learning
Successfully operationalizing machine learning models in production environments can be incredibly difficult, as the industry has already seen. In fact, Gartner has predicted 85 percent of AI projects in the next few years will fail to produce results. So how do you set your AI projects up for success? ML implementation requires a unique approach that is a complete shift from the traditional approach to IT projects.
Data and analytics are the centerpieces of ML projects. Data is the foundation that informs decision making and drives increased efficiency. When implementing ML, you have to identify key questions and objectives, assess your data, identify data architecture gaps and then build systems to support those objectives. The questions that are answered through ML will vary greatly depending on your business process.
There are three key question areas to ask at the onset of ML projects:
- The Goal
What are the important questions about your business process that you want to answer? Is it around lead scoring, revenue or forecasting? Think beyond the use of a single ML model. The business goals should drive the implementation of ML and the model should be one piece of the bigger picture. Rather than injecting ML into a single part of your business process, it is useful to think about the long-term vision of how you can use data-driven decision making to fundamentally change how things are done today. Once this vision is sketched out, then you can generate a road map to get there by breaking it down into attainable questions that can be answered using ML.
Questions need to have direction, and must be answerable with a single, measurable criteria. For instance, asking at the front end if you can become the industry leader in your market is too broad. Instead think about measurable outcomes that can help lead to this broader goal, and what data you have to support those measurable outcomes. Key KPIs for your business are typically a good place to start, but don’t be afraid to get creative. Utilize your data, think about what you can do with your historical data. For example, what is the propensity for a customer to buy a product based on their attributes and past transactions with our business? You can see if there are actionable insights that would allow your sales reps to make changes that would increase the probability of making a sale.
- Measuring Success
Unused models aren’t helpful to anyone. If people in your organization don’t think the machine learning implementation is adding value, they won’t trust the results or use them to help drive change. This means that finding a means to effectively communicate the results of the model is as important as developing a good model -- in some cases, it’s even more important. Additionally, you need to identify the end users that will take action based on the results of the ML implementation. How will the results of the model be surfaced to the people who need to use them? For example, if you predict the propensity for a customer to churn, you can surface that propensity for a service rep who may be able to focus energy on high-risk customers and increase service efficiency.
- Data Collection
You have to know if the data exists to help answer the questions you are asking. Start by considering what factors/indicators you currently collect data on. For factors that might affect the outcome and for which you don’t currently collect data on, how can you set up a process to bridge that data gap? If no historical data exists, you can’t build useful predictive models. For instance, a company interested in studying the dynamics of a new product might be better served implementing analytics around new product data than trying to fit a predictive model if little to no historical data is available.
Knowing where your data comes from is key to knowing the current state of the data. You also need to assess how long data has been collected in a specific way. If an organization recently made changes to their data collection process, but only has small volumes of quality data, you may need to change your approach until enough data has been collected. For example, a company may want to predict the propensity for closing opportunities but only collects a few pieces of information on those opportunities, none of which are highly predictive of closing a deal. The beginning of the ML implementation can involve discussions with the sales team to understand the sales process and the factors involved in closing a deal. This is where you start to identify gaps in the data. Through those discussions, you can determine what information should be collected in order to build a more effective model to look at this propensity.
Implementing ML to enhance business process can lead to enhanced efficiency, better understanding of business processes and ultimately fuel a business transformation. Not all ML implementations are successful. The best way to avoid a failed implementation is to ask good questions and keep goals and expectations aligned from the outset. If you start with clear goals and questions, you can focus on the bigger picture which allows you to build a data-driven system. Don’t forget, you need to determine how your insights will be used by members of your organization. Being able to convert model results into actions and coaching moments helps teams better respond to their customers. By tying good questions, good data and a plan of action together, organizations can drive much more value from their ML projects.
Eric Loftsgaarden, is VP of Innovation and Data Science and a c-founder of at Atrium, a technology consulting firm.
