Data Science for All: How Automation is Transforming the Data Scientist’s Role in Retail

Data Science for All: How Automation is Transforming the Data Scientist’s Role in Retail

16 March 2020

In 1913 Henry Ford revolutionized automobile production by introducing the first moving assembly line. This one innovation drastically reduced the amount of time required to build an automobile by nearly 80%, and also made the automobile cheaper, faster to produce, and less expensive to purchase. Just a decade later, Ford’s Model T was the vehicle most American drivers had learned to drive, and the automobile became ubiquitous and accessible for more Americans. Automation forever disrupted the auto industry for workers and transformed daily life for the public.

Data Science is on the verge of a similar disruption and will transform the retail business. In the same way that automation democratized the automobile for the public, automation will democratize data science for those of us in retail, making it possible for all to benefit from high quality data science. But to fully capitalize on this opportunity, the role of the data scientist must change. And it will change, as automation allows data scientists to spend less time managing the algorithmic “heavy lifting” and more time identifying and communicating solutions to their organizations, which in turn will create greater efficiencies and opportunities.

Automation will disrupt the way analytics are delivered. To understand why and how this will occur, we need to examine the Analytics Value Chain. There are many models used to describe the analytics process from start to finish (e.g., CRISP-DM), but I’ve found the following generalization to be helpful when managing analytics processes:

Data Science for All: How Automation is Transforming the Data Scientist’s Role in Retail

Automation will largely affect the “middle” part of the value chain – namely, the steps of acquiring data and conducting the analysis. These steps will get much easier as automation will make the time-intensive processes of managing data, integrating data, cleaning data, testing models and identifying the best analytical solution much faster and less costly while producing better results.

As a result, like the auto worker, the role of the data scientist will change. They will spend less time producing analytics and more time scoping and planning the analytics, then summarizing the findings and telling a compelling story. “Softer” skills will become increasingly important, and the best data scientists will differentiate themselves on this basis. According to McKinsey Global Institute’s Notes From the AI Frontier: Modelling the Impact of AI on the World Economy, “Demand for jobs could shift away from repetitive tasks toward those that are socially and cognitively driven.”

This is not to say that quantitative expertise will no longer be required. Rather, all data scientists will need to understand the toolkit – the appropriate use of analytical techniques, the data required to be successful and how to properly interpret the outputs to determine good from bad. What they will no longer need to do is create and “tune” advanced statistical or machine learning models, as automation will fulfil most of those tasks.

As a result, Data Science will be accessible by more people, and in some sense everyone will be a data scientist.

According to Gartner, citizen data scientists, “are now able to perform sophisticated analysis that would previously have required more expertise, enabling them to deliver advanced analytics without having the skills that characterize data scientists.” Data Science will be ubiquitous in most organizations and will be available to us all, in some form, in our jobs.

Not only will there be more people serving as data scientists, but everyone will have access to technology that aids in creating analytics. There are many software companies that have created automated machine learning capabilities, and these technologies will continue to evolve rapidly. In the same way word processing software like Microsoft Word eliminated the need for a typing pool (Google it…it was a thing), automated analytic software will move most analysts out of the central analytics team and into business teams. This will allow companies to create analytic communities on the same scale of the typical Finance organization in a large business, where each department or division has a designated finance person supporting them.

At dunnhumby, we are already applying automated science in many of our products. For example, in our assortment planning software we have automated the process of identifying shopper need states and customer decision trees, to allow our assortment science to be easily accessible for retailers – regardless of their size or sophistication. We have also created an automated machine learning platform to accelerate data science within our clients’ organizations and our own. This platform is focused on solving complex retail challenges, such as understanding customer churn and predicting propensity to purchase. Finally, we have instituted a comprehensive training plan, including courses in critical thinking, problem solving and effective communication.

Similarly, Walmart continue to grow their data science capabilities and have a vision for growing their business through data-driven decisions and automation, and Microsoft have a vision for bringing advanced data science to Retail. Retail is in the midst of rapid change.

Here are four steps retailers can take now:

  1. Get started. Make an investment, even if small at first, in data science applications and people to start your journey. Begin with a department or small business unit and equip them with a few data scientists and the right tools and support to find quick wins that demonstrate the ROI.
  2. Equip your people. You’ll need at least one seasoned data science expert who can lead the business community as well as the analyst community. Without it, real change won’t occur.
  3. Partner. Find trusted advisors to help step-change your capabilities over time through strategic guidance and resources. You can buy tools and hire data scientists, but you’ll need applied know-how and change management to get value from your investment.
  4. Plan for scale. Once you start to earn some wins with data science, demand in your business will exceed supply. You will need to scale your partnerships, people and technology rapidly, so plan for the growth and budget accordingly. When done right, the ROI from your quick wins will support the business case.

We are in the midst of great change in data analytics, fuelled by automation. It will require shifts in how we resource, train and develop our people to take advantage of all it has to offer. It’s an exciting time, as this moment of change creates new opportunities for retailers to grow their customers’ loyalty and build their businesses using customer Data Science.

Managing Director of Category Management, North America

  Back to all content