Supporting Retail’s data revolution through automation

dunnhumby’s Kyle Fugere, Head of Innovation and Ventures, and Victor Robin, Data Science Director, share their perspective on Supporting Retail’s data revolution through automation.

Over the past few years, grocery retailers around the world have been putting an increased focus on in-housing data science expertise. Many of the industry’s biggest names have been investing in the expansion of their analytical capabilities – testament to the growing contribution that data science makes to retail today.

This is great to see; the better-equipped that retailers are to gain insight from their data, the better the outcome for shoppers; something that we feel quite strongly about. This trend also represents a significant step on the road towards the true democratisation of data, signalling a shift towards an age in which analytics is an integral part of the business rather than just a silo to be drawn from or a singular resource ‘owned’ by one department.

This ongoing trend led us to the launch of dunnhumby Model Lab on Microsoft Azure last year. An evolution of technology that we’ve been using internally for some time at dunnhumby, Model Lab is a data science tool that helps retailers make faster, smarter, and more customer-centric decisions at scale.

Ali Rezvan, Retail Executive at Microsoft, explains more: “Over the past 18 months, the grocery and retail industry has had to adapt to ever-changing customer needs and purchasing habits. Ecommerce and physical stores had often been treated separately, with customers engaging differently with each channel. But in the past year we’ve seen a shift to a hybrid approach – but when more people shop online, how should retailers react?

“Whether they are at the beginning of their digital transformation or have been on the journey for some time, grocery retailers are adapting to a new data strategy, dedicating more time and resource to analysing the huge influx of data, and the new data segments and challenges this presents.

“Data scientists need their datasets to be to be succinct, federated and secure. The agility, capability and security offered by Microsoft Azure allows data scientists to build the AI and ML of the future, constructing new segments and producing new outcomes. And this is where the pedigree and expertise of dunnhumby’s customer data science comes in, unlocking data science from its traditional silo and helping grocery retailers to meet their customers’ changing demands.”

A shift away from time-consuming tasks

Data scientists can easily spend too much time on repetitive tasks, worrying about the validity of a dataset and testing a variety of techniques to prepare the data. A solution was needed to help data scientists reduce the administrative burden that comes with analysis and maximise the time they have to focus on creative aspects such as research, ensembling, and the testing of new techniques.

The solution? dunnhumby Model Lab. By automating the process of connecting and cleaning a retailer’s data, Model Lab allows data scientists to spend more time on activities that deliver real value. Automation also helps to reduce the risk that analysis will fail due to errors in the data.

When applying a huge number of machine learning algorithms to a dataset at once, each algorithm will process data differently. While some will deliver great insights, others will effectively be useless – and it’s largely impossible to predict where each will fall in advance. Model Lab not only allows data scientists to test hundreds, if not thousands, of algorithms at once, it will automatically calibrate them based on the dataset in question.

Putting the customer at the heart of every decision

For us, enabling this transformation goes beyond just efficiency or quality of life; we see this as a way to help retailers serve their customers more effectively.

Take the ability to test multiple algorithms at once as an example. In an ordinary environment, a data scientist might be able to run perhaps five or six different models on a dataset before needing to provide the business with a path forwards – and those models are probably chosen because they’ve delivered results in the past or look like a good fit for the data in question.

While this is a sensible approach in terms of efficiency, it also means that the retailer – and, subsequently, their customers – could be missing out on a different and potentially more valuable set of insights just because the algorithms required to generate those results didn’t make the cut. Model Lab tackles that problem head-on, providing the scale required to ensure that data is tested against every possible variable.

Not only does this mean deeper, more comprehensive data analysis for retailers, it means better informed and more beneficial decisions for their customers, too.

The power behind the process

All of this, of course, requires a huge amount of processing power, which is why we chose to build Model Lab on Microsoft Azure. Not only does Azure provide us with the ability to execute data analysis at gigantic scale, the vast majority of our retail clients already use Microsoft’s cloud platform for other purposes. As a result, integrating Model Lab into their workflows will be as simple as purchasing a subscription for the enterprise version of the tool when it launches later this year.

As retailers continue to dedicate more time and resource to analysing the huge amounts of data they hold, initiatives like Model Lab are our way of helping them spend more time on what really matters: putting their customers first.