Understanding evolving customer needs in the age of AI and data abundance

More customer data than ever before, hundreds of ways to use AI to transform data into potential insights, and dozens of channels to reach each consumer along their shopping journey. This may sound like a blessing to retailer and brand decision makers, but their job just got significantly harder.

In our new blog series, Nate Kovarik and Leo Nagdas will discuss the key trends coming out of dunnhumby ventures’ recent Retail Innovation Forum 2025 in London, starting with demystifying how decision makers can use new data innovations to better understand evolving customer needs.

Here are six themes to incorporate into your customer understanding strategies:

1. Best tech providers are seamlessly augmenting, not disrupting, shopper journeys
    
   The ability to seamlessly gather insights along the consumer journey, especially immediately before and after the purchase, is changing the legacy surveying model. Agentic web scraping, advancements in linguistic science, machine learning, and creative in-store integrations are providing us with the tools needed to passively uncover insights from online behaviours (reviews, comments, search terms, Gen AI prompts) and more accurately target specific segments of most relevant consumers at scale.

   Generational changes in consumers (especially Gen Z+) and trends-driven buying behaviours are further emphasising the need for continued monitoring of changing consumer needs, with increased frequency and depth.

   Thankfully, advancements in AI are enabling qualitative research at quantitative scale. Natural language processing and machine learning are being applied to surface patterns, motivations, and unmet needs, providing depth and agility in one integrated system.

2. Customer-conscious in-store signals drive differentiation opportunities
    
   Hardware-based augmentation of consumer journeys inside the store to gather critical in-person signals is transforming the level of data available. This is a no brainer given that 80-90% of purchases still happen in the physical store (varying by research source), engagement is proven strongest nearest to point of purchase, and economics of in-store innovation do not rely on excessive activation tax (e.g. offsite retail media).

   Retailers are using in-store technologies such as beacons, routers, LIDAR, foot traffic tracking, audio capture, and computer vision, to track in-store behaviours such as dwell time, foot traffic, and product interaction. These signals are contributing to development of ‘digital twins’ which possess the ability to provide predictive improvements of in-store experiences, measure retail media performance, optimise operations, and objectively test in-store innovation.

   As a retailer gathering these signals, it is important to keep the customer in mind – just because collecting certain type of data is privacy-compliant, it does not mean that doing so won’t have an adverse effect on customer perception, hence the need for customer-consciousness rather than merely privacy-compliance. The market is still largely in exploration mode but making the investment today can reap meaningful long-term returns.

3. Behavioural signals without context fall short of decision needs
    
   Combining data sets to ensure behavioural first party signals (often the ‘what’) are appropriately contextualised with enriched attributes (often the ‘who’ and ‘why’) is proving critical to transform data into insights, and insights into decision tools. It is not enough to look at transactions the way the consumer does and infer the rest. A true ‘digital twin’ of the consumer needs to incorporate shopper, product, and engagement attributes – for example, how consumers see products (e.g. via enriched product attributes such as diets), journeys (e.g. via geolocation, delivery/instore mix, and discovery behaviours), spaces (e.g. via instore video), cross-retailer behaviours and drivers (e.g. competitive differentiators of locations, brands, and retailers), shopper psyche (e.g. via linguistic or emotional analytics), and other features.

   The complexity (and opportunity) here lies in the creation, maintenance, and matching of data hierarchies that talk across organisation teams and between data types – i.e. product, transaction, shopper attribute, and other signal data needs to be harmonised via standardised hierarchies and then matched across them.

4. Increased consumer variability requires deeper, community-based research
    
   Behavioural differences are widening between consumers, as defined by generation, psychographics, demographics, geography, habit, and individual differences. It is therefore increasingly important to understand the level of customer needs variability which requires each retailer and brand to build a customer data strategy that follows an increasingly customised and layered approach – starting with the basic profiling of their loyal and non-loyal customers by persona type, then diving into behavioural drivers and need states using new, and continuing towards deeper community research opportunities (e.g. top CPG using Discord groups for Gen Z customer research).

   Increasingly, real-time customer feedback is being used to drive decisions across operations, merchandising, and marketing. Instead of relying solely on periodic surveys, retailers and brands are embedding feedback mechanisms into the shopping journey, capturing customer sentiment at the point of interaction and within customer communities that matter the most.

5. AI provides a helpful means of coping with customer signal overabundance
    
   While AI applications are not enough to replace purpose-calibrated actionable insights that drive human expert decisions, they are a helpful means of coping with overabundance of customer signals, as well as an enabler of shortening signal response times that provide real-time or fast-start actions. With digital and physical channels blending more than ever, organisations are also recognising that customer behaviours are increasingly fluid. The ability to collect and synthesise these signals in an increasingly real time fashion – across platforms, store formats, and touchpoints – is becoming foundational to effective decision-making. Machine learning provides proactive sense-making across different data types which drives incremental observations to human decision experts which they could otherwise miss.
6. From data silos to ‘signal networks’
    
   One of the most important takeaways is that understanding the modern customer requires collaboration across the ecosystem. Brands, retailers, technology providers, and researchers must work together to connect fragmented data into cohesive insights. The future lies in building robust ‘signal highways’ – near real-time data pathways that power better decisions across merchandising, media, and customer experience. As these networks become more sophisticated, the ability to deliver truly connected, insight-driven strategies will become a major competitive advantage.

   This principle of alignment between retailer, brand, and consumer utility is also a means of accelerating adoption of innovation. Prioritisation of innovation which brings value to all three stakeholder groups, and which is in turn co-funded via partnership between brands and retailers, presents a fast-track to not getting buy-in around innovation, but also which often brings harmonisation of siloes assets into improved consumer experiences.

Retail’s next chapter will be defined by how well businesses listen to and act on the evolving needs of their customers – not occasionally, but continuously, and not reactively, but proactively. The tools exist. The expectations are clear. The opportunity to differentiate yourself is now.

Innovation insights powered by the dunnhumby Retail Innovation Network

These insights were collected by gathering feedback from our Retail Innovation Network, the fastest growing global open innovation programme for retail technology leaders. We would like to thank contributing panelists and showcase participants, including James Tamblyn (dunnhumby), Timo Lassak (REWE Digital), Chuck Ennis (Becausal AI), Paul Wilkinson (former Deliveroo), Hakan Yurdakul (Bolt Insight), Gareth Johns (TruRating), and Iman Nahvi (Advertima Vision), Dmitriy Pavlov (Stitched Insights), and Cem Kent (Harmonya).

To receive insightful thought leadership and engagement, explore joining the Retail Innovation Network, the fastest growing open innovation networks in retail, for free at dunnhumby.com/ventures and find out more about the next Retail Innovation Forum in Bentonville at dunnhumby.com/retail-innovation-forum.