As with many other industries, automation has the potential to deliver a radical shift in the way that data scientists approach their work – and a huge boost to the value that they can deliver. In this article, Vijay Balaji Madheswaran – Director of Applied Data Science for dunnhumby APAC – looks at automation’s potential contribution to the future of data science, and the obstacles in the way of wider adoption.
“What do you do for a living?” It’s one of those questions that we all get asked, and the answer to which can change wildly depending on who happens to be asking and the level of insight we think they might have into our particular career path.
For data scientists in particular, what we “do for a living” can be particularly hard to explain. Data science is a broad label that disguises a multitude of tasks and skills, many of which require deep specialisms. That said, I think that most of the answers you’d get when asking a data scientist what their job entails would fall into one of the following five categories:
While I think those five areas are a fair encapsulation of what data scientists do, I also think that something is missing: automation. Most data scientists probably wouldn’t include automation as a core part of their job, but it’s increasingly vital – particularly in terms of measuring ROI.
Much like data literacy – something that has the potential to fundamentally redefine what it means to be a data scientist the more ubiquitous it becomes – automation looks set to have a transformative impact on data science. Crucially, the potential benefits go beyond just making processes faster and more efficient; when it’s employed effectively, automation can aid in everything from employee morale through to the creation of competitive advantage. Here’s how.
Automation is the enemy of boredom
Doing the same thing over and over again is inherently boring. Moreover, it’s a waste of time, effort, and – most dangerously – the intellectual capital that you’ve invested in. Automation prevents talented data science professionals from having to repeatedly solve the same problems or carry out the same tasks. As a result, it frees them up to deliver greater value and do more engaging work.
Computers can help people make better decisions faster
If you’re a Retailer running 15,000 promotions every week, putting the analysis of those offers into human hands is likely to be overwhelming no matter how talented your team. By automating that analysis, and asking a machine to provide a recommended course of action accompanied by projected results, a human team can make much smarter and clearer decisions.
Automation can help you replicate success and expand
Say you have a team of 15 people working on the data science that powers 1,000 campaigns a year. By automating some of the processes behind their work, you can bring that team down to just two. That frees up a huge amount of talented resource to find a new problem to solve, expand into new territories, or progress new ideas – all at an identical cost.
Speed is competitive advantage
Decisions delayed are decisions unmade. The faster that a business makes decisions, the faster it can outmaneuver its competitors. Automation gives organizations the opportunity to drive competitive advantage, a factor that will only increase in importance as the application of AI becomes more common.
As with AI, the sheer volume of discussion around automation means that it can be easy to assume that just about every business everywhere has already charged ahead and automated the vast majority of their processes. That isn’t the case, unfortunately, and nor will it be unless we can overcome some of the fundamental barriers standing in the way of wider adoption.
The first of those relates to sheer awareness. Many organizations – and people – just aren’t cognizant of the benefits that automation can bring. And even those who are can still be uncertain about how best to proceed; successfully, at least. Uncertainty, of course, also makes it difficult to justify the expenditure required to drive automation forward, particularly when most organizations already have large technical debts that only increase their resistance to change.
We also need to take into account the human factor here. As referenced above, automation inherently reduces the amount of human resource required to solve problems. That’s less of an issue in data science, where expertise can be diverted elsewhere. Nonetheless, the “humans vs. machines” association is undoubtedly hard to shake, something that can also manifest in the belief that human analysis will somehow end up being more “creative” than that delivered by a machine.
Finally, for some, automation simply won’t be a priority. They may believe that their needs change too fast to benefit from it, or struggle to find the incentive required to take it into serious consideration. That, of course, is true of anything that alters the status quo.
For those organizations that do want to embrace the potential of automation, focusing on a few priorities can help to make that a reality:
Vijay Balaji Madheswaran is Director of Applied Data Science for dunnhumby APAC. Focused on investment, partnership, access, and customization, Vijay is passionate about helping Retailers and Brands realize the full value of their data.
Cookie | Description |
---|---|
cli_user_preference | The cookie is set by the GDPR Cookie Consent plugin and is used to store the yes/no selection the consent given for cookie usage. It does not store any personal data. |
cookielawinfo-checkbox-advertisement | Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category . |
cookielawinfo-checkbox-analytics | Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Analytics" category . |
cookielawinfo-checkbox-necessary | This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary". |
CookieLawInfoConsent | The cookie is set by the GDPR Cookie Consent plugin and is used to store the summary of the consent given for cookie usage. It does not store any personal data. |
viewed_cookie_policy | The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |
wsaffinity | Set by the dunnhumby website, that allows all subsequent traffic and requests from an initial client session to be passed to the same server in the pool. Session affinity is also referred to as session persistence, server affinity, server persistence, or server sticky. |
Cookie | Description |
---|---|
wordpress_test_cookie | WordPress cookie to read if cookies can be placed, and lasts for the session. |
wp_lang | This cookie is used to remember the language chosen by the user while browsing. |
Cookie | Description |
---|---|
CONSENT | YouTube sets this cookie via embedded youtube-videos and registers anonymous statistical data. |
fs_cid | Set by FullStory to correlate sessions for diagnostics and session consistency; not always set. |
fs_lua | Set by FullStory to record the time of the user’s last activity, helping manage session timeouts. |
fs_session | Set by FullStory to manage session flow and recording. Not always visible or applicable across all implementations. |
fs_uid | Set by FullStory to uniquely identify a user’s browser. Used for session replay and user analytics. Does not contain personal data directly. |
VISITOR_INFO1_LIVE | Set by YouTube to estimate user bandwidth and improve video quality by adjusting playback speed. |
VISITOR_PRIVACY_METADATA | Set by YouTube to store privacy preferences and metadata related to user consent and settings. |
vuid | Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos to the website. |
YSC | Set by YouTube to track user sessions and maintain video playback state during a browser session. |
yt-remote-device-id | YouTube sets this cookie to store the video preferences of the user using embedded YouTube video. |
yt.innertube::nextId | This cookie, set by YouTube, registers a unique ID to store data on what videos from YouTube the user has seen. |
yt.innertube::requests | This cookie, set by YouTube, registers a unique ID to store data on what videos from YouTube the user has seen. |
_ga | The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. |
_ga_* | Set by Google Analytics to persist session state. |
_gid | Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously. |
_lfa | This cookie is set by the provider Leadfeeder to identify the IP address of devices visiting the website, in order to retarget multiple users routing from the same IP address. |
Cookie | Description |
---|---|
aam_uuid | Set by LinkedIn, for ID sync for Adobe Audience Manager. |
AEC | Set by Google, ‘AEC’ cookies ensure that requests within a browsing session are made by the user, and not by other sites. These cookies prevent malicious sites from acting on behalf of a user without that user’s knowledge. |
AMCVS_14215E3D5995C57C0A495C55%40AdobeOrg | Set by LinkedIn, indicates the start of a session for Adobe Experience Cloud. |
AMCV_14215E3D5995C57C0A495C55%40AdobeOrg | Set by LinkedIn, Unique Identifier for Adobe Experience Cloud. |
AnalyticsSyncHistory | Set by LinkedIn, used to store information about the time a sync with the lms_analytics cookie took place for users in the Designated Countries (which LinkedIn determines as European Union (EU), European Economic Area (EEA), and Switzerland). |
bcookie | LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognise browser ID. |
bscookie | LinkedIn sets this cookie to store performed actions on the website. |
DV | Set by Google, used for the purpose of targeted advertising, to collect information about how visitors use our site. |
ELOQUA | This cookie is set by Eloqua Marketing Automation Tool. It contains a unique identifier to recognise returning visitors and track their visit data across multiple visits and multiple OpenText Websites. This data is logged in pseudonymised form, unless a visitor provides us with their personal data through creating a profile, such as when signing up for events or for downloading information that is not available to the public. |
gpv_pn | Set by LinkedIn, used to retain and fetch previous page visited in Adobe Analytics. |
lang | Session-based cookie, set by LinkedIn, used to set default locale/language. |
lidc | Set by LinkedIn, used for routing from Share buttons and ad tags. |
lidc | LinkedIn sets the lidc cookie to facilitate data center selection. |
li_gc | Set by LinkedIn to store consent of guests regarding the use of cookies for non-essential purposes. |
li_sugr | Set by LinkedIn, used to make a probabilistic match of a user's identity outside the Designated Countries (which LinkedIn determines as European Union (EU), European Economic Area (EEA), and Switzerland). |
lms_analytics | Set by LinkedIn to identify LinkedIn Members in the Designated Countries (which LinkedIn determines as European Union (EU), European Economic Area (EEA), and Switzerland) for analytics. |
NID | Set by Google, registers a unique ID that identifies a returning user’s device. The ID is used for targeted ads. |
OGP / OGPC | Set by Google, cookie enables the functionality of Google Maps. |
OTZ | Set by Google, used to support Google’s advertising services. This cookie is used by Google Analytics to provide an analysis of website visitors in aggregate. |
s_cc | Set by LinkedIn, used to determine if cookies are enabled for Adobe Analytics. |
s_ips | Set by LinkedIn, tracks percent of page viewed. |
s_plt | Set by LinkedIn, this cookie tracks the time that the previous page took to load. |
s_pltp | Set by LinkedIn, this cookie provides page name value (URL) for use by Adobe Analytics. |
s_ppv | Set by LinkedIn, used by Adobe Analytics to retain and fetch what percentage of a page was viewed. |
s_sq | Set by LinkedIn, used to store information about the previous link that was clicked on by the user by Adobe Analytics. |
s_tp | Set by LinkedIn, this cookie measures a visitor’s scroll activity to see how much of a page they view before moving on to another page. |
s_tslv | Set by LinkedIn, used to retain and fetch time since last visit in Adobe Analytics. |
test_cookie | Set by doubleclick.net (part of Google), the purpose of the cookie is to determine if the users' browser supports cookies. |
U | Set by LinkedIn, Browser Identifier for users outside the Designated Countries (which LinkedIn determines as European Union (EU), European Economic Area (EEA), and Switzerland). |
UserMatchHistory | LinkedIn sets this cookie for LinkedIn Ads ID syncing. |
UserMatchHistory | This cookie is used by LinkedIn Ads to help dunnhumby measure advertising performance. More information can be found in their cookie policy. |
yt-remote-connected-devices | YouTube sets this cookie to store the video preferences of the user using embedded YouTube video. |
_gcl_au | Set by Google Tag Manager to store and track conversion events. It is typically associated with Google Ads, but may be set even if no active ad campaigns are running, especially when GTM is configured with default settings. The cookie helps measure the effectiveness of ad clicks in relation to site actions. |