One data set to rule them all

One of the best things about working in the digital marketing and analytics industry is that almost everything is measurable – you want to check how many pageviews your blog titled “Are We Living In A Matrix?” received from users living in Wagga Wagga using Google Pixel 5? – you can!

As beneficial as this extensive data availability is for analysts, it can also cause frustration. Typically, organisations collect data across various platforms – for example, an organisation running a simple lead generation campaign on Google Ads is probably collecting data across three different platforms – Google Ads, an Analytics platform like Google Analytics, and a CRM platform like Salesforce. Now, throw in additional campaigns and other channels like Microsoft Advertising, email, etc., and you end up with a plethora of similar sources.

Below is a very limited list of platforms and technologies (by organisation usage) we usually come across when working with clients –

This fragmented nature of data, where various platforms don’t often speak to each other, is what causes the frustration I spoke about earlier.

Now, because there are so many platforms in use, there’s always going to be data discrepancies –

• As an agency, we usually use Google Analytics (or another Analytics platform) as our primary reporting tool, but for a c,lient it could be Salesforce or something equivalent.
• All these platforms will usually have different tracking, data collection and attribution methodologies.
• Not all platforms collect the same metrics – for example, Google Analytics might be collecting lead submissions, but actual sales data is being collected in Salesforce.

To get the most out of all this data available and to minimise any discrepancies, organisations must work towards unifying all their data into a single source of truth. It’s not usually an easy undertaking, but the benefits can be significant –

• Increase efficiencies when everyone is looking at the same data
• Reduce friction between various parties or departments
• Optimise campaigns towards metrics that matter – for example, optimise towards sales, not leads
• Improve your budget allocation and forecasting
• Enables proactive decision-making

So, how do we get there? Captain Planet isn’t my favourite cartoon series, but the message of unity is quite relevant here. As in the series, with all our data sources combined…

The most important step to get to this “Captain Planet” scenario is identifying the “key” that can be used to unify or join all these fragmented data sets. This key is a unique identifier that must exist across all the data sets you are looking to join. There are various approaches available, but I’ll provide a couple of the most often-used ones.

Google Analytics generated unique identifier – This is probably the most common approach if you are using Google Analytics on your website. It can be tested before you move on to something bigger and better. Every user that comes to a website with Google Analytics is assigned a unique identifier called “Client ID”. I won’t go into the details of what the Client ID is, but you can read more here.

This Client ID can be captured using Google Tag Manager and sent back into Google Analytics as a custom dimension. This same Client ID will then need to be captured in other platforms being used across the organisation. To keep things simple, if your website uses Gravity Forms (or any other provider), you can configure the forms to capture the Google Analytics Client ID as a hidden field and send it back along with other form fields.

Now that we have the Client ID being captured in both platforms, we can download the relevant data and join using reporting tools like Excel, Data Studio or Tableau.

Here are a few caveats to keep in mind when using this solution –

• Google Analytics cookies are set in first-party context and should be available across all devices and browsers, but this solution depends entirely on a third-party platform and is subject to risks associated with any changes that Google makes
• The Client ID is unique per device, per browser, meaning it’s not going to be unique per user – for example, someone browsing the same website on a laptop and mobile will have two Client IDs associated with them

Client-generated unique identifier – This is the more accurate and long-term solution that clients can and should be considering. Instead of relying on Google Analytics to generate the unique identifier, clients can configure their websites (or other platforms). This is not going to be easily possible for every website, but where users have to login or provide a customer number of some kind, can leverage this solution straight away.

The basic idea remains the same: capture the website-generated unique identifier as a custom dimension in Google Analytics and other relevant platforms for this exercise. Once this unique identifier is available across all the data sets you want to join, use Excel, Data Studio, etc. to create a unified data set. Across similar projects that we’ve undertaken for our clients, we’ve seen a “match rate” of over 95%, meaning over 95% of the records across the data sets were matched to each other. There is always going to be some margin for mismatch or error due to various technical issues.

Because this solution relies on a website or client owned platform generated identifier, which is typically going to be the same for every user, it can overcome the cross-device, cross-browser issues that Google Analytics Client ID can’t. And the client retains full ownership of this solution which reduces the risks associated with relying on a third-party platform.

There are considerable benefits of either solution – if you are using Google Analytics and have data collecting in fragmented silos, consider using the Client ID solution if possible as a starting point and move to a website or system generated solution at a later stage. Using this unified data set for reporting and optimising campaigns will open opportunities that would otherwise remain unexplored. You can work out customer lifetime value by channels, ROI analysis by customer type (new vs. returning), churn rate by city and so on – valuable and actionable business insights.

So, with unified data available, if you want to check how many users from Wagga Wagga using Google Pixel 5 actually signed up to either the Blue or the Red package after reading your blog titled “Are We Living In A Matrix?” – you can!