As we push on in the information age you have probably grown sick of the word data or big data or data lake, data cleaning, data pipeline, the list goes on… Everyone is talking about data but very few people are using it because there are a million and one things that data does to try and stop you. If you can jump through the hoops and over the puddles that data puts in your way then it’s green pastures and blue skies.

“Without big data, you are blind and deaf and in the middle of a freeway.” — Geoffrey Moore

So, how do I get to the promised land?

Clean Data

What is clean data? The easiest way to describe clean data is as the antithesis of bad data. Bad data is an inaccurate set of information, including missing data, wrong information, inappropriate data, non-conforming data, duplicate data and poor entries (misspells, typos, variations in spellings, format etc).

As more and more information enters your organisation from different points of origin, the likelihood that mistakes will be made automatically increases. The first mistake that many organisations make is disregarding the potentially damaging impact of bad data that’s already flowing around between their enterprise applications and playing down the problem by assuming that their data is clean and accurate.

“Torture the data, and it will confess to anything.” — Ronald Coase

The solution?

It’s simple really, just remove duplicates, verify new data, update existing data and the most important one IMPLEMENT CONSISTENT DATA ENTRY. By the way, consistent data entry is not Tom and Steve being the only ones to enter a product specification in a text box because “they know how to do it”. Data should be entered into a standardised format with validation rules implemented for every data feature.

Disconnected Data

The second mistake that organisations make is to allow different parts of the business to become completely isolated from one another, as this creates departmental silos. Once these silos have been created and taken root, not only are they hard to break down, but they also encourage the use of data definitions that are different to those used by other parts of the organisation. This makes it difficult to manage and reconcile important data between teams and applications.

“If we have data, let’s look at data. If all we have are opinions, let’s go with mine.” — Jim Barksdale

The solution?
There is no one size fits all solution to this problem and unfortunately, the solution is usually a long one. It involves integrating all of your data, gaining agreement between teams on data definitions and a historical review of all your data to sort the wheat from the chaff.

No Data or Not Enough Data

Although some organisations have been collecting data for a very long time and that data may even be clean and connected. Once an organisation tries to use this data for something other than simple reporting they usually run into problems. They realise the data they have contains no information and they don’t know where to find the information they want.

The solution?
It all starts with a plan. Gather some big brains who have domain expertise and think about what questions you want to answer with your data yesterday, today and five years from now. Then for each question hypothesise five different ways you could answer these questions and the data you would need to get to these answers. For each data feature write accurate and specific definitions of what it means and for the love of all that is good in this world, try to make the data feature categorical or quantitative!

Now, there is a fourth, fifth, sixth … and two hundred and fifty-eighth reason but we will stop at three.

This all seems like a lot of effort…

It is. And it is not a one-time effort, large organisations will continually have to fight to keep their data clean, their information accessible and their insights true. However, when it comes to marketing procurement information, Axiom is here to help. With domain expertise and advanced data collection, cleaning, modelling and analytics capabilities we can optimise your procurement process, reduce your environmental impact, ensure you’re buying responsibly and save you money.