Ten years ago, we would have laughed at anyone who said there was such a thing as too much data. For the longest time, marketers strived to secure more data, from a wider range of sources, to inform.
Now, 80% of business leaders say they have too much data to weigh up when making decisions, according to The State of Decision Making Report 2021, published by Signal AI. Instead of striving to secure more data from a wider range of sources, companies are now saturated with more data than they know what to do with.
The term “big data” was coined by O’Reilly Media’s Roger Mougalas in 2005 in reference to the vast amounts of data that was being generated by platforms like Facebook and YouTube – more data than companies would be able to handle through traditional means.
“[Big data] refers to a large set of data that is almost impossible to manage and process using traditional business intelligence tools.” – Dr Mark van Rijmenam, A Short History Of Big Data; Datafloq.com
Initially, access was the big challenge because how were businesses expected to pull in real-time data from multiple sources like Google Search, Facebook, LinkedIn, ONS and the stock exchange?
Fast-forward to 2021 and businesses have all the tools they need to access real-time data without developing custom programmes or manually downloading databases.
Technology has solved the access problem by opening big data up to businesses of all sizes. But the same solution has also created a new problem: companies now have access to too much data and lack the ability to manage it and extract meaningful insights.
With 80% of business leaders saying they have too much data to consider while making decisions, the immense promises of big data come with some challenges of equal proportions.
One of the biggest data challenges raised by businesses leaders and marketers is how long it takes to get value from analytics. According to a joint survey from Matmillon and IDG, 97% of IT, data science, and data engineering professionals say data preparation and analysis is too time consuming – and these are data professionals.
We ran into this issue ourselves here at Vertical Leap as we started to pull in more data from wider sources. Our team was spending too much time on analysis, which was taking away from the time left over for actual marketing. We were getting great insights but we couldn’t get them fast enough and implement strategies before the data started to lose relevance.
It was clear, we would never get to a point where we could handle real-time data and get instant insights through manual processes.
One of the first problems you’ll encounter when pulling data from multiple sources is conflicts in the data – especially when you’re getting related data from several different places. For example, we pull search marketing data from dozens of sources and this gives us different readings for search volumes, CTRs and minimum bids on the same keywords.
Each of the tools we pull this data from gets their numbers from slightly different audiences and they all encounter the occasional reporting issue. So, we can’t take any of the data we get from an individual source as 100% reliable. We need to have a system in place that compares the data from each source and verifies the reliability of the readings we get.
You’ll also encounter broader conflicts when you’re pulling data from unrelated sources.
Let’s say you’re getting consumer insights data from ONS and comparing this with spending behaviours from a survey published by a major credit card provider. The data you get from ONS may suggest you prioritise your marketing efforts in one area while the survey points you in another direction.
There are many reasons this can happen: different sample audiences, different questions asked, different interests of the data provider and so many more.
Who’s to say the credit card provider isn’t publishing data that promotes the use of its own services. Even when an impartial body like ONS is involved, conflicts can arise through different sample audiences and you also have to consider how closely respondents align with your own target audiences.
The issue with the first two challenges above is that you can’t simply feed raw data into a reporting system and expect to get reliable or valuable insights. Every data source comes with its own biases, gaps and weaknesses that you need to identify and manage, either by removing low-quality data or replacing it with higher-quality data from elsewhere.
This level of data management requires specialist expertise in the form of data scientists.
We’ve got our own team of data scientists here at Vertical Leap but the reality with big data is that no team can manage the volumes of data we’re talking about.
Remember Dr Rijmenam’s description of big data?
Even with our team of data scientists and analysts, we were still overwhelmed by the amount of data we were getting access to so we had to pick and choose our data sources. This allowed us to get quality, actionable insights but we still couldn’t get the full potential out of all of the data available to us – and the amount of data available was growing all the time.
This was until advancements in computer processing power and technology like AI and machine learning gave us the tools to develop an intelligent automation system that would do the heavy lifting that no data science team ever could.
The big data burden is a problem of scale and the manual input required to get meaningful insights from raw data. If we, as a data-driven marketing agency, were finding it difficult to take full advantage of all of the data becoming available to us, how is the average business expected to benefit from the promises of big data?
We had the required expertise but the automation technology available to us 10 years ago wasn’t powerful or reliable enough to manage the more time-consuming aspects of data analysis.
This all started to change in 2015 as several technological advances came together, namely computer processing speeds (and affordability), cloud computing taking off and breakthroughs in several AI technologies (machine learning, neural networks, natural language processing, etc.).
We implemented these same technologies into developing an intelligent automation system capable of collecting data from thousands of different sources, cleaning the data and finding patterns in the data to automatically produce meaningful insights for us.
We called this platform Apollo Insights.
Apollo handles the time-consuming tasks of data management so our data science team can put its time into prioritising insights from all of the data, not only part of it. Apollo puts all of the data we collect to work, meaning nothing is wasted. It automatically compares results, verifies the quality of data points and improves the value of our insights – without any manual input.
As a result, we can take full advantage of big data without getting bogged down in data management and analysis. Instead, we can focus our attention on putting insights into action and getting results from data-driven marketing campaigns without being held back by data processes.
This means our customers only pay us for the marketing actions that have the biggest impact on their biggest performance, not to sit around comparing graphs and spreadsheets.
Likewise, companies that don’t have the data setup capable of leveraging big data can come to us and use our system, whether they lack the personnel, the technology or both.
Scale is a huge part of effective search marketing – now more so than ever before. This includes the depth and comprehensiveness of the data environment you have access to, right through to the ability to make fast and evidence-supported decisions with new and ever-changing data sets.
With Apollo Insights you get these fundamentals, plus so much more through intelligent algorithmic insights in areas such as content gap analysis, action prioritisation, performance change and forecasting.
Before I joined Vertical Leap a thorough site audit would take months for me to complete. Some sites are so large they take days to fully crawl, weeks to parse the data manually, and then months to schedule tasks and action. It’s why I only ran site audits once or twice a year.
Apollo Insights continually audits our sites, picking out vital actions and prioritising them for us based on our continually-updating rules that adapt to the constant change of SEO. It pulls in data from multiple tools and sources and allows us to get to the vital pieces of work, and spend much less time in excel spreadsheets sifting through data.
We used to spend a lot of time doing audits on websites, and then doing the work. Now the audit is done automatically we can get more actual work done on a site.
The ability to be able to filter and segment data from a range of different sources in one place is key to identifying SEO opportunities and improvements. Apollo Insights allows us to do this, analyse data more quickly, and be able to prioritise changes that will make a real difference to our customers.
If you’d like to see what opportunities Apollo Insights can reveal for your business, contact us on 02392 830281 or click below and we’ll get you connected for a free trial.
Mel is a Senior Business Development Manager at Vertical Leap. She is highly-experienced at advising companies on how to use SEO and PPC to maximise their visibility online and drive leads to their business.
Website under-performing but not sure why? Our free review will reveal a list of fixes to get it back on track!
Categories: SEO