Get Real BI Through ETL Implementation
A Texas hospital spent $62 million adopting a promising, breakthrough business intelligence (BI) tool, but four short years later, discontinued the system that was supposed to drastically improve cancer care. Whether it was because of botched implementation, an overhyped, buggy product, or internal politics, the end result was a gut-wrenching money pit.
Upon hearing the doleful news that big data didn’t pan out for big business, I’ll bet all those CEOs who were ready to pony-up for new business intelligence systems, quietly put the company card back in their wallet.
CEOs across the world protected themselves and their companies by deciding not to adopt the newly available, cutting-edge cross-platform data collection and centralization systems.
But did they shoot themselves in the foot by staying with their existing disjointed, clumsy, error-prone, high-maintenance data tracking BI systems?
The Problem with Data and Marketing Business Intelligence
We live in an era impressed by data. Impressed … but overwhelmed.
We can pull granular data out of everything from our web pages, apps, social media, point of sale system (POS), QSR kiosks and customer relationship manager (CRM), to name a few. We have access to all the data we could ever want, but the problem with data is not that there is too much of it—the problem with data is that we’re blockheads at capturing the data from separated systems and translating the disparate data into tidy, usable dashboards. We should be able to synthesize all that beautiful data and provide succinct insight that fosters business growth and aids decision-making.
We have data from our website, app, CRM, and POS, so why can’t all this data feed into one pretty dashboard that shows us where the business is losing, where it’s winning, which pots to put the money in, and what can be done to improve growth?
It’s not that easy to integrate data from disparate systems.
Systems are notorious for not playing nicely with each other. In effect, it takes an expert, or team of experts more likely, to make everything synchronous—and even then, it’s a game of whack a mole. None of the systems were built to easily integrate with every system out there. The systems are continually updating, and when one system updates, the connecting one doesn’t update automatically—so now that connection is broken. When another system gets updated, and the corresponding data path isn’t updated, suddenly data collection is lost—and it goes on and on. It’s a data geek’s daily nightmare.
Data integration and subsequent BI analysis issues are high-level problems. Is the only solution to hire an expensive team of data experts to continuously monitor data extraction and system integration?
There is another way, although it’s not free and it is still a mix of man and machine—it’s a more sophisticated, automated form of an ETL function.
What is ETL?
E = Extract
T = Transform
L = Load
ETL is a tool used to pull data (extract) from any of your existing systems (website, app, CRM, POS etc.) and reformulate the data (transform) before it dumps it (load) into an analytic dashboard. The ETL tool is layered over your existing system, enabling it to communicate with other systems. Typically, the data from your system isn’t readable by another system, nor is the data from your system formatted in exactly the same way as the second system—so the second system can’t read your data.
Think of ETL as what happens when you create an Excel spreadsheet. When you create a spreadsheet, you take a bunch of disparate data and organize it into tidy columns you can use to inform business practices. You can sort and label the data any way that makes it most usable. ETL does basically the same thing—it collects data, sorts and labels it properly, and then gives it to another system for display. The collection is extraction, the sorting is the transformation, and the giving is the loading.
How ETL Works
Google Tag Manager (GTM) is a commonly used ETL—GTM extracts information from your webpages, transforms the code into identifiable data and loads it into Google Analytics.
Let’s say you sell shoes online, and at the end of the month, you’d like to see how many website visitors you had, your total sales, and how many people clicked on your branding video.
Using GTM, you tell Google what types of data you want collected from your website (video clicks, website visitors etc.). GTM then extracts the data, arranges it into the format Google Analytics can read, and then delivers it into Google Analytics to facilitate BI.
The whole point of leveraging data collection systems for your business is to gain factual evidence about your business to help you make educated business decisions. When working with the destination data, you can see what’s working, what needs more budget, and what activities can be cut.
Why Do We Care About ETL?
Why do we care about ETL? Well, it helps us understand the problems that crop up in our integrated systems that use ETL. When we are shopping for scalable system integration solutions for our business, it’s better to have a basic understanding of the ETL process—how things work and what the limits and drawbacks are within ETL system integrations.
The Problem with ETL
It’s amazing to live in an era with the technology to track everything known to man, from simple things like phone numbers and click counts, all the way to blood sugar levels and granular medical drug trial data.
But data and systems always seem to cause problems, don’t they? ETL is no different.
ETL is a beautiful thing, but the problems with ETL system integration are:
- ETLs must be manually set up at the source (someone has to put the codes into the website, app, etc)
- ETLs break because either the data source or destination updates and the data can no longer transfer (so your dashboard is showing no sales, but you actually had sales)
- Because ETLs break, you must have staff dedicated to monitoring and maintaining the system integrations
- When your business starts to scale, the number of data points and systems needing connection can be infinite—it will become overwhelming for one person or even one team to implement and control system integrations
- Some systems just don’t play well together and can’t be integrated (as of yet)
ETL Solutions for Medium and Large Business
By manipulating source data, ETL enables your systems to talk to each other, but as you scale your business, you adopt more systems and want to track an increasing number of data points—and the system integrations become a nightmare.
When there are too many ETLs functions for staff to create and maintain, most of the work must be relegated to a machine.
Why don’t businesses simply use the ETL-automation-tool from day one?
Most small- and many medium-sized businesses don’t need an ETL automation tool. They can develop system integrations in-house or with the help of an integrated marketing and advertising agency. It’s not perfect, but it works if you hire the right experts. Using a tool to automate ETL functions probably doesn’t make sense until your business starts to scale to the point where automated ETL integrations are the most efficient and effective choice.
When business begins to scale, the integration staff costs increase—and at some point, the cost and ease of an automatic ETL integration tool becomes necessary and more cost-effective than hiring integration staff ad infinitum.
Graduating to ETL Software
Congratulations, your business has scaled, you’re running a medium - to large-sized business—and you’ve got big problems.
On the one hand, you have a ballooning team of systems integration specialists, but on the other hand you have business analysts complaining that the data is no good.
You need your marketing business intelligence team to get ahold of the data they need to effectively analyze the market and provide you with the insight and recommendations you require to make crucial decisions that have dramatic ramifications.
It’s time for you to graduate to a tool to help your team work effectively with your data.
When your business adopts ETL software, you eliminate significant labor costs because the tool takes care of the communication between your data points and your data destinations. But the biggest selling point is that when any system updates, the tool will handle the changes without human involvement—all system integrations will run smoothly on their own.
However, robots haven’t taken over the world quite yet, so you still need humans to complete two functions:
- Re-tag your data origination points (website, app, CRM, etc.) for all the items you want to track
- Analyze the data within the destination dashboards to provide actionable insights to improve decision-making
Like most systems, in order to get things going, someone has to set things up. However, tagging the data origination points is a one-time job if you use consistent naming convention strategy. Once you’ve implemented the tags, you won’t need to redo that process, even if you add additional destination points.
And, as always, smart businesses don’t rely solely on data for decision-making. The human mind is still needed to bring practical intelligence and big-picture-thinking into the mix when creating smart, actionable business decisions.
Using Big & Little Data Without Killing Your Company
When you have properly functioning ETL system integrations, you have access to a lot of good data. As you’ve read, getting good data is often an uphill battle. Unfortunately, it’s only half the battle. The other half is knowing how to use data to make smart business decisions.
Just like connecting your disparate systems isn’t easy, neither is analyzing the data and using it to grow your business. Data comes quickly and incessantly. Sometimes you should act on the data you received in the last hour and sometimes that is the most dangerous thing you can do for your business.
Data is like a loaded gun—if you don’t know what you’re doing, you can shoot yourself in the foot with it.
When our clients log in to their dashboard to see how their new Google AdWords campaign is doing, we often see a knee-jerk reaction to that day’s numbers. If today’s results say that the campaign selling pink shoes is doing 33% better than the ads selling blue shoes—the client wants to immediately stop production on blue shoes, cut the ad, and put the full ad budget into the pink shoes.
Clients tend to panic when they see data. It’s totally natural—we want to make smart business decisions and protect ourselves from losses. Data is supposed to enable us to pivot quickly, right? Yes. But we need to remember that we can’t solely rely on data when making crucial business decisions. With 30-years’ experience in the marketing and advertising world, we’ve learned to look at today’s data, but also look at the data for the week and this time last year. Not to mention taking a peek at how the shoes and all your other products are faring on social platforms, in store, and with your influencers. Looking at the big picture and the overall integrated marketing and advertising strategy is key when analyzing data.
You have to learn how to dance with data—when to fast dance, when to slow dance, when to let data lead, and when it’s the right time to take the lead yourself.
Business intelligence takes part man and part machine to integrate disparate data systems, parse the data, and gain smart, forward-thinking actionable insight to enable growth. There is a lot of nuance to integrating data with decision-making. It takes a team of experts to make it happen, but when executed properly, you’ll be happy you made the investment instead of fearing the change that new data points bring to business.
FabCom is a high-tech marketing and advertising agency that embraces high-tech data integration to provide next-level marketing business intelligence. We love to help brands integrate systems, holistically analyze numbers, and formulate true BI that creates exponential growth.
