What the hell is Big Data Anyway?
What the hell is Big Data Anyway?
WHAT IT’S NOT.
“Big data” doesn’t mean just a ton of data, but that’s certainly part of the definition. And it’s not simply referring to the large enterprises collecting massive amounts of data, although that is also a key piece of the puzzle. Adding confusion, the term is often used in the negative way, such as “Big Government” or “Big Brother,” but that’s actually more misleading than defining.
Big data really designates the ever-growing interconnectedness of data, the merging of large, complex information sets which are virtually impossible to process using traditional database management and processing tools. Rather than simply storing and presenting information in the legacy database fashion, big data gives researchers and marketers the ability to predict human behavior. In a very recent example, big data was able to make a connection between rising food prices in the Middle East and the “Arab Spring” uprising.
SO HOW’S IT DIFFERENT THAN WHAT I WAS ALREADY DOING 20 YEARS AGO?
To go all geeky on the topic, the emergence of big data represents a transition from the Boolean to the Bayesian. Now, don’t let your eyes glaze over at this point. Here’s a straightforward explanation for the non- geeks among us.
Marketing intelligence and business technology was formerly ruled by Boolean logic, named for 19th-century English mathematician George Boole. This refers to the common form of algebra which is binary in nature and just represents a pair of different, absolute values – such as “1 or 0,” “true or false,” “yes or no,” etc.
Today, marketing research data and analysis is being scrutinized with Bayesian probability. Named for another English mathematician, Thomas Bayes, this technique utilizes conditional logic that can change based on the information streams. A visual metaphor may help here — think of Boolean mathematic as two parallel lines and Bayesian probability as a star, with multiple streams of data converging in the center to produce a conclusion. With the Bayesian method, you can combine data sets in practically unlimited combinations, and if you do it intelligently, you will discover new insights about your customers’ and prospects’ behaviors.
While the genesis of Bayesian statistics date back to 1763, marketers’ awareness of the concepts is fairly recent, dating from the late-50s to early-60s. It was thought that Bayesian methods would be used widely in the marketing field, but up until the mid- 1980s the approach was considered impractical due to a lack of computing horsepower. The recent renaissance of Bayesian applications is due mainly to the advancements over the last couple of decades in computational methods, hardware/software and the explosion of detailed marketplace data which is available – primarily due to the explosion of the Internet, e-commerce and social media.
THIS AIN’T YOUR FATHER’S DATABASE.
“Most people think it’s just the same old data processing as in days past but more of it,” says Kenneth Cukier, data editor for The Economist magazine and author of the book, Big Data: A Revolution That Will Transform How We Live, Work, and Think. “That’s not true. Big data isn’t just how people interact with lots of information, but about how computers can process vastly more data to do new things.”
In big data, you merge separate, but related, data sets to discover actionable insights that are not apparent when you look at the sets individually. Essentially, marketing intelligence is getting . . . well, more intelligent. “By tapping into vastly more data, we can ask new questions and do new things,” Cukier explains. “And we’re harnessing types of information that we never had before. With this, we can find correlations that before escaped our notice.”
A number of large enterprises – Facebook, Amazon, even Walmart – are using big data to improve the efficiency of their marketing initiatives. They are discovering, interpreting and utilizing behavior patterns of prospects and customers for more accurate target marketing and product positioning. As Cukier says, “In economics, when we improve transactions, price setting, and liquidity, all parties gain. Big data makes markets more efficient—so everyone stands to benefit.”
Better Homes and Gardens Real Estate brokerage was formed in 2007 specifically to gain competitive advantage through the use of big data. The real estate company, Realogy, combined with the Meredith Corporation to gain access to its Better Homes and Gardens magazine and other publications’ readers. Realogy now uses Meredith’s database of over 100 million consumers, and an average of 800 data points per consumer, to create sophisticated audience segmentations for marketing. This enables “mass marketing to the individual.” If someone in the database is looking to buy a house and they have asked a question about or expressed an interest in gardening, Realogy doesn’t just send them a standard photo of the front of the house. They send them a photo of the home’s garden area. Clearly, this heightens the efficiency and effectiveness of real estate marketing efforts.
Why is everyone suddenly so excited about big data? Because they know – or at least suspect – if they can lead their company or client to understand big data is not just more of the same data, they can utilize the many more data sets, processed with much different logic and programming rules, and the company can fly past all others in the pack.
The full range of benefits of big data are still emerging. As it moves beyond the early stages and into the mainstream, it will allow marketers to get much more targeted and predictive with their integrated marketing and prospecting. As advertising technologies and marketing techniques become more advanced, the results may start looking like something out of science fiction.
YOU’LL BE HEARING VOICES… IN A GOOD WAY.
According to the peer reviewed scholarly journal, MIS Quarterly, “The excitement surrounding big data has arguably been generated primarily from the web and e-commerce communities. Significant market transformation has been accomplished by leading e-commerce vendors such as Amazon and eBay through their innovative and highly scalable ecommerce platforms and product recommender systems. Major Internet firms such as Google and Facebook continue to lead the development of web analytics, cloud computing and social media platforms. The emergence of customer-generated Web 2.0 content on various forums, newsgroups, social media platforms and crowd-sourcing systems offers another opportunity for researchers and practitioners to “listen” to the voice of the market from a vast number of business constituents that includes customers, employees, investors, and the media.”
Unlike the standard transaction records collected from the various legacy systems of the 1980s, the data that e-commerce systems collect today from the web are far less structured and often contain rich customer opinions and behavioral guidance. Social media analytics of customer opinions, text analysis and sentiment analysis techniques are often utilized. Various analytical techniques have also been developed for product recommender systems, such as if/then rules for data mining and sophisticated audience segmentation, clustering and mapping to specific products and services to offer. “Long-tail marketing” facilitated by reaching the millions of niche markets at the shallow end of the product bitstream is now possible utilizing ultra targeted searches and specifically personalized recommendations.
KIDS – DON’T TRY THIS AT HOME
Kenneth Cukier also believes that marketing is fertile ground for big data methodologies, though most companies will likely be somewhat removed from their actual application, working with specialist leading-edge marketing firms, “I don’t think small players will work with big data directly, but they will use it,” he says. “This is in the same way that consumers benefit from Google searches, targeted coupons based on their purchasing behavior, and the like. The small players will improve what they do when their information technology systems tap big data or when they subscribe to services and databases. In the same way that Amazon recommendations are very accurate, one can imagine that small to medium businesses will be able to improve what they do too, tapping into the big data analytics that bigger firms provide.”
Big data isn’t exactly new. Market leading enterprises have been storing and analyzing multi-set data types to gain deeper insights into customer behavior patterns for years. But even some of the big guys can’t afford to store and manage enough raw, detailed big data over time in traditional data warehouses. They may warehouse, for example, the last few quarters and then store the rest of the history offline, which isn’t quickly accessible. Recovering massive amounts of older data not only presents technologic challenges but is also expensive. And, obviously, the situation is even more out of reach for small to medium businesses. In order to play on the big data field, businesses will need to develop an entirely new skill base, and the human capital cost will rapidly exceed the technology investment.
YOU NEED A PARTNER. CHOOSE WISELY OR FEAR THE CONSEQUENCES.
The only viable solution for most businesses to take advantage of the big benefits of big data is to work with an intelligent, experienced specialist in the specific marketing application of big data. A specialist that not only has expertise, qualified technology (hardware and software), and perhaps most importantly, a depth of experience in marketing strategies and tactics. High horsepower technology is not enough, you also need the outside experience and marketing centric perspective necessary to productively “massage” and apply the data and the insights derived there from. Absent that type of partner, you will only end up with very expensive, linear, Boolean and most likely wrong answers. After all, masses of good data won’t help you if you’re using faulty assumptions to process it.
There are a few strategic integrated marketing agencies across the country that practice their craft at the intersection of the best of business strategy, creative strategy and new technology. They are led by non-linear thinkers who resist the conventional wisdom of binary, yes or no analysis. When we think about the human brain and how it processes advertising, we find that great advertising and marketing creators for years have been the ones that could think in the terms of the conditional probabilities the consumers’ behavior at the point of marketing, based on the variable conditions of why the consumer is buying, who the competitors are and the “S-W-O-T” of the company delivering the services or products as well as the brand itself.
What is the value of knowing exactly, under widely variable conditions, how your customer or prospect will think and then developing marketing and advertising that reacts in a if/then manner to dispense just the right message in just the right time in just the right channel? It’s solid gold when you consider most companies struggle to survive on the typical, paltry less-than-2% return on advertising campaign investments. What is the value of doubling, tripling or quadrupling that return? Do we have your attention now? In our business as a technologically-advanced integrated marketing agency, doubling or quadrupling our clients’ marketing ROI is a daily reality.
