How Big is Big Data

Posted on Posted in Uncategorized

Raj travels from outskirts of Mumbai, Thane to his office at Bandra Kurla Complex, a distance of less than 25Kms takes him more than an hour and half each way. He heads marketing at a billion dollar global firm, gets driven in a nice car, uses top of the line smartphone and is a heavy user of his iPad while in the car.Raj has been with his current telecom provider for as long as he remembers. His own as well as family mobile connections is now close to 10 with monthly expenses above Rs 10,000. All in all, a premium customer for this telecom provider.

For last five years since he shifted to Thane, he has been plagued by dropped voice calls at least at three spots while he gets driven to and from office. Of late the connection drops at five to six spots. In addition he observes his data connection keeps switching from 3G to 2G quite frequently. He recently subscribed to Internet TV option on iPad, hoping to catch business news while in car, however frequent switching causes bad experience. He has had enough and is seriously thinking of switching his telecom provider.

Priya is a busy housewife. She takes care of her husband, two kids and her mother in law at home and works for a company, providing communication training for senior executives. A job that requires intense preparation. She takes 30 minutes to reach office every day. Like Raj she too gets driven however she takes this time to be with herself. She refrains from taking any calls or going to internet, spending travel time planning her priorities for work and home. Her mobile usage (both voice and data) is therefore quite limited. Her telecom bill runs to less than a thousand each month. However she likes to dress up, believes style as an expression of self, has to look sharp at her work to be taken serious by executives that are 10-15 years senior. She spends on good clothes, accessories, salon and spa treatments. In addition she manages household expenses, all in all a premium customer for the brands she believes in.

I wrote these two examples as my case in point for the topic at hand. The telecom service provider of Raj need to know that one of his premium customer is looking for alternatives while telecom provider of Priya needs to know her spending power and her circle of influence.

We have seen systematic shift in macro economic needs of the world. Few generations back it was all about innovation and quality of products. Customers did not enjoy enough options. For suppliers it was all about R&D and quality of manufacturing. This industrial differentiation started to minimise itself with competitive products starting to look and operate quite similarly. Soon it was the service that became a differentiation. Brands vied for consumer mindset on the grade of services they offered. There are umpteen examples of brands ability to charge more just because they offered strong service differentiation. Emirates and Singapore Airlines are two such brands that I know of.

In recent past, the service differentiators too are not holding water. The shift is now towards relevance to each customer. It is all about how a brand is able to identify customers like Raj and Priya and interact with them with solutions that fit their needs. Business Analytics started to move from pure Intelligence and Planning technique and got adopted more and more into transaction management. Advancement in techniques, better processing ability and need by customers for instant gratification brought about “Big Data” paradigm.

Look at Raj and Priya again, their transactions are being tracked at each point of sale, by each credit card transaction, by their presence on social media sites, by their loyalty programs, by their mobile usage, by their interactions with call centers and so on. Technology has enabled capturing all what we do. Some in Structured format such as Point of Sale data and some in Unstructured format such as comments on social media or emails. So clearly the data we are dealing with is growing rapidly. Imagine now a brand trying to analyze each of their transactions has to deal with so many data points both in structured and unstructured formats. However, huge quantity of data is not Big data. Big data, as it is coming to be known as, is actually ability to store, process and analyze large quantity of data at very fast pace such that it meets the need for instant gratification or quick decision making. I must emphasise here that not all problems are in need of Big Data solution while most business problems are in need of Analytics.

Lets look at our examples again. In case of Raj, the service provider captures something called CDR (Call Data Record). It is a record generated whenever we do anything with our mobile device or for that matter as long as device is on. Even when we travel and switch towers (unkown to us) CDR is generated. CDR’s are therefore being created in vast quantity. For the telecom provider to see patterns of disconnects, CDR’s would need to be analyzed. Once the provider understands the problem faced by Raj, they might think of corrective action. Lets assume for this case, the telecom provider acknowledges the problem to Raj and promises to provide him with credit of Rs 10 for each call drop on his route till the time they change their network and resolve the problem for ever. So they analyze this vast amount of data (Volume), provide immediate credit with SMS to Raj demonstrating speed to keep Raj from evaluating anyone else (Velocity) and use variety of data points including Raj’s browsing habit and comments on social media to ensure complete understanding of Raj’s needs (Variety). Such business issue combines Volume, Velocity and Variety are best delivered using big data architecture.

On the other hand if the telecom provider of Priya was creating a loyalty program, it would invariably put Priya as below-average customer. However looking at her purchasing power, areas of influence and ability to create positive word of mouth to high paying customers, I would actually categorize her as an important customer. The brand would therefore need to create loyalty program that includes profile parameters and not only “spending on my brand” parameters which most of current loyalty programs are doing. These profile parameters would need access of various databases and mining social media interactions of Priya. However once the profiling is done, the brand does not need to revisit such profiling exercise at least for a year. There is need to go wide and get wider data points about Priya but there is no need to have it done on frequency less than an year. So while to some extent there could be Volume and Variety in play, Velocity is not needed in this case. In addition, since profiling is about Priya, we can extract only those data points that pertain to Priya, thereby reducing the volume of storage. Analytics processing techniques thereby would be advanced but not big data techniques.

To summarize, large volume of data by itself does not warrant use of big data techniques. It is largely defined by the business problem at hand. Brands could save a lot and deliver better outcomes if they identify business issue first and then look for technique and technology to be adopted. A five C framework of Curiosity, Capture, Curate, Crunch and Create Visualisation should be adopted for each and every business issue at hand.

This post was originally published in Analytics India Magazine.

The author is passionate about applicability of Analytics to Business Issues. Having had experience of more than 25 years in various marque capacities including being at the helm for last few years, he believes in data science and its impact on decision making. He has worked in global environments, written in important publications and is a sought after speaker. Awarded as “Marcomm Professional of the year” and “Global Achiever Award” for brand leadership, Anil these days is helping startups in Analytics space to improve their business and marketing strategy.