Sunday, January 27, 2013

BigData Interest Explosion!!





BIGdata Video

Because I frequent analytical groups, blogs and virtual cafe's, I have been inundated by the term “BIGdata”. And as the Greek poet Menander says: “Bad company corrupts good character.” my vision could be a bit distorted. I got so excited I even did two videos about it – my most watched. http://bit.ly/BIGdataJRM So when I decided to explore Google's new Search Trends I thought it would be fun to see the history of this term.

Chopping the scale off makes it look more impressive
This also gives you a nice view of how this new tool works. As you can see, BIGdata has gone from basically non-existent into the stratosphere! I was so excited to be part of this 'trending' (here's the trend on that). Well, as with most analysis, if it looks too spectacularly exciting, you might want to do some comparisons. I noticed that the scale was 0-100 and that seemed like a pretty low number... then I realized that this was a relative scale. So it can be millions and millions of searches – or not... hard to tell.
Chopping the scale off makes it look more impressive

There is an additional feature which allows you to compare a second, third or fourth (not sure the limit) search term. I thought that might give me some greater perspective on this exponential growth in BIGdata (of course it is BIG, look at how it is growing).







When we compare it with 'Database Marketing' we see that there has been a huge decline in what once was a trending phrase. I remember discovering that I could charge almost double if I talked about 'database' instead of the tired 'Direct Marketing'. Google doesn't offer trending before 2004 but this graph suggests that it may have peaked around that time. What the comparison graph shows is that even compared to a pretty 'niche' term like 'Database Marketing', BIGdata hasn't really gotten much use outside of our analytics cafe group.

As I mentioned in my video Data Science video, the first and most important character trait is curiosity... so of course I had to go deeper.

Even though most of my friends and readers will recognize the “Database Marketing” term, I decided to compare still a more general phrase to see what its scope might be.

You can see that the simple term, “Database” while declining, crushes the other two terms to such a large extent that both completely disappear. What this suggests to me is that many many more people are doing things with data and databases than are concerned about 'Database Marketing' and/or BIGdata. Which is a super great illustration of the problem.

We want to build data, work with data, pile up database but we really do not want to take much time letting marketing do much with it. Which is also why we find it far easier to make money with data when we do not work within the traditional database environment. And that 'out of the box' thinking is why we beat the best modelers in the world. So keep your attention on your work, your data and get some real data scientist to help you make sense of the back end.