8/24/2014

Fiddling over #Hadoop #BigData

When I was a kid, my mother asked if I wanted to play the violin.  Sure, sounds like fun.

So we purchased one.  Brought it home.  Strummed the bow across the bridge.  Didn't sound too good.

Many company's are hopping aboard the Big Data Express.  They download the software, have an IT guy install it.  Not much value.

Soon I began violin lessons.  Learned the scales.  Practiced them over and over.  Soon learned to play twinkle twinkle.

Some company's have imported some internal data sets into Hadoop.  Perhaps in a 10 node cluster.  Perhaps created some HIVE tables, some PIG scripts, maybe even some Streaming data.

After that, I learned to play some intricate songs.  Mozart.  Complex songs with challenging note sequences.  Played concerts at the school.  Switched teachers to learn more advanced songs.  Was eventually one of the better violin players in school.

Next, these company's with 10 node clusters are parsing unstructured data, implementing HBASE.  Perhaps scheduling jobs to import data, crunch through it, resulting in aggregated output, consumed by OLAP for Web reporting in Dashboards and Reports.

Eventually, I was playing 1st chair at the Elementary School, performing concerts for parents night.  When I first got the violin, it was a piece of wood and some horse tail bow.  Didn't have much value.  It wasn't until I learned how to use it properly, to make it play songs which people wanted to hear, that it gained in value.

Integrating Hadoop into your existing Reporting structure, a hybrid solution, to compliment your Data Warehouse is the ideal scenario.  When you first install Hadoop, it doesn't have much value.  It's when you get the thing humming and crunching data to find insights for user consumption that it provides value.  Otherwise, it's a fancy open source software product sitting idle on a dozen commodity hardware devices.

It doesn't happen overnight.  Learning the violin or learning Hadoop.  I'm no expert at Hadoop at this point, perhaps some advanced twinkle.

And there's plenty of wrong ways to play the violin or Hadoop, which sound horrible.  I imagine that once you get them in tune, you could be playing wonderful melody's for your organization.

Have a listen to one of the better know fiddle players:

Itzhak Perlman

Babalon