When I got into IT in 1995, the big thing was Visual Basic.
Get the data into the Database.
Then report on it.
Fast forward, getting the data into the database seems to be the easy part.
We accumulate data from many different sources.
And we have different naming conventions and meanings across each of the DBs.
This causes great confusion when trying to roll up the data and present an Enterprise View for the executives.
Even at the level of small data, mashing data seems to be an issue.
Throw on top of that bad data, modified data, deleted data, out-dated data, bad database design.
Expand this issue exponentially for big data, I'm sure the same problems exist, just amplified.
You see, everyone is running like a chicken with their head cut off.
No-one has time to sit down and think through what they want, how they want it, how do we fix the problems, how to integrate the data in such a way to architect an Enterprise Level Data Warehouse, a single source of the truth.
There's simply no time. Everyone is busy putting out fires every minute of every day.
And when not plugging holes in the ship, they are on to creating the next system with a new set of problems.
There are plenty of opportunities surrounding data, both big and small.
Users just want their data, they have deadlines. They don't want excuses.
Which puts all the weight on the already time / resource strapped IT department.
When people say they want to put the power in the hands of the users, Self Service BI, I'm not sure what they mean.
Give them a tool that sits on top of the data, to query in real time with drill down and graphs and charts and KPI's.
To me that's similar to saying give people the ability to drive from NY to CA.
First the road needs to be built, the infrastructure to be put in place with traffic lights, stop signs, rest stops, gas stations, restaurants, etc.
The user stomps their feet and says 'are we there yet?' I have to be in California by end of the month.
Sure, IT can build the infrastructure if sufficient knowledge, time and resources.
And IT gets to maintain the infrastructure, the network, the web servers, hardware, databases, applications, etc.
A lot of effort needs to be applied on the setup in order to achieve self service.
In the meantime, IT carries the weight / blame for inability to provide to their users.
Perhaps IT will always be blamed by the business, no matter what obstacles we have to overcome and what miracles are expected of us.
I signed up for the Hortonworks Certified Associate exam last Thursday. Figured if I sign up, I'd have to take the test. And if I tak...
Saw a post today on Twitter, " Microsoft releases CNTK, its open source deep learning toolkit, on GitHub " This is big news. Be...
This blog post is in no way an attempt to steal other people's work. It's basically an conglomeration of notes from research I did...