8/23/2015

Shortage of Data Scientists

in 1995, when I worked as a Crystal Reports developer, there were no peers. Nobody was strictly a report writer.

There weren't many vendors. Almost no literature. Reports were an after thought. Right behind documentation.

Then, I noticed Business Intelligence spring up. Scorecards, KPI, Cubes, Dashboards.

Now, the market is flooded with report writing tools. And lots of developers. And the occupation of mere report writer is limited at best.

Data is now the hot topic. Variety of formats. Ways to manipulate and mash. Delivery mechanisms. Self Service.

The report writer has had their legs knocked out from under them. The underlying technology has grown. And a new generation is stretching the limits of what's possible.

Hadoop. Cloud. Mobile. Unstructured data. Streaming data. Statistics. Algorithms. Neural networks. Deep learning. Artificial Intelligence. Micro services. Internet of things.

The industry has matured overnight. And left the traditional report writers in the dust.

The term Data Scientist appeared out of nowhere. One who knows advanced Math, Statistics, algorithms, programming, domain knowledge, communication skills, visualizations, Hadoop, Spark along with a PhD or advanced degree.

how does a report writer accumulate all the required skills overnight. They don't. The new breed already has an assortment of require skills.

Hiring companies want experienced rock stars out of the gate. There is no established career path allowing report writers to make the leap across the chasm. The number of skills required is quite staggering.few people have PhD degrees, advanced Statistics knowledge and programming skills. It's possible to acquire these skills on your own, but that takes time, and effort. And how does one gain practical experience on the job.

Granted, there are a number of sites allowing people to gain skills such as Kaggle. Still, one does not become expert in Statistics in a few days.

There is a shortage of Data Scientist. I'd day one reason is because one enters the field out of school, not grow from the bottom up.

However, like anything in technology, in order to open up to the masses, they'll need to "dumb" it down a tad so everyday people can contribute as well.  Otherwise, the demand will exceed the supply until the schools can pump out enough qualified people.
 

No comments:

Post a Comment

Root Cause Analysis