7/01/2015

Traditional Roles Are Feeling the Pressure to Adapt to Survive

It used to be simple.

We had a DBA and a Developer.

The DBA maintained the database.  And the Developer wrote T-SQL and create reports.

And along came the Data Warehouse, which required a mix of table creation, writing queries to populate data known as ETL, and reporting / Dashboards.

This role became the Business Intelligence or Data Warehouse developer.

And then a distributed architecture was created, known as Hadoop, and some people dabbled there.

Then we had the ability to work in the Cloud, in the Microsoft world, Azure.  This allowed for remote development, hybrid solutions as well as fault tolerant redundant backups as well as ability to scale on demand.

Along with other technologies like PowerShell, Service Broker, BIML, Mobile BI, PowerBI, Team Foundation Server, Machine Learning, DocumentDB database, Graph Database, and Artificial Intelligence.

So it's no wonder that many traditional Microsoft DBAs and Business Intelligence Developers have expanded into these new technologies.

We have dedicated conferences for Business Analytics now.  And a plethora of Big Data and NoSQLDB conferences.

The world of data has splintered into many subtopics.  To some degree, the traditional roles have been watered down.  You can still make a descent living, but the need to grow one's skill set is increasing.  Or get left behind.

And we're just talking about a specific vender here.  There are dozens of venders each with their own flavors.

The world of technology is changing.  Especially the world of Data.  I would say it's very difficult to be great at everything, there's just too much to learn and it changes constantly.

Therefore, you have to take a reading of the current trends to see if they align with your current skill set.  And then see if that's the direction you'd like to be in a few years and make the necessary adjustments.

It's not to say that any one technology is the best, because that will change over time. 

One methodology is build upon your existing skill set, continually adding the technologies in demand.

Or you could side step completely into a new realm.

It basically comes down to each person's particular path and you get to decide which direction to take.

Traditional roles are feeling the pressure to adapt to survive.  Our careers are like marathons.  You don't want to waste all your energy on the first few miles.  Save some for the rest of the journey.  And make good decisions on how to structure your career so that you'll find work when the wind changes direction.  And it changes quite often.

6/30/2015

Paying for Data Protection

Data breaches are everywhere.

Here's a list of some recent ones.

The only one I was personally affected by was the Target breach.  My credit card company suggested I close the card and open a new one.  I waited.  Then a $2,300 purchase was attempted, and declined, for a bunch of pens ordered online.  They closed the credit card and re-issued new ones.  And all the auto payments tied to the old account had to be changed.

I did some research and found a good site from Equifax.  They are one of the three credit bureaus that keep track of people's credit.  They have a list of products to choose from, listed here.

For me, parting with money is almost painful.  But I signed up for the family plan, month to month, no contract.  They're supposed to inform the other two credit bureaus to watch for activity.  We'll see what happens.

Also, there's a web site to let you know if your information has been pawned, here's the article.  And here's the URL https://haveibeenpwned.com/ where you can check your emails as well as sign up to be notified if something happens.

Fraud has been around since the beginning of money, with fool's gold.  In today's world, the level of complexity surrounding theft of finances and identity is increasing and we have to take preventive measures against becoming a victim.

Even if that means paying a monthly fee for protection.  Mafia style.


6/24/2015

Business Intelligence Utility Player

One of the good things about consulting is end to end programming.  Typically you're involved in the entire life cycle of the project.

From scoping a project out, to creating a Statement of Work, to designing the data model, the ETL, the OLAP cube, the reports, the Dashboard, perhaps integration with SharePoint, Active Directory, Security Groups, documentation and moving to production.

I prefer playing the role of Utility Player.  I like to jump from task to task, wherever needed on any particular day.

A few weeks ago, I was working on a project, much of the ETL was already done, dozens of reports were in the process of passing Quality Assurance.  I mentioned to the client that I could speed up the Crystal Reports.  They asked how.

So they gave me a sample report, which was running slow, over 8 minutes.  I did a few tweaks, all withing the Crystal Reports developer - no changes to the SQL pointing to the Business Object Universe, the report ran in 4 minutes.  Shaved off 1/2 the time.  Then I did a few more tweaks, and it ran in under 2 minutes.  Shaved another 50% off.

The goal was to run under a minute, but the Production server should have more processors and memory, so I'm guessing it will meet the 1 minute requirement.

Now all the reports will get the same tweaks, and the run times should be much faster.

And then on to the next task.

Service Oriented Business Intelligence Consulting

When scoping out a new project, you have to work with the client or customer to discuss the features of the project.

Meetings are required, typically face to face, to review the list of items they'd like.  By discussing what the end product looks like, you get to dive into the details with the customer.

Typically this means getting up to speed real fast on the clients business.  You learn the business rules, how the data flows through the systems, who does what when.  And sometimes you the client admits that they don't always follow an internal process 100% of the time, which makes it harder to code for.

What's interesting about programming Business Intelligence solutions, you get to learn about every facet of the business.  From products and geography to customers and accounts payable and GLs.  You get a birds eye view into the entire business and it's helpful to get up to speed fast.

And sometimes the Business Intelligence person becomes the resident expert into the complete ecosystem picture, which is good for repeat business.  Having a consulting firm that understands the business, the people, the process', having a signed NDA, it's a natural fit for a contract to get extended and re-extended over years.  Because having to re-train new consultants from scratch takes time.  Establishing a good working relationship with the client staff is key to long term relationships.  That way, they are not threatened by loss of job and become willing members of the consulting team.

A lot of clients have a need of consolidating various data sets into a central repository, that complies with traditional data warehouse methodology.  And some reports and perhaps some dashboards and scorecards and KPI for upper management.  Some are internal, others are client facing.  So allocating time for good Quality Assurance is key.  Or tweaks to the data or reports.

Sometimes it's good to let the client explore the possibilities.  Perhaps they've always had an idea of how to turn the data into insight, but lacked the resources or expertise.  By asking opened ended questions, suggesting possibilities, you can almost see the clients mind turning, brainstorming ideas.

Sometimes items get added to the project just by letting the client know some of the possibilities.

What if we gave you the ability to do xxxxxx?  Or what if we could generate reports sent directly to your email inbox, at whatever frequency you'd like?  Or what if we moved this process from Access and Excel to a real data warehouse that ran like clockwork.

Being a consultant typically means providing a solution, and moving on to the next project.  But sometimes you get in there, learn the business, help the client achieve good results, so they ask you back for more projects.  Consulting is also providing a service, a lending hand to a client, a win win situation.

And so it goes~!