7/31/2014

Prepare for Meeting

I think one of most surprising things, people spend enormous time preparing for meetings.  I never knew you could do that.

Today I had a two hour conference call working on SharePoint Dashboard placement.  Afterwords decided to head out for a bit, grab some food for a change.

Upon return, pulling into the parking lot, received a text, "Are you here?".  Replied, "Just got back".

Turns out we had a meeting in two minutes.  Got to my desk, some people were filing into our war room office space.  They handed me the chord for the projector.  Wasn't aware we were having a meeting, let alone it was my meeting.

Over ten people showed up, all were senior executives, and a graphic designer.

Hmmm.  Wonder what I should talk about.  So we dove into the data, discussed some business logic, winding through each of the five dashboards.  Around the last one, I had some concern, because that data was not part of this phase, so it shouldn't be part of the scope.  Luckily I did the ETL the other day so the data was already available.

So there's a bit of work, due next Tuesday.  Looks like I'll be working over the weekend again.  I thought the meeting went rather well, for winging it.  I guess public speaking works better when you don't have time to get nervous, especially when I found out one of the attendees was an owner of the company.

And so it goes~!


Repeat Your Row Headers in #SSRS

Would you like to repeat your Row Headers in SSRS SQL Server Reporting Services Report?

Adding these 3 lines to your SSRS Report Code (make a backup prior to this~!)



            <TablixMember>
                           <KeepWithGroup>After</KeepWithGroup>
                           <RepeatOnNewPage>true</RepeatOnNewPage>
                           <KeepTogether>true</KeepTogether>
            </TablixMember>

Add it here:

 

I found this tidbit here on MSDN (thanks!):


Business Intelligence Developers Are Hard to Find

From observations, it seems qualified Business Intelligence developers are hard to find.  Many company's have to search for talent across the country.  Which is the current trend of having a mobile workforce.  Teleconferencing, working at the client site, work anywhere.

BI has remained the same for a while now.  If you learned Data Warehousing concept 20 years ago, not much has changed.  I'm a late bloomer with Data Warehousing, although been creating reports for many years.

So why are BI developers so difficult to find?  My opinion, people don't like to think.  And BI requires thinking.  Not just programming thinking.  But working with numbers.  Reconciling reports.  Takes a lot of mental processing to crunch the numbers plus program 8 hours a day.  Today's workers prefer to not think, go on auto pilot, walk the line.  So inability to think for extended periods of time is probably the number one reason for BI shortage.

BI requires multiple skill sets.  From SQL to data modeling to ETL to Reporting to Dashboards to data quality to Cubes to meeting deadlines and deliverables and following the Project timeline and meeting with clients and executives and keeping track of time and time sheets.  So the number of skills required to perform end to end solutions is high.

More often than not, things don't go as planned.  And when that happens, there's really nobody within the organizations that can assist because they don't know exactly what you're doing.  This requires problem solving mojo.  And many people, similar to thinking, do not have the problem solving skills required to perform BI solutions.

There are a great many vendor tools available to perform Business Intelligence.  Too many to count.  So perhaps the reason BI devs are hard to find is because there's such a variety of flavor of technology that finding someone with the specific skills becomes difficult.

Another reason BI devs are hard to find is because they are in demand.  Which means they probably already have a good job so they're not interested in making a change.  They are already well paid, working with great clients and doing cool stuff.  What benefits does the hiring company offer which the devs don't already have.

Lastly, the schools have not kept up with technology.  Therefore, they are not preparing today's youth for the challenges of life by providing definable skills to succeed in the workforce.  Students are funneled through the system, tested and retested, and sent on their way after a 12 year prison sentence to fend for themselves in a tough workforce.

So I tried to document some of the reasons I believe Business Intelligence Developers are hard to find.

And there you have it.

Full Time vs. Consulting

When you work in a full time position, you take for granted the amenities.

Most likely you have a desk.  A phone.  Chair.  Dedicated network connection.  Copy / fax machine.  Printer.

As a consultant, you don't get any of these things.  You go where they tell you.  Sit where they tell you.  Do what they tell you.  In the time allotted.  And fill out weekly time sheets.

A renegade hired gun brought in for a specific purpose.  Traveling across the land from company to company.  Meeting new people.  Driving to new locations.  Never a dull day.  And never the same day twice.  Always a new project on the horizon.  Chance to learn new skills.

Full time workers have stagnant jobs, day in day out, predictability.

Consultants fly by the seat of their pants.  At double the rate.

And so it goes.

7/30/2014

Hired Gun

I'm kind of excited about the latest project.  Started last Tuesday.  Build a data warehouse from a SQL NAV system.  Luckily a previous developer started a DW, then left.

So I picked up where they left off.  Unfortunately, they didn't use proper naming conventions nor did they implement a staging area.

So I gutted almost everything except for the Dim Tables, as they pointed to the correct source tables, not much to change there.

I mapped out all the fields needed for the reports, a total of 83.  Next, mapped the fields to the NAV database.  Then created the fact tables and constraints pointing back to the Dim tables.

So the client said it would be okay to create the first set of reports in SSRS page viewer in SharePoint.  So I built all seven reports.  This morning they went to QA.  Had some executives sit at my workspace in the war room, and we went over each of the reports, couple of changes here and there on the fly, deployed back to SharePoint, saved in Team Foundation Server and it's looking pretty good.

One of the executives said they've been waiting on these reports for two years and couldn't believe I did them in a week.

I've always considered my skills to be at the Senior level.  This project was a confidence booster to some degree.  As I performed the work unassisted, with a tight deadline and no prior knowledge of the line of business or database.  I think the end result is solid and hopefully the clients are pleased with the results.  I think the role on this project was more of an architect.

And I built a data warehouse with 10 dim tables, 10 fact, complete ETL in SSIS with some complex SQL and SSRS reports into SharePoint.  And I built the Security model as well to use the NT Authenticated user passed in from SharePoint to limit the result set to specific users based on a table in the NAV system.

Based on my original estimate, 98-119 hours, I should finish up at 73.  And I built an SSAS cube because I thought the PowerView Dashboards were getting added to this phase.

I may look a little funny, but I think this project was a win.

This is what consulting is all about.

How to Solve Any Problem

How do you solve a complex problem?

First, you identify, label and analyze it.  Then you have power over it.
  • By understanding your domain.
  • Deducing the problem to more simpler problems. 
  • Applying the appropriate methods available.
Once the problem has been identified, you solve it by using your skills, acquired by learning and experience, available tools, logic, reducing and deducing, circular approach.

In essence, the solution will eventually reveal itself.

The solution could have many possible solutions, some easier than other.  Some simple and some complex, some shallow and some deeper.

If you reduce the problem to manageable chunks, which are familiar to you, you can solve the problem.

An good example is the Rubic's cube.

There are countless possibilities.  You can move each plane in any direction at any time.  When you move one side, the other sides move.  It appears to be utter chaos.

So when setting out to solve it, the goal is to match all 6 sides.  It appears impossible to conquer.

When i was 11, my brother showed me a simple pattern of movements.  If you apply a series of combinations, you can alter the cube while keeping one side in tact.

So let's say you are trying to get one layer complete, the red side.  That's a fairly simple task.  Once that side is complete, the difficult part is that any move you make disrupts your completed layer and all is lost.

Yet when you apply this specific series of combination of movements, the end result is the four corners of your current side remain in tact, while changing the opposite side.

What does that mean, it means that you can then begin work on the opposite side of the cube, virtually keeping the original side unaffected.

So with a lot of time and experimentation, I was able to complete two opposite sides of the cube.  Which mean that the four corners of each side were in the precise position to form the correct structure.

With all the corners in place, and a lot more trial and error, I was able to complete three sides.  Although the original pattern helped to solve three sides, I had to learn more patterns as I went and eventually got four sides.  Finally, all six sides were complete.  And the problem was solved.

The reason it was solved was based on a very basic pattern of movements, when applied correctly, gradually, to allowed the structure to remain in tact without disrupting the current side,

So once you know the very basic series of movements, you know the methology of solving the cube.

And the magic and mystery dissolves into thin air.

And once the pattern is identified, labeled, and re-applied, you have cracked the code.

However, without knowing your domain, your possible cube positioning and how each move affects the downstream results, you're grasping at thin air.

I give this example to show how many seemingly impossible problems can be solved.  There is a level of intelligence required, however, by learning some basic steps, reducing the complexity into simple processes, which are repeatable, a problem becomes less daunting and maybe even solvable.

And there you have it.

7/26/2014

Intro to Turing Test

I'd like some coffee.  How should I go about obtaining a cup.

I can brew a cup.  Ok, do I have coffee?  If yes, do I have water?  If yes, does the coffee pot have water?  If yes, was coffee grinds added to coffee pot?  If yes, is there a cup beneath the spout?  If yes, is the coffee maker turned on?  If yes, press the start button.

There's pre-defined logic that can be accounted for.  If any of the questions had been answered no, there's a whole new set of logic questions that need to be handled.

It's the job of the programmer, to identify, document, translate business logic into code, that has a set of rules to handle the flow accordingly, and trap errors along the way.

Let's change the story.  Let's get a cup of coffee at the local Bux.

Is the store open?  If yes, Is there a line?  If Yes, get in line.  Is it your turn?  If yes, greet nice sales person, order coffee.  Is coffee ready?  If yes, do you have money?  If yes, is it credit card or cash?  Etc. etc.

The point is, there are still predefined rules to adhere to.  Yet when you add in a human factor, you expand the level of complexity.  There are too many extraneous factors to account for.  So it becomes increasingly difficult to handle every scenario.

Let's add in artificial intelligence.  A computer brain, doesn't necessarily need a body, must handle millions and millions of different scenarios, apply the appropriate algorithm, and based on the response, must update it's memory bank / neural network with the results.  A neural network is a type of memory bank that is trained to respond based on weighted variables.  It gets fed information, which triggers a neuron to fire true or false based on a a weighted percentage, which in turn triggers more neurons downstream.

They are able to train models to handle specific scenarios.  The goal is to mimic the human brain.  The results thus far have been hopeful, yet it's still not there yet.  And my guess is the human brain is so complicated, we are unable to decipher its behavior with any practical results.

At some point in the future, perhaps we'll get closer to mimicking the human brain so much that differentiating between the computer and human is nearly impossible, called the Turing Test.

7/22/2014

T-SQL Add Drop Constraints

When building a Data Warehouse, there are times when you must drop and add Foreign Key constraints.

Here's how you check to see if the Constraint exists, if it does, Drop it:


IF EXISTS (SELECT 1 from INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS where
       CONSTRAINT_NAME = 'FK_Posted Claim Fund_Vendor')
ALTER TABLE [fact].[FactPostedClaimFund] DROP CONSTRAINT [FK_Posted Claim Fund_Vendor]
GO

Likewise, to see if the Constraint doesn't exist, Add it:


IF NOT EXISTS (SELECT 1 from INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS where
       CONSTRAINT_NAME = 'FK_Posted Claim Fund_Vendor')

ALTER TABLE [fact].[FactPostedClaimFund]  WITH CHECK ADD  CONSTRAINT [FK_Posted Claim Fund_Vendor] FOREIGN KEY([VendorSK])
REFERENCES [dim].[DimVendor] ([VendorSK])
GO
ALTER TABLE [fact].[FactPostedClaimFund] CHECK CONSTRAINT [FK_Posted Claim Fund_Vendor]
GO

This technique is quite handy.

Mountain Living