Predictive Analysis

Predictive Analysis.

What is it?

Look for patterns over time to guesstimate future behavior.

When I worked in the Mortgage business for Nations Bank in 1995, they purchased a new software system which calculated if a customer was credit worthy.

They based the customer's credit score, including charge offs, slow pays, foreclosures, etc. and bounced that info off the customer's length of residence, length of employment and number of recent inquiries.

So based off a limited number of factors, they could predict whether a customer was going to fulfill the length of the loan or if the customer would default.

Quite fascinating.

A fortune teller, for profit.

 We were instructed to only override the system a certain percentage of times, for extenuating circumstances.

My job was to compile the approvals and declines on a daily basis.

That's actually how I entered IT, because I convinced the bank to pay for a college class at the St. Petersburg College - I took C++ and got the highest grade in the class, and the professor spoke with a friend of his and placed me at a part time position doing Visual Basic 4.

Anyhow, the basis of predicting future behavior based on previous action has been around for a while.

However, because data and big data have become so big recently, along with data mining, predictive analysis can be utilized to process larger data sets in quicker time, by the average BI person.

So for example, if my company sold sporting goods equipment, and I gathered weather patterns for the past 100 years, and I noticed a steady increase in snow in the NE USA, I could then stock up on gloves and scarves.  However, I could also bounce that data against the average annual income of the people in the NE, to see if disposable income has increased.  If so I could promote the latest high end ski equipment and make record profits.

Now take the opposite, sun screen.  If the weather patterns indicated increased warmth, obviously people would be stocking up on sun block.  To confirm this, you could mash the data with medical data such as skin cancer treatments over time to see if there was a medical necessity.  Thus you could clean up on that too.

Both those examples are weather based, but they are only examples, and you can substitute numerous others.

You could look at baseball batting percentages, stadium seating capacity / tickets sold, average age of hall of fame players, best coaching statistics, anything really.

This is all well and good, assuming the people crunching the numbers are looking to profit and better humanity in some way or educate people or medical purposes.

Overall Predictive Analysis can't tell you who will win the World Series for the next 5 years and what the scores will be in game 7 five years from now.

There is only so much predicting one can do, as outside influences / forces skew the future behavior.

A pitcher gets injured, a team goes on a hitting streak, a team brings up 3 minor league players who rock the majors.

You can study the past as much as you want, but some things can't be predicted.

You would have to know all factors at the time of decision, which is nearly impossible.

So we have to make decisions from the data after careful analysis, looking for patterns over time, blended with intuition and business experience, to make the best possible decision at the time.

That's my take on it anyway.


Still Reporting After All These Years

Just to let you in on a secret, I still like creating reports for a living.

Yah, I know, I've been complaining a lot lately on the blog.

Too much to do, not enough resources, changing design specs, working conditions, etc.

And I've been doing more report delegation than report creation.

My contractor has been pulled off to do some SSIS work for the DBA team.

And my other programmer got moved to the AS400 team.

So that leaves me with one full time programmer + me.

So I've had to pick up the slack.

And write reports.

And I love it.

Back in the saddle again.

Writing SQL + SSRS.

And filling out tickets.

And closing tickets.

And assigning tickets.

And meeting with customers.

And emails.

And phone calls.

And meetings.

I've been busier than a one arm paper hanger.

And I really like interacting with the customers.

And being of services.

And working fast.

Sure people work fast, most do.

But I work really fast.

Most all of my customer's who watch me code while at my desk have the same conclusion.

"You're fast!".

So we'll see how things go.

If things work out for me here, I'll stay.

If not, we'll find something else.

Full time means nothing now a days.

We are all independent contractors.

Just some of us are at full time positions, with benefits.

But we can all be shown the door any day of the week.

I know one thing, I like reporting.

And there you have it!


From Programmer to Supervisor

Throughout my career, I have ALWAYS wanted to stay a programmer.

During an interview, I would respond consistently, "I do not want to go into management."

If you've been following this blog, you probably know that I got this supervisor position by accident.

I thought it was a Microsoft SSRS position, even during the interview.

It wasn't until my reference called to congratulate me for moving into Supervisor role.

So you can see, I've always had a hesitation to leave the world of programming behind.

Don't get me wrong, I still code every night on my part time job.

But on the day job, I'm doing more delegating than coding.

I'm slowing getting removed from the source code.

So I'm not the go-to guy and business rule expert on every project.

Which is a little scary.

For when a user enters a bug, I don't have a quick retort.

I have to go to my programmer let them dig through the code, translate it to me, and then me to the customer.

This happens to all programmers who enter management.

Willing or not.

At some point, they become distant from the day to day programming.

Now I do get to code from time to time.

As the queue demands it.

And I maintain and create reports for certain customers.

But the fact is I can not know all the minute business rules for all customers.

That's the bottom line.

Oh crap, what have I gotten myself into...

Quality Assurance

In the early days of programming for me, there was no such thing as Quality Assurance.

We coded, we moved to production.

This was the day of Visual Basic 4,5&6.

Rapid application development - RAD.

Then one day, I had someone review my work before moving to production.

The name of this person was called QA.

Quality Assurance.

There job was to find all my bugs.

And as we know, most people think they don't write bugs.

And it's the QA job to knit pick every little thing, so we think.

However, I have grown to like my QA person.

For they find the bugs before the business does.

And that's one less headache.

Instead of troubleshooting bugs with eyes watching, can fix casually, before it's a problem.

So go on, give you QA person a hug.

They are just doing their job.

Although I image they get some satisfaction from infuriating the developers.

And so it goes!


SQL Stays Crunchier in Milk

Yes it's true SQL stays crunchier in milk.

Because SQL language hasn't changed much since I started to code in 1995/6.

Sure, they've added a bunch of slick new features.

To make our lives easier (more complex).

Yet if you know the basics of SQL: Select, Insert, Update & Delete, you can program in almost every relational database.

And that's quite powerful.

If I had to direct someone in the IT field just starting out, I would point them in the direction of learning SQL.

They can always build on top of that with .net, Java, PHP, etc.

Or perhaps learn the new Big Data concepts.

 Truth be told, you can spend an entire career in SQL Development and be successful.

And there you have it!


Business Model

When I was a kid around 8 years old, many moons ago, my father explained something to me.

We were talking about McDonalds.

I sure liked to eat fast food as it was a luxury back then.

What he said was, they were originally in business to sell empty paper cups and empty wrappers.

However, they did not sell that many.

Someone clever decided to add something to the products to get more people to buy them.

So they decided to add cheeseburgers and fries and beverages.

And you know the rest of the story, they are probably one of the biggest and most well known company in America.

Years later, I heard that their business model was even more complex.

They are actually in the land business.

They are one of if not the largest business land owners in the country and own some of the best parcels of land in every city and every state.

So there you have it.

I also like Chinese food and had chicken & broccoli for lunch today.

You notice I didn't say fried rice, soy sauce, sweet and sour sauce and a fortune cookie.

Because you assume that it's part of the meal.

So if you think about it, their core business model is the meal, the egg foo yung, the chicken & broccoli, the general cho chicken,etc.

We assume the extra's are part of the meal.

So every person gets these extras no matter what their primary meal is.

So in my opinion, those are the things that they should strive for to ensure quality.

Because everyone who walks in the door for a meal gets these standard products are assumed to be part of the meal.

So when I go on a job interview, they are asking for someone who has chicken & broccoli.

Sure we all have that, but it's the side items that are part of the meal that are not publicized that are what makes the difference.

That's what I think anyway.

You tell me~!


SQL-Saturday Tampa #SQLSat110

I enjoyed the SQL-Saturday in Tampa this weekend #SQLSat110.

It started off with the Pre-Con with Bill Pearson on Friday.

Some people from my office car pooled and we drove over to Ybor City together.

Once we checked in and found the conference room, Bill gave an excellent presentation on PowerPivot.

I'm sure we'll be able to incorporate a lot of this into out BI  going forward.

He also presented on some advanced SSAS ways to speed up performance.

I got up early Saturday to drop my two dogs at my mother in laws house and Maddie got injured jumping into the car.

She was okay but I was upset none the less.

My car pool picked me up at 7am and we headed over to SQL-Saturday Ybor City.

The place was packed and I bumped into @SQLGator at the checkin.

It was good to meet face to face after much Twitter interaction.

Then it was on to the demo's.

I sat in on a demo for 'Always on' - looks like some great features in the new SQL-Server 2012 for Mirror'ing, etc.

Then I watched 3 back to back SharePoint demos and learned a bunch.

Lunch was good, authentic Cuban food as I sat with some co-workers.

After lunch I watched Jen Underwood @IdigData who talked about Mobile BI which was quite impressive.

Then I roamed the halls and introduced myself to several people and had a chance to speak with Jen & Robert Skoglund, both from Microsoft, for awhile about SharePoint / SSRS integration, new features in 2012, as well as the ability to 'Format' code in T-SQL (the need for it).

Then I spoke with a bunch of the vendors - good to catch up with Jose Chinchilla @SQLJoe.

I spoke with Karla Landrum @KarlaKay22 from Pass.

I met Richie Rump @Jorriss at the pre-con too.  I saw him present at the last conference for Project Managers and I enjoyed it.  I think more BI people should know what it's like to run a project.

Also met some of the presenters and talked SQL-Server, as well as some local people too.

I learned a bunch, had good food and did some networking.

All in all, I thought this was one of the best SQL-Saturday's I've been too.

I would like to thank Pam Shaw & all the volunteers for such an excellent event!



Unix Killer

On one job in 1999, I was asked to convert a Visual Basic / Access application to point to Oracle.

I coded the app, tested and went live.

And the Unix server crashed.

Everyone took notice.

I got called into a meeting.

What did you put in that code to kill the Unix server?

Turns out, the VB app was calling the Access app every 10 seconds, to check for Status messages.

An early form of Instant Messaging.

So 150 users, calling the Unix server every 10 seconds, took it down.

They asked me to disable that feature and they purchased a 3rd party app to replace.

And all was well.


SQL-Saturday Tampa Style

Tomorrow in the Pre-Conference for SQL-Saturday Tampa.

My boss was able to let me and another developer attend the conference with paid leave and paid conference.

He is also sending 3 people from the DBA team, 1 of which will attend the BI tract and the other 2 will take the DBA track.

We are car-pooling to the event.

I'm really looking forward to the paid training.

If you've been following my blogs over the past year you may have mentioned once or twice the lack of training I've received in my IT career spanning 15+ years.

Then Saturday is SQL-Saturday, which we are also car-pooling.

So overall, this should be a great weekend.

I plan to bring my laptop tomorrow in case they have wireless connectivity.

Hopefully nothing major goes wrong at work as we'll have skeleton crew available to put out fires.

So stay tuned for blog posts talking about the upcoming SQL-Saturday Tampa style!


Visual Source Safe DELETED vs. DESTROY

Well, did you ever write a report, send to customer, and then have the report gone, from BIDS, SSRS Web and Visual Source Safe?

That happened to me yesterday.

I thought I had deleted the report.

And I did.

So I went into my Recycled bin.

Found the Excel Spreadsheet, which I restored, to obtain the file name.

The SSRS.rdl file was not there.

I created a dummy .rdl file placed it in the original work folder, renamed it to the old report name.

Then I tried to re-add it to Visual Source Safe.

Guess what, it found the report, and asked if I wanted to restore the old version.

So I did.

And then brought down the file to my local hard drive.

And there it was, safe and sound.

So I didn't have to re-write the report.

And I pushed the report to the SSRS web server and the user can run it.

So the trick is, in Visual Source Safe, never, ever allow DESTROY rights to ANYONE.

Only allow DELETE rights, because of situations like this where the file needs to be recovered.

And there you have it - hot off the presses...


Data Quality Services 2012 Presentation

Tonight, after a long day going since 5am, I drove to Tampa Microsoft building to attend the Tampa Bay SQL Server Business Intelligence meeting.

I got their late as my to-do list never ends and I arrived at 6:30 right as the meeting started.

The meeting was about:

Topic: Data Quality Services 2012
We will explore data quality issues that face today’s business and how Data Quality Services 2012 can help IT Pros and Data Stewards meet those challenges. We will start from zero, build a knowledge base, profile data to train our knowledge base and then use that knowledge to highlight both the cleansing and matching capabilities of DQS. This session will also cover the DQS SSIS component that’s available in SQL Server 2012. 

And the speaker was from Pragmatic Works

Speaker: Chris Price
Chris Price is a Business Intelligence Consultant with Pragmatic Works. He has his M.B.A from the University of South Florida and has worked as a software developer and architect since 1999. For the past couple of years his attention has been focused primarily on the world of Business Intelligence with a specialty in ETL and Data Integration. His interest spans the Microsoft BI stack from SSIS and SSAS to SSRS and Performance Point.

Overall looked like a quality tool, well presented.

I asked one question, there appeared to be very little coding in the 'rules' portion of the application.  I was wondering if there was a way to view all the business rules in a report or something in case the business user wanted to verify the accuracy of the logic in a Data Governance meeting or something.  Answer, not at this time.

Didn't get a chance to eat the free food or drinks, but I brought a chocolate bar my wife gave me to snack on.

Home by 8:10pm so can't complain.

There is so much new technology to learn it baffles the mind.

They guy from Microsoft said Big Data is the next big thing.

Sounds good!


Face to Face Trumps Social Media

Social Media is great for interaction, information, promotion, instant access.

I'd say good 'ole fashion face to face communication trumps Social Media.

Being with people, interacting, getting immediate responses, facial expressions, body language, can't be simulated by 140 characters.

Social Media ain't going anywhere, nor is face to face contact.

Best of both worlds.