The Three Rs

Reading Writing Arithmetic.

We had to learn them growing up.

That meant writing in cursive language.  Reading from books.  And multiplication tables.

I'm not sure what's required in today's classrooms.

Except I'm sure there's a lot more kids growing up that are more tech savvy than Three R savvy.

It's funny, I was good at the multiplication tables.  And solving the math problems.

In fact, they moved me up to the 3rd grade class just for math.  I was there for a little bit and then re-joined the 2nd grade math class.

Today the computer can do all your addition, subtraction, multiplication and division.  So can a physical calculator, which is kind of obsolete anyway.

And kids type more than they write.  And they read websites, not books.

I'm all for change and progress.  Just wondering if the kids today are losing out on some basic skills.

And let's not talk about social skills.  The online social network and emails are causing people in general to dissolve face to face contact.

The basic stuff we grew up with is slowly fading away, specifically the Three Rs.

And there you have it!


Concentration Dispite Distractions

Passion determines Success.  No wait, it's really Creativity that determines Success.

Maybe it's Concentration.  I find that working as a consultant, there are far fewer interruptions.  Less emails.  Less face to face interruptions.  Less drive by's.

And that allows a programmer to think for hours at a time.  Can you imagine?

The world has become an Attention Deficit Disorder breeding ground.  You typically get about two minutes before bombarded by emails, requests, people, meetings, etc.

It's gotten quite chaotic.  Almost like a battlefield.  And television is partly to blame.  People can no longer hold a thought.  There minds drift like screen flashes on the tv jumping from subject to subject.  A sort of dumbing down if you will.

So if you add up ( effects of television on society + the chaotic nature of the workforce + the constant barrage of interruptions from co-workers + social media distractions + cell phones ), then you can see how precious it is to have quiet environment in which to concentrate uninterrupted and get lots more productivity.

And I find working as a consultant provides that type of environment for the most part.  And the ability to concentrate for extended periods of time for a programmer is a commodity difficult to find in today's workforce.


Improvise on a Daily Basis


It's a key ingredient in the world of Business Intelligence.

Long ago are the days where IT sits in an ivory tower, distanced from the customer.  Where they got their 100 page design specification and went to work for 3 to 6 months locked away.

Now we are on the front lines, face to face with the customer.  Interacting daily, setting up meetings, getting design specs, asking for business rules, reviewing the data, overcoming obstacles.

The only problem, IT was not formally notified of this change.  So those developers sitting back in their chairs, raising their hands, waiting to be spoon fed, are in for a shocking eye opener.

Because they will be left in the dust.  As the nimble workers dance circles around them, producing results, on to the next project, while they are stuck in the mud.

A new reality for the world of BI.  Dance in the rain or be washed away from the tides.

Learn to improvise on a daily basis, as no two days are similar.  We get new stuff thrown at us all the time and we must adapt to the changing circumstances.

Yes, the ivory tower has been stormed, we are naked before the client, now think of something quick to say while you tend to their needs, or be out of a job.

For the record, you've now been notified.


Business Process Rules Provide Value

Here's a tidbit of information for you.

Do you know what is as important as data?

Business Processes.

The person who knows and understands how things happen in a company is golden.

How do leads happen?  How do leads flow into Salesforce?  How are leads given to Distributors?  How does an item get sold?  Where is the price book for SKUs?  How does a sale get booked into Accounting system?

You get the point.

In order to get at the data, the Data Professional must know and understand the Business Processes and how the data flows between systems.

Typically, the person(s) who knows this information can use it to rise to the top of the pack.  And sometimes they retain this info, releasing bits and pieces sparingly to retain sole ownership.

Because if this info was documented and easily accessible, then that person loses value and job security.  And we can't have that now can we.

So if you want to report on all the data, you must first know the flow of data through the systems and the business processes.

Make is so!


Bee Hive Effect

Steve Jobs from Apple, in this video, discusses the Bee Hive Effect.

http://www.youtube.com/watch?v=BstTBw6BLrE --> Minute:Second 20:07

He states, if you start a company in Montana, the person moves his family to work there, and the company fails, the person has to pick up and move again.

With Silicon Valley, you basically have to convince somebody to turn left instead of right in the morning to work at Apple so it's easy to find good talent.

Basically, Silicon Valley has the intellect, the University to feed talent, the Investment Capital and most important, the Risk Factor.

In the video, the financial backers in the early days, stated they had a passion, a purpose, evangelical, there was a lack of structure but there were processes hidden which everybody knew and followed.  To give back to the community.

You have to admit, this guy was ahead of his time, a true visionary and technologically contributed to society.  "You can change the world!"

Free Thinking Problem Solving Programmers

Do you know how to think?

Can you solve problems?

Do you know how to look at the same problem from multiple angles simultaneously?

Can you weigh the pros and cons to deduce the best practical approach?

Do you know how to view a situation from a unique perspective?

Can you see through the chaos for visions of clarity?

Do you ever get flashes of insight?

Can you solve a problem through non linear approach?

Do you know how to look at a problem with an untainted bias?

Can you harness your innocence to unravel a knot?

Do you enjoy the challenge of solving problems which nobody else can solve?

The education system does not teach problem solving.  It teaches memorization skills.  That is why we have a shortage of thinkers and that is why most company's can not find the talent they're looking for.  Schools teach you how to follow orders, how to prepare you for lifelong servitude.  The system is designed to keep people from becoming a free thinker.  And free thinkers are some of the best problem solvers.  And problem solvers are some of the best programmers.


Save Us or Replace Us? Robots

Where are things headed?

Algorithms.  Number crunching.  Predicting behavior.

Loss of jobs.  More automation.  Intellectual division between menial labor and thinking jobs.

There is pressure from the bottom.  From foreign counties.  Willing to work for less money and fewer benefits.

There is pressure from the top, education becoming more expensive, more competition for fewer positions, inflation chewing away at cost of living, baby boomers can't afford to retire with people working into their mid and upper 60's.

Will computers take over.  I believe they already have.  Production lines can create quality products in shorter time at less cost and work 24x7.

Traffic lights funnel the masses automatically controlling (or should we say preventing) the flow of traffic.  Big Data predicting who will by baby diapers.  Election results known hours before the polls close.  Weather patterns studied up to the minute to regulate transportation.

Automation is systematically removing humans from the work equation.  Makes for happy shareholders.  And more unemployed.  This trend will continue.

Until the average man can no longer afford the goods and services which he/she produces.  Which is the exact definition Marx predicted will trigger the collapse of capitalism.

Robots are slowly integrating into society.  As they develop in complexity and become more human, they will do the work of humans at minimal cost.

Is this a good thing?  As most technologies designed to free up our time for relaxation, they tend to have the reverse effect and make things more complicated.  Soon the robots will replace the migrant worker jobs, then the middle class jobs.

And if there ain't no jobs, well then, that could be a problem.  Are robots here to save us or replace us.  You tell me.


Coulda Woulda Shoulda

I played competitive tennis from the age of 10.  Rising through the ranks of club play, club tournaments, city tournaments / leagues, state tournaments and finally high school tennis team.

I played the number one position in both singles and doubles during 10th, 11th and 12th grade.  My sophomore year record was 3-9, that is 3 wins and 9 losses, basically I was outmatched physically.  I ended my senior year with a 9-3 record and getting to the finals of the District tournament, losing to the best player in the city who went on to play at Aurburn University and then the pros.

So what were my strengths?
  • Speed
    • I was fast and had good footwork from years of soccer
  • Anticipation
    • I could see the ball early to get in position, some people said I was deceptively fast
  • Endurance
    • Because I worked out every day I could play in the Florida heat for hours
  • Two handed backhand
    • Since I threw left handed, my two handed backhand was powerful and accurate
So what were my weakness'?

  • Serve
    • Since I played tennis right handed and threw a ball left handed, I did not have the natural throwing motion in my right arm, so I had to have a strong second serve with lots of spin
  • Keeping Score
    • For some reason I lost track of the score easily to my disadvantage
  • Seeing the ball 
    • I had difficulty seeing the ball to determine if it was in or out and didn't want to be called a cheater, so I played many out balls
  • Temper
    • Sad but true, there were many a times I lost my temper into a free fall spiral
  • Perfection
    • I expected to win every point and even the slightest error upset me
During High School I lived and breathed tennis, that's all I did, every day, including weekends.  Many of my team mates got tennis scholarships.  I ended up quitting tennis cold turkey which severely impacted my life.

Without tennis, how does one compete?  Stay in shape?  Have goals?  Identity?

The short answer is they all go down the drain.  The lost 17 years.  Until the age of 34 I picked up the racquet again.  To play club leagues.  City leagues.  State tournaments.  Earning a national ranking of 14 in the men's 35s.  And taught tennis full time.  Indeed a minor comeback.

And then proceeded to quit again as I found a wife and went back to computer programming full time.  I'll never forget those couple of years they were lots of fun.  Teaching tennis was an experience I never had before and I was good at it.  At the moment, I have no interest in tennis or competition.  I took it as far as it could go, gave it the best shot, had some good victories and that's that.

How far could I have gone had I played tennis in college?  Who knows.  At this point, it doesn't really matter.

I'll settle for living the married life, having a family of 4 legged creatures, taking walks in nature smoking cigars, relaxing by the pool and exercising.  For that I'm thankful.

Review of the Social Network

I recently watched the movie: The Social Network

It portrayed Mark Zuckerman, the founder of the popular social website Facebook.

A smart nerdy Harvard student, who was eager to find status in the elite groups of Ivy League schools, designed a website for Harvard students to rate women students based on looks after a breakup with a girl from Boston University.

The site was extremely popular, which brought the University network down, in which Mark was reprimanded.

It also brought notoriety to a few student entrepreneurs, who asked Mark to design a website they had the idea for.

Mark agreed to help them, at the same time, created a website of his own, without using any of the original source code, to become the eventual Facebook.

The entrepreneurs eventually sued for property right theft, and Mark paid them a hefty sum to not discredit the website.

Mark also had a partner / co-founder, who did help tremendously in creating the original algorithm and finances to get the ball rolling.  This guy and Mark eventually had a falling out and his role in the company was washed out, which triggered a lawsuit and settlement amount undisclosed.

There was another investor, the founder of Napster who assisted Mark in getting his site into the right people's view, however, the guy was into partying and it seemed he used Mark to get revenge on previous vendettas.

All in all, everyone involved became mega rich, the site is a huge success and the movie left you with a feeling that Mark is a brilliant guy who followed through with an idea and should be credited for that.

He did create enemy's along the way, and his brash persona only inflamed the rivalry's.  It seemed his original drive and motivation was to get girls and get included into elite groups.

I'm sure with his wealth he can get basically anything he wants.

On another note, it's interesting to notice that Bill Gates and Mark Zuckerberg were both Harvard students who went out on their own, struck it rich.  These two men will never need to worry about money in their lifetimes.

They seem to have the same drive, ambition, lack of social graces, technologically brilliant, but most of all, able to capitalize on once in a lifetime opportunities.

Perhaps its that exact type of personalities, take no prisoners combined with technological prowess, that is required to remap the social fabric of society.

They're both brilliant at what they do and deserve credit.


Microsoft BI Stack Consulting

Working in the Microsoft BI stack, you really get a wide assortment of technologies.

Recently I've been working with SSIS, SSAS, SSRS, MDX, ETL, MDS, Sharepoint, PerformancePoint, PowerPivot, PowerView, PowerBI, KPI, Database Administration and Performance Tuning.

What do they all have in common?


Many of the clients I've worked on have overlapping technology so each iteration I utilize experience from the last project.  There is a pattern to it.  Which is repeatable.

And the goal is to become a master at each of the technologies in this space.

In addition, there's also the business knowledge associated with different clients.  Getting up to speed in different business models is kind of fun, from Dental offices to Government Agency's to Financial Institutions to Health Care to Travel to Shipping and Receiving.

I don't see things slowing down anytime soon and couldn't be happier going to work each day.

Life is good!


Productive Hours in a Day

Consulting is a lot different from full time work.

Full time workers, if you condensed actual work time of an average day, may be 2 or 3 hours.

Trust me, I know how it works.

Consultants work 8 hours out of an 8 hour day.  They are busy nonstop.  There is no time for goofing off.  In fact, I'd venture to say that because of the tight deadlines, there's probably 10 or 11 hours condensed into an 8 hour day.

Full time workers have a good deal, as long as they show up on time, don't steal and don't piss off the boss, they can coast for a good portion of the day.  Granted there are lots of people working in full time roles who work hard, but I would go out on a limb to say that much of the day is wasted for a good percentage of the work force.

But that's okay.  Much of the medial work has already been automated or outsourced.  It's just a matter of time before much of the jobs today will follow the same path, in time.

Because job security no longer exists.  If you aren't providing value, you are just taking up space.

And even if you're the greatest worker in the world, a downsizing, merger, acquisition or dip in the stock price, and you're out the door.

So remember, make sure you're skills are sharp, network and do more than is expected.

And try to do at least 4 hours of solid work in a given day.



Who Reads These Blog Posts Anyway?

When you think about blogging you think about content.

Typically a blog has consistent blog posts, subject matter remains the same.

The blog writer must have his or her audience in mind when creating content.

What should we write about today?  Who is my intended audience?  How will they perceive my article?  Will my blog post offend anyone?  Should I write technical content that actual programmers will use one day to solve their problem?  Should the blog be more of a story with start, conflict, resolution, end?  Or how about perceptive observations?  Or what about telling the truth on touchy subject matters which most people know but don't talk about?  Should the post be cutting edge?

Basically, who the audience member in which the blog post is guided towards?

Personally, I feel my reading audience has changed somewhat over the years.  Based on the page visits, it seems to be a mix of technical interests, some regular followers, some people who read dozens of articles at a time, some who get here, do a quick scan and get out.

I don't really have a clue on who the typical readers are.  I've gone to events and meet people who regularly read my posts.  Even met some workers at client sites who've known me from blogging.

Suffice to say, most blog posts are like spewing mental vomit into cyberspace with no idea who the intended audience might be.  That's kind of a scary thought.

Nonetheless, the blogging must go on.  New content must flow from the ether world into the blog sphere.  Round and round the wheel goes, where it stops nobody knows.


Consolidation of Information Not a New Concept

Jesus Christ.  Lived a long time ago.  And he was the inspriration for many of the Bibles.  And the bible is the most read book in existance, I think.

Except the modern day bible is not the original.  It has been condensed, modified, re-written, passages removed based on the desires of the leaders throughout the ages.

A sort of cleansing if you will.

Fast forward to today.  The internet blasted open the world of information.  It exposed all knowledge to anyone with an internet connection.  And all was well.

There is a subtle change occurring before our eyes, completely hidden.  And that is the consolidation of information.  And that information is now in the hands of a few.  Which opens the door for cleansing, hiding, customizing what you can and can't see, as well as deliberate hiding.  Similar to what happened with the Bible.

The information holders can filter what you see.  They can track what you view, to build a Social Credit Report to determine your habits and risk factor.  And they can even change history if they deem necessary.

They are already doing this with television.  You watch what they feed to you.  If they want to portray the world as aggressive, that terrible people are plotting to bring down society at any moment, well then society responds to that and in essance, they are molding society through the flow of information.

And the same thing will happen with Information.  Long gone are the days of book burning to rid the threat of tainting the minds of little children.  Now they just suppress the information they don't want you to see.  And in another generation, that information will be a forgotten memory.

So this subtle transformation of consolidating information into the few is actually a great threat to the prosperity of society.  It could be a tool used for greatness, or as a prison to keep humanity down.  A form of control.  To keep us dumbed down, so we're obedient consumers and workers and do what we're told.

A tool to control the masses.

My MDX Journey

I worked for a company called Z-Tel in 2001.  While there they had some Cubes.  I had to report on those cubes.  Through Crystal Reports.  Using MDX.  Which I did.  Except I did not understand exactly what I was doing.

So that set me on a quest.  To learn the language MDX.  Except I didn't put much effort into it.  As other tasks required my attention.

And a decade went by.  And I soon had interest in the language again.  Except I didn't have a real project to work on.  I dabbled in Cube creation.  On my own.  At home.  In a sandbox environment.

Recently, my Consulting job required me to query against Cubes.  Through SSRS.  I did not tell myself that I didn't know what I was doing.  I simply got started.  And slowly.  After opening SSRS.  Creating a connection to the OLAP cube.  Opening the Query Builder.  Dragging and dropping Measures and Dimension fields.  The query was soon built.  Added the fields to the report.  And soon the report was created.  And sent to the Client.

I now understand what I'm doing.  Understand the relationships between Fact tables (Verbs) and Dimension tables (Nouns).  What a Measure is (Sums, Avgs, Mins, Maxs).  What an Axis is.  How Dimension tables Intersect at Points of Axis.  What a Column values vs. Row value.  How to Select from a Cube.  Apply Filters or Where clauses.

I understand it.  I can create queries using the Drag and Drop wizard.  I can read the MDX query and see what it's doing.  I can troubleshoot the code.  I can reverse engineer a Cube into an SSAS project.  Interrogate the Dim / Fact tables, what is their source. How are the Measures created.  How were the Hierarchies created.  The Attribute relationships.  Refresh the Cube when underlying table structrures change.

Yet, I have not sat down to write raw MDX code from scratch.  I have not used some of the built in functions.  Perhaps in time.

So 13 years after being introduced to Cubes and MDX, I now have a better understanding of how they work.  I tend to follow through on things.  Sometimes, more than not, it takes some time and effort.  You have to keep the faith.  With curiosities, hopes and dreams.


Intro to Microsoft MDS

Microsoft has a solution for centralized data management.

It's called MDS or Master Data Services.

It allows data to be stored in a separate database with Models, Entities and Attributes.

A Model is basically a database schema.  An Entity is a table.  And Attributes are fields.

You can import your data through staging tables.  You truncate the table, load your data and execute a Stored Procedure to upload the data.  Once verified the data is available.

You can also create Views to access the data.  And you can scan your data using the online website.

Another cool feature is a downloadable plug in for Microsoft Excel which allows Inserts, Updates and Deletes via Excel.

Why go through all this?  To have a centralized storage location for specific data to be maintained by end users or database developers.

No longer do we store employee information in 9 different places, making reporting difficult and inaccurate.  Now we have a single source of data to query against and populate the data warehouse.

Here's a quick link to get you started:  http://msdn.microsoft.com/en-us/sqlserver/ff943581.aspx



What makes a good #BI Consultant.

What makes a good Business Intelligence consultant.
  1. Flexibility.  
  2. Agility.  
  3. Determination.  
  4. Problem solving.  
  5. Social interaction.  
  6. Meet deadlines.  
  7. Finish last 10% of project.  
  8. Juggle multiple tasks.  
  9. Technical knowledge.  
  10. Ability to learn fast.  
  11. Mentor clients.  
  12. Design code for maintainability.  
  13. Code reuse.  
  14. Best practices.  
  15. Networking.  
  16. Sales.  
  17. Marketing.  
  18. Business knowledge. 

Microsoft #BI Admin Console

The Microsoft Business Intelligence suite is great.

From one IDE, you can ETL data, create Cubes, write reports.

SSRS has a web portal, administration site and can run independent or embedded in Sharepoint.

And MDS has a website, as does DQS.

Why not have a central admin portal to administer all BI websites?

And why not integrate some SQL-Server administration there too.

Like add users, permissions, etc.

MySQL has this functionality built in, why not Microsoft?

In may even create a new position, BI Administrator who maintains the BI Portal which in centralized.

Perhaps an on-premise portal such as Azure in the Cloud.



Consulting Projects

Well, the new year is upon us.

And we're busy as beavers.

I'm wrapping up a contract this week, full life cycle Data Warehouse, SSIS ETL, SSAS Cube, SSRS Reports, MDS.

And started a new one last night, full life cycle Data Warehouse, gutting Excel & Access move data/logic to DW, document processes.

And another one tonight, MDX SSRS reports.

Being a consultant means leaving the 9-5 predictable job behind, I love the change, rapid pace and different clients.



Sports Prepare Youth for Real World

Part of being a good programmer is the final process.

Getting all the ducks in a row.


Verifying all code found it's home in source control.

Releasing code into production.

And showing the final product to the clients.

Recently I had to present a live demo via Webinar / Bridge line.

There were about 6 people on the line, including the project sponsor and several of their key VPs.

To prepare for the meeting, I outlined the bullet points in which to speak about.

Nothing fancy, just some scribblings on a notepad.

Then I ordered the list, removed some items and was ready to go.

This reminded me of preparing for a sporting event as a kid.

You don't know exactly what's going to happen, you have a general idea, but you can feel the adrenaline flowing.

I was involved in sports early,  Bowling league at age 6, swim team at age 7, soccer team at age 7, baseball, basketball and finally tennis at age 10 and violin in the 2nd grade.

If you've ever competed, you know what's involved with the pre-game rituals.  Try to stay loose, yet focused, mentally prepare for the event, and when the time comes, allow things to flow naturally.

My presentation lasted 45 minutes and it seemed well received, answering any questions, showing the methodology we used, some of the exceptions we ran into, how we handled them, and what the next steps were.

I believe that youth sports help prepare in the daily happenings of the business world.

Because when it's game time, you have to produce.

It's just a fact of life.

Business Intelligence has Evolved

Business Intelligence and Data Warehousing is actually more complicated than most people think.

Just grab some data, create a query, throw it in a report, run and send to user.

Truth is, that's only part of it.

In the Microsoft BI stack, for SQL Server 2012, there's lots to know.

You have DBA administration, T-SQL, SSIS, SSAS, SSRS, Cubes, MDX, Pivot Tables in Excel, Indexes, Indexed Views, Clustered Indexes, Data Modeling.  You have to know how to deploy to the SSIS Catalog, set up users to run jobs, Virtual Machines, networking, ports, Active Directory, you have to become business knowledge experts, how to relate to Executives and understand how data flows through each of the system.

You also have to know about Data Quality Services, Master Data Services, Change Data Capture.

Data Quality Services allows people to clean the data before it's imported into the database.  It requires administration, setup, user roles and permissions, installing clients, and then actually using the application.  You have new roles such as Data Steward, Information Workers and Data Governance.

Master Data Services also requires installation, setup, user roles, permissions, creating web sites which point to MDS database, you have to learn about Models, Entity's and Attributes, how to Insert, Update and Delete data, create views and how to integrate the centralized master data into your Data Warehouse.

Change Data Capture basically captures data as it's inserted, updated and deleted in your database, so you can keep track of who did what when.

And then you can do these tasks in SSIS through components.

Suffice to say, Business Intelligence developers really have their work cut out and perhaps it explains why they are in demand.

We've moved away from simple Select * From Table queries we did back in the 90's.

We ain't in Kansas anymore Toto!


Lastest Data Warehouse Project

Working with a financial institution, we were tasked with building some reports based on 7 metrics.

So we gathered the specs, went in search of the raw data from a variety of sources, built the Staging tables, the Data Warehouse tables, moved some data to Master Data Services, created the ETL SSIS packages, created the SSAS cubes, then built 14 SSRS reports via MDX and some detailed reports using OLTP, and pushed the reports to SSRS web server.

Today I gave a presentation to the project sponsor, a webinar with bridge line.  I thought it went well, I spoke for about 45 minutes, displaying reports, explaining methodology, relaying some of the exceptions, showed how the reports provide value, how they export to Excel, and how the Cube is viewable from Excel via OLAP.

A few more days on this project to close things out, then on to the next project.  To think I started this project less than 3 weeks ago, fun stuff!

And so it goes!


ITIL Fever

Let me tell you about change control.

I worked in a place that got the ITIL fever.

And production came to a halt.

A one line code change would take days, weeks to implement.

Electronic paperwork, documentation, approval, more approval, scheduling issues, deployment issues, backout issues.

Holy toledo.

The good thing was I got to do all the reporting on the data.  Who was doing what when within the IT department.  Who was bypassing authority, doing unscheduled changes, who was doing the majority of work, who was sloppy in their process, you name it, I reported on it.

And the rumor going around was, if you didn't close your tickets in the allotted time, 3 strikes and it shows up on your review which translates to no raise.

So people were closing the tickets whether the work was completed or not, they didn't want to get dinged.

So a priority one ticket, which meant production was down, you got 4 hours to close the ticket.

Priority two, you had a day, 3, you had 2 days, etc.

With the implementation of ITIL, not only did work slow down, but people felt they were now longer judged by the quality of their work, they were no judged by their paperwork skills.

And they lost the good ole boy relationship with the customer, now all communication took place in the ticket, so it could be documented.

I think ITIL is great, only caveat, don't tie the programmers hands so much they are incapable of completing their jobs in a timely manor.

Beware of PCI Compliance with Data

When working with data, you need to be aware of the implications of not following protocol.

When I worked for the County Government, we were mandated to have our applications PCI Compliant.


That meant complying with the rules or face penalties and fines.

And since I supported multiple applications which accepted credit cards, I got to sit in a lot of meetings.

Basically, we could not keep credit card info on site, in paper format, electronic format, etc.

My Java applications had a text box for user to enter their credit card info, pin number, expiration date, etc.

That was a big no no.

So our architect devised a plan to utilized one of the credit card vendor sites, which allowed us to redirect the customer from our site, to theirs, where they paid by credit card, and returned back to our site with confirmation.

Sounds great, right?

Except let me tell you about pissed off customers, upset people and a lot of chaos and mayham.

After we went live, we immediately got complaints.  People claimed they paid, we had no record, the credit card company took their payment.  Somehow they were closing the browser before making the round trip.

And if they didn't make the round trip, we had no confirmation receipt number, so basically we didn't know they paid.  Which resulted in customer paying again, and again.  We even shut their water at their house due to non payment.  And we had people showing up for campsites which they paid, they had a receipt of payment, yet no record of reservation.

Now bring in the client who asked us to do the work.  They were upset.  We had to do a three way validation of the payments, on the credit card side, the bank side and our internal record side.

What a mess.  I ended up finding another job with the School Board.  And the guy that architected the system, he got a promotion to Senior Architect.  Somehow they never got fined for violation of PCI compliance, yet the client had a bad experience as did many of the customers.

For clarification, my code was solid, it accepted the info if the customer paid and the round trip handshake completed, it then updated the Utilities Java Web Service, which updated the database and processed the payment on our side.  I even supported the IVR for Utilities payments, Traffic Fines and Citations, which updated the mainframe using IBM Copybooks and Java Web Services.

I was a real Java Programmer before immersing into Business Intelligence full time.

It's funny how life works.


Lots of Hype in 2013

2013 was the year of hype.

Big Data.  Data Scientist.

Everyone jumped on the band wagon.

Everyone had all the answers.

Except nobody could pin down the specifics.

What is Big Data?

What is Data Scientists?

There's still no conclusive answer.

I contributed by blogging about it, promoting it.

I only worked on one project that was considered Data Science without using Hadoop.

Those who are true Data Scientist tend to strut on another level.

And others who claim to be DS may in fact not be.

The old joke in Software development was Vaporware.  Where the sales people go out and sell a product that hasn't been created yet, while the developers are back at the office designing and building it.

The main difference is Hadoop was already created.  With a strong community base expanding in all directions.

While the evangelist were out in full force.  The Analysts were pushing it full steam.  The business knew it was important but didn't know how to implement it.

And not much discussion of how this new tool can be used as a weapon, the darker side of Big Data.  Some talks of Best Practices, a code of ethics.  That works for the majority, except for the rogue and national orgs who are not mandated to comply.

The University's are teaching the bright young minds of today.  Shortly these college grads will join the workforce and bring it to a whole new level.

I hope 2014 we get more into the details, more clarity, more structure, better definitions, better business cases, more real world examples.

Let's better define what a Data Scientist is, how it integrates with existing Data Professionals.  The number of job descriptions in the Data ecosystem is already staggering from Database Developer to Report Writer to Data Warehousing to ETL to Business Knowledge Expert to Statistician to Data Mining to Data Visualization Expert.  There has never been a cohesive job description which encapsulates everything.  Not sure that will ever change.

The Big Data phenomena swept the data community in 2013.  Now let's turn that talk into action and bring it to the common folks.


Top 16 Answers on the Board

1. Technology changes really really fast.  It's very difficult to stay current without putting in extra time off hours.

2. There are tons of really smart people entering the workforce, that will work for a fraction of the cost, except they don't have the business experience yet.

3. Although technology is supposed to simplify things to free up your time to do other stuff, in reality, technology is extremely complicated with all the hooks into other systems.

4. The workforce is a battlefield, like it or not, not everybody is on your team, although they should be.

5. Your employer and your boss are not responsible for keeping your skill set current, they have problems that need solving and if you can't do it, they'll find somebody tomorrow.

6. Full time workers have very little job security, which is the opposite of what the work full time means, so brush up on your skills, your social network and your resume.

7. Most employers are watching your surfing habits at work, and if they wanted to, they could take action at any time against you.  Most people abuse the surfing policy set in place or surf on their mobile device.

8. Recruiters can and will help you find a gig, however, if they don't find you a job right away, chances are you'll go down to the bottom of the list, until some new recruiter start working there and goes through all the past resumes.

9. Working 30 years for a company used to be the norm, now it would be ludicrous, most people switch jobs ever few years to avoid becoming obsolete casualty.

10. Many coworkers talk about their private lives at work, which is good topic of conversation, except it's not considered best practice and could be used against you.

11. Many employers don't mind you working over 40 hours per week, but as soon as you request a few hours off for a doctor's appointment, they'll dock you without hesitation.

12. Yearly reviews are typically devoid of personal feelings, they are descriptive narratives written in robot language with hidden meaning to let the next boss know your work habits.

13. The majority of jobs today can and will be automated in the near future, if you aren't providing value or have proprietary knowledge of how the business works, you could be out the door in tough times.

14. Your pay raise will consistently be in the very low percentages, which means you are falling behind the rate of inflation, the only real way to get a descent raise is to switch jobs.

15. You should never accept a counter offer, your boss will hold it against you and you'll end up quitting anyway.

16. The skills they teach you in college have very little to do with succeeding in the real world, if you're smart, you'll take an internship or work on the side to get a taste of how the real world works.

To summarize, roll with the tide, take responsibility for your career, try to do the right thing even when nobody's watching, and most important, enjoy what you do for a living, as you'll be in the workforce for more than 40 years, take is slow, it's not a race, it's a marathon.