6/29/2012

First Week

Today completed the first week of my new job.

Basically thrilled.

Will get to do advanced BI reporting.

Tabular Model, Power Pivot, Power View, Performance Point, KPI's, T-SQL, SSIS, c#, Team Viewer.

Almost no traditional reporting what I've been writing for so many years.

Like I've said before, the technology is no more difficult than let's say writing Java code.

In fact, there's LESS coding and more architecture and process' and training the users.

The thing that stopped me in my track prior was the setup of SharePoint 2010 on a VM.

A guy at work gave me a link today to assist with that.

http://www.microsoft.com/en-us/download/details.aspx?id=27417
 
He's a c# / DBA programmer, we actually went to high school together :)

Everyone seems nice, they have an edge of sarcasm that underlie' every conversation, much to my liking, as I've been called sarcastic once or twice.

I actually thrive in a non-structured atmosphere where I'm allowed the freedom to get things done and to be myself.

With that said, I haven't actually done anything yet.

I've documented all the BI stuff and created a doc to store all the server and database names, etc.

Looking forward to after the end of the 2nd quarter / month end to dive into some work!

And there you have it.

My first week.

6/28/2012

I Won a Device

I have several emails and I don't check them all the time.

So today I checked my Hotmail account:


Hi Jon, Thank you for attending the New World of Data Symposium in Orlando, FL on Sunday, July 10th – and for completing the post event evaluation. We appreciate your feedback. I have great news for you! Your name was randomly selected as the winner of the Samsung Series 7 Slate.

http://www.bloomconsultingbi.com/2012/06/big-data-symposium-orlando-6102012.html

http://www.bloomconsultingbi.com/2012/06/microsoft-big-data-symposium-orlando.html

So I emailed back, sure enough the sender was from Microsoft.  I will check the mail next couple of days and we'll see if the Microsoft Santa is good to me this year!  YIPPIE!

6/26/2012

Intro to Team Foundation Server

I was the Visual Source Safe administrator for Florida Power in 1998.  And for Paymentech after that.  And WestPoint Underwritrs after that.  Even for the County I administered VSS.

So on my new job we use a product called Team Foundation Server.

Latest version of VSS with a new name.

Tight integration with Visual Studio.

It's also web-based.

Quite a nice improvement over the years.

There's also an Admin piece but I have yet to see it.

Microsoft really has their foot in every product line.

And the quality is usually very good.

I always thought they should have built a Microsoft Virtual Machine to run on Linux & the Mac.

Instead they got in trouble for J++ and missing the boat on the internet & reporting.

However, those battles are long ago, now with the hand held devices.

Microsoft definitely has some great products across the board that interact nicely with their cousin MS applications.

Power Pivot is the MS BI Savior

I attended a SQL-Saturday Pre-Conference session from Bill Pearson a few months ago in Tampa.

I thought the class was great: good content, good presentation and fun stuff.

Today while at work at my new job, I see the impact that knowledge is going to have going forward.

The problem: disparate data sources across multiple product lines and acquired business'; previously incorrect BI report disseminated to top exec (bad rep for BI team); need ability to propagate data to various internal clients with various permissions.

The Solution: Power Pivot via SharePoint.

You see, once the data is consolidated and validated, it can then be brought into Power Pivot plug in for Excel which can store whopping amounts of data, where the dimensions and measures are displayed to users as slicers, where reports are centralized and uploaded into SharePoint where permissions are set.

Single source of the truth!

The PP is refreshed daily automatically.

A Microsoft Business Intelligence solution from end to end with built in security with nice visualization and user interaction to discover patterns and insights, to base decisions and steer the business.

Quite nice!

Officially back on Microsoft Path

I started my career in programming with Microsoft Visual Basic in 1995-6.

And Oracle & Crystal Reports.

It took some time but I switched over to MS SQL-Server and SSRS reporting.

Having said that, I've been out of programming for a year and a half.

So now that I'm getting back into programming, I'm really glad that it's Microsoft Visual Studio this time.

It has such a great IDE for Client programming, Services, Web, even Mobile.

It integrates easily with XML.  And the suite of Office products.  And MS Reporting Services.  And it runs atop MS Windows Operating Systems in MS Internet Explorer.  And MS Sharepoint.  Just about all Microsoft products integrate well.  And MS SQL-Server.  The IDE is just more intuitive.  And there's so much sample code out there.  And a lot of online resources.  And you can pump out apps fast.

They really have a product for every type of product.

Quite amazing.

For some bizarre reason, while programming in Visual Basic, I really wanted to program in an Object Oriented language.  And it took my many years to accomplish that.  And what I discovered from programming in Java, it's not all its cracked up to be.  Multiple IDEs by different vendors, multiple iterations of J2EE, Struts, Spring, Hybernate, etc.  Too many moving pieces.

Write once, debug everywhere.

So I'm now officially back on the Microsoft pathway to higher enlightenment.

Sure, I see the mass exodus in the news, LinkedIn and Twitter.  Top programmers heading to other company's.

Either way, MS will be pumping out quality programming utilities for years to come.

And I'm along for the ride!

My Intro to MS Data Mining Excel Add In

Well, being in my new position I've had a few days to research.

Today I chose Data Mining add in for Excel.

The download is straight forward.

At the end it prompts you to find a SQL Server 2012 Analysis Server to connect to.

So I just happened to have one as we installed it yesterday.

So I opened an existing Excel Spreadsheet, formed a table of data, clicked on Data Mining, and had the entire arsenal to begin mining.

I played around a bit and created some models, not really understanding what I was doing.

Nonetheless, it produced some results which I thought were great.

Then I logged onto SQL-Server Analysis Server and sure enough, my 3 new models were copied there as well, which can then be re-processed, etc.

So first of all, I'm getting to dabble in some new technology.  Secondly, this new technology will hopefully pay dividends once I get more understanding of the new databases, new business rules and new customers.  Third, I have learned more in 2 days on the new job than in 1.5 years at the old job.

Just sayin'!

6/25/2012

New Job - First Day

Well, today I started a new job.

Filled out paperwork, got my new cube, installed software all day.

SQL-Server 2012 and Visual Studio 2010 along with Team Foundation Server.

Got SSRS web working locally.

They have Great Plain, SalesForce in the Cloud and they are really into Power Pivot and Sharepoint.

I won't be blogging too much about the actual work, etc. after reading the non compete paperwork and possible implications and such.

However, it looks like a place to settle down and get some real good experience!

They have an espresso maker, all you can drink as well a plain coffee.

Who knows, may have actually found a descent place to work!

6/24/2012

Report Writer, Data Warehouse, Data Discovery vs. Data Scientist

There is a real difference between Report Writer, Data Warehouse, Data Discovery and Data Scientist.

The Report Writer implies a relational database, with a 4GL language sitting on top.  With SQL queries, Summary & Detailed reports,which are scheduled and consumed.

The Data Warehouse is a layer up, moving data from relational sources (ETL) to a Star / Snowflake Schema database which flattens the rows for fast consumption.  Changes must be thought out in advance and with precision, which slows down the process.

Then there's the Data Discovery role, which uses tools such as Power Pivot, Qlikview or Tableau.  These tools allow mashing of disparate data sources in a desktop environment.  Users can see data instantly, creating graphs and charts, slicing data, etc. which could them be ported to a server for enterprise consumption.

And on top of that sits the Data Scientist.  Where they mine the data.  And forecast.  And use statistical analysis.  I like this video by Mark Tabladillo giving overview of Data Mining using Power Pivot & Excel.

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DBI204

Mark gave a presentation to the Tampa Bay BI group a while back but this video gives good intro to Data Mining.

So the four roles could be separate.  Or they could be combined into one person.  Depends on the organization and developer skill set.  Either way you slice it, users want and need access to their data to base decisions.

There's also the advent of Big Data.  Not sure which bucket this would be yet.

6/20/2012

Windows Azure DataMarket

Power Pivot is a great new utility for Microsoft Excel.

It's a plug in where Power Users can mash data sets and view in Excel Pivot Tables.

One thing that's kind of cool is mashing external data found on the internet.

One cool place to obtain that data is Azure DataMarket.

https://datamarket.azure.com/

There, you can purchase or obtain free data for your mashing.

You can search for specific data types, run sample queries prior to purchase and then download.

Once downloaded, you can immediately begin mashing and slicing the data in Power View.

Tell 'em Jon sent ya!

Quite nice!

Power Pivot for Excel 2010 You Tube Intro

Microsoft has jumped into the easy to use analytical tool for Power Users.

Leveraging it's powerful Excel application.

Plug in called Power Pivot.

Here's a few demo's to whet your appetite:

http://www.youtube.com/watch?v=gAxX3NHVlkc&feature=related

http://www.youtube.com/watch?v=zaKUQrwDPcs&feature=related

http://www.youtube.com/watch?v=WBLyH0bssXo&feature=relmfu

It's kinda funny video!

Data Quality

User called in today.

End of year reports.

Asked why some data missing.

I researched.

We have the same name in three different locations with 3 different spellings.

I can't believe it.

Data quality, echo, echo, echo.

Can anybody here me, echo, echo, echo.

I am so ready to start my new job.

6/19/2012

Query IS Correct

I worked for a company once where I was to implement some process' for the Month End.

So I reviewed the existing code.

Basically, it was written in Visual Basic 6.

There was an outer query, which looped thousands of times.

Each time it called an inner query.

I reviewed the code.

And found a bug.

So I brought it to my boss' attention.

He said "that is impossible, the entire company bases it's financial life on that query.  The query IS correct".

I said "okay boss".

And that was that.

A few days later he called me into his office.

"So tell me again what is wrong with the query."

I showed him.

Sure enough it was wrong.

So he told me to fix it.

And then I reviewed the code for performance.

I removed the logic from Visual Basic Inner / Outer queries and put the code into SQL-Server stored procedure.

It reduced the run time from 5 hours to 10 minutes.

And then I was the month end specialist for the next 3 years.

6/18/2012

Be Curious

I read a quote recently, about the famous Albert Einstein.

He self proclaimed that he wasn't that smart, just very curious.

I'd say he was 'open minded' as well.

And a 'tinkerer', someone who likes to see how things work.

Which I suppose is another name for 'curiosity'.

And I've seen another poster where he claimed 'Imagination' was the key to intelligence.

The world could use more 'doers' than 'watchers' and more open minded people.

And it's not a race, sometimes the Tortoise wins over the Hare.

And the end of the day, have you made a difference?

Have you left the place better than it was before you got there.

Most of the strides of mankind were a derivative of some other quest, a by-product discover.

The journey is long and winding, each step a block in the final destination.

Everything happens for a reason and nothing is wasted effort.

So look around, figure something out.

And be curious!

6/17/2012

Learning #Tableau Desktop

While running reports today for a client, I had a chance to review the product Tableau.

What was most impressive was the ease of use, the simplicity.

They have a desktop edition.

User can connect to Files or Servers.

Once connected, the program deciphers the Dimensions and Measures.

User can drag n drop fields onto the Pivot table and data instantly appears.

You can connect to a data source (single or multiple tables), with links back to the source interactively, and you can import entire data structure or partial data by writing custom SQL.

User can hold down the shift key, click on multiple fields and then use Best Practices Chart suggestions, meaning that based on your field selection, the appropriate Charts appear.

Great for Visualizations.

When bringing in multiple tables, I would say understanding Joins to be the toughest part for non-SQL developers.

However, user can select Inner, Left or Right without having to write code.

Connecting to Excel, flat files, Databases, etc. is easy and straight forward assuming you have the credentials.  In addition, they have an ODBC to connect to anything not listed specifically.

User can view data easily.

Users can save data sets to the repository, which are also distributable and can be published to the server.

And you can connect live, import all or subset of data.

In addition, you can actually copy and paste raw data off the web and drop into the Tableau utility clipboard  and it auto creates a table which can be viewed and immediately joined and Visualized.

So you can analyze and mash external data sources.

Customization, every item can be customized from multiple angles.

Colors, fonts, size, grouping, the list goes on and on.

When I think of traditional reporting, I think of data sources, business requirements, writing code / SQL, pasting into report designer, formatting report,  grouping, adding parameters, loading on server, setting permissions.

However, with these new GUI based report visualization utilities, you don't need most of that.

Instead of learning all that, you basically take the data and play with it.

It's a playground to view the data from infinite angles and see the trends and the points of interest and things that look out of place.

So in traditional reporting, you have to ask the questions ahead of time which the report should answer.

With the new methodology, you don't know the questions up front, you discover the questions and the answers as you go.

This is quite amazing!

I haven't gotten to the more advanced concepts yet, as I've been viewing the online tutorials for basic stuff.  They also have Advanced features which I haven't had time to view yet.

Really exciting!

Here's a post from last week regarding Tableau as well...

http://www.bloomconsultingbi.com/2012/06/downloaded-installed-ran-tableau.html

6/15/2012

Logic Moved as Far Away as Possible

Back in the old days of programming, there was structure code.

Then someone decided to have reusable code.

So they put the logic into a sub-routine.

And if the routine returned a value, it was a function.

And that function was moved to a Class.

And then the Class was moved to a DLL.

And then the DLL was moved to a consolidated server which pooled processes COM+.

And then the COM+ was moved to the web as Web Services transmitted through SOAP.

So we have moved the logic as far away as humanly possible.

But it's still just a function, a snippet of logic that performs a task / routine based on parameters.

Nothing really changed.

Bloom Consulting, Sole Proprietor, Jon Bloom, Florida

I started Bloom Consulting in year 2000.

I first liked 'Small Shop Software', except I didn't go with that.

I went through an attorney at the time and registered with the State of Florida.

Got a Federal Tax ID Number and put ad in local newspaper 'doing business as'.

At the time, a friend of mine who owned a software company asked me to write some code for him to integrate Visual Source Safe into his new product so users could check out, check in, etc. from the Visual Basic client.

Tons of custom DLLs for Middle tier & Database Tier.  Let me just say his code was a masterpiece.

I worked from my home office and we chatted through email & phone.

I got paid and that was that.

Later, I did some web work for a small company in St. Petersburg.  After some time, the programmers weren't getting paid so I demanded my last pay check and never went back.  That was after hours & weekend work.

Many years later, I found a gig through Craigslist.

A British company, who has tentacles all over the world, had me write SQL queries to pull data from international databases.  I wrote about 500 queries and added to SQL-Agent.  However, I also got some good SSRS experience.

Then I worked for a prior client doing SSRS & SSIS & T-SQL from home.  That gig is going on 2 or 3 years now.  They are going through some turnover with the Manager and Developers at the moment.  So I think I could get some good hours in while they transition.

Lastly, I did some contract work for someone local who just started a new company.  I did remote work from home, dialed into the client, doing T-SQL, SSIS and Master Data Services.  Was a good gig, just didn't get renewed.

So there you have it, the history of Bloom Consulting, Sole Proprietor, Jon Bloom.

If you search the web, there are plenty more company's with the same name or similar.

I'm not sure how that conflicts but I went through legitimate attorney when setting up shop.

And I've renewed every 5 years with the State of Florida.

Oh yeah, I also created a web site for tennis called Tennis In Bloom.  It was a 'ladder' system where people could challenge up and if they won, they got that person's spot.  It was live for a summer, where a local women's group accessed, however, I shut it down before it got bigger.  There are many free sites out there now similar.

I basically just program for fun, after hours and on weekends to learn new technology and earn some extra cash.

Not a bad deal if you ask me.

Everything you wanted to know about current BI Article

This is a very good article explaining the current Business Intelligence Magic Quadrant Platforms:

Magic Quadrant for Business Intelligence Platforms
http://www.gartner.com/technology/reprints.do?id=1-196WFCB&ct=120207&st=sb

I've used Business Objects / Crystal Reports, Microsoft SQL-Server Reporting Services, OBIEE, Actuate and I prefer SSRS myself.

As far as the up and comers, I've tested QlikView & Tableau.

If I were developing reports, because I've been in the Reporting industry for 16+ years, I might prefer QuikView because it's very powerful and has great depth of customizable features.

However, if I were pushing the software upstream to the Exec's & Power Users who have limited or no formal SQL training, I would prefer Tableau, based on my initial evaluations.

Granted, I have not spent an enormous amount of time with both products.

That's my take on it, for what it's worth.

6/14/2012

Downloaded, Installed & Ran #QlikView


Earlier today I posted a Blog explaining my first attempt at Tableau.

In all fairness, I thought I should follow up with a Blog about QlikView.

So I installed the product, actually have the option of 32 bit or 64 bit.

At first I was looking to attach to a data source.

By using the wizard, you get to choose from an Excel .xlsx file.


The Wizard is self explanatory, user has ability to select options and such.

However, if you click the cancel button, and then click the 'Edit Script' button, you then get the option to choose from multiple data types, ODBC, OLE DB or QVSAdminDataProvider.dll (64bit).


I choose Table Data, pointed to a pre-existing Excel doc, selected Headers and clicked okay.

Then changed the data table name followed by a colon:

You can then proceed to add multiple Excel docs, set labels, give it a name, add more files / tables.

After saving, you can click the ReLoad button and view your data.



You can then click on Data View and see your new table with Field names:


You can then click on the drop down menu --> Layout --> Select fields...



From first glance, it looks like there are a lot of customization features included.

Similar to a programming language.

And user has ability to see and modify the underlying scripts.

It looks like an awesome application for users to view data and discover insight!

I will need more time to work with a real life example to get a better feel for everything.

I will also recommend this product at my next company as a possible solution for Power Users.

Cheers!

Downloaded, Installed & Ran #tableau

I've been in IT for 16+ years.

And of that time, I have been creating reports the entire time.

Crystal Reports, Access, Actuate, JReports, SSRS, etc.

So I've been hearing a lot of discussion about .


On Twitter & LinkedIn, I've actually seen some Microsoft BI people leave and go to Tableau.

So today I downloaded, installed and ran the trial version (and registered).

And I was impressed.

Assuming you have sufficient credentials, knowledge of the Database (data source), and some understanding of the Tables & Field names, you are good to go.

I connected to a SQL-Server data source, kept it simply by bringing in 1 table and was off and running.

You can drag and drop fields onto the Dashboard, add some totals & sub-totals, and within minutes you can see your data.

I added some graphs, put some selection criteria, did some sorts, etc.

Very impressive!

A fully functional reporting utility for Power Users.

I did not have to write a single line of SQL.

This has lots of potential.

The other thing to keep in mind is the potential Data Sources:

What else can I say?

Awesome!

Obviously without a real world problem to solve at this point, I have not spent the time to create an actual data mining expedition, so I don't know the full capabilities concerning + and - features.

However, I will present this option as a possible solution in my next endeavor, as I'm changing jobs in a week to work for a Software company.

Cheers!

6/13/2012

Conserve Memory / Battary Life

Back when computers first appeared applications had to be loaded into memory from floppy disk.

And I remember conversations with my father who programmed in the late 1960's.

He said they really had such limited amounts of memory that they really had to conserve it.

Fast forward to now, when programming for the mobile, you really have to conserve how much activity you do as to preserve battery life.

Things change over time, yet they remain the same.

We used to have centralized mainframe, then PCs, then client server, then web, then Cloud - full circle again.

Solving Problems, Insights & Action

Business Intelligence is about solving problems.

About providing insight.

And guiding actions.

If you as a developer can do all three, you will be a success.

And that's what I want to do for a living.

Actionable Insight

Users don't care about the infrastructure.

They don't care about structured data.

Or unstructured data.

They don't care about Cloud architecture.

What they do care about is "actionable insights".

The want value from their data, from their business.

And you have to have 'Big Data' to do this.

To predict.

To get insight.

To look for patterns.

To give you a competitive advantage.

To look for the 'white space', what the data isn't telling you.

For new opportunities.

For growth.

To better mankind.

And get the economy going again!

Learn Like a Sponge

It seems like our culture has been living through the rear view mirror the past few decades.

Same old songs on the radio, same re-runs on the television.

As well as a complete dumbing down of people through movies, 24 hour news/entertainment channels.

However, there is SO much to learn in programming now a days.

So much freshness.

It's alive with energy!

Just in the last year, with the
  • Public & Private Cloud,
  • Business Intelligence / Analytic,
  • Big Data
  • Hadoop (unstructured data)
  • Social Media
  • Databases
  • Web Programming
  • Mobile
  • Security
  • Open Source Technology
  • Current trends in the marketplace 
  • And the list goes on
It's mind blowing indeed!

I'm doing my best to keep up with all the changes.

And the new lingo.

And all the technology that seems to be springing from the head of Zues.

Not only that, but the speed in which new technology appears is also mind boggling.

It verges on impossible to keep up.

It's like running as fast as you can for as long as you can and still get further and further behind.

However, that's what I love about this field and this career.

With all this newness, it's difficult to get bored.

You really have to be a generalist across a dozen technologies and you have to have deep knowledge of a half dozen as well.

You have to be flexible, agile, ability to problem solve, be social-able and learn like a sponge.

6/12/2012

Shut the Dev Server

I worked on an application that used Oracle.

And we had an offsite location on the West Coast.

And we went live with this application.

And the tech support guy called my boss and said the app was way too slow.

What did Jon do to slow it down so much.

So we called him from my boss' office, I asked him if he verified the entry in the TNSNames file.

Guy said he did.

So we couldn't figure it out.

I went back to my desk and troubleshot the situation in my head for awhile.

Then I went back to my boss' office.

And said, "I think they are pointing to the dev server."

My boss said, "Go shut the dev server."

I said "Okay boss".

Within a few minutes we got a phone call from the guy out west, said "Why did you shut the production server?"

So as you can see, troubleshooting this problem, in a black box, with people unwilling to help was possible.

And that was a major win for us.

And so it goes!

BI Sentiment

Business Intelligence is a definite part of the business culture today and going forward.

With Social Media, people are now able to track 'sentiment'.

By that, not only are they viewing who, what, where, when, they are looking to find people's moods as well.

How was a particular experience?

How do you feel regarding a certain event?

Do you have a positive or negative opinion surrounding a product?

This information is extracted from Twitter feeds, Facebook, etc. to look for 'keywords' indicating + or - reactions.

This data can be aggregated, reported, sliced & diced so that people can change their actions based on feedback.

This is quite a powerful concept.

And it's popularity will only increase as time goes on.

So how do you FEEL about that?

Retirement On Hold

We get schooled in our childhood.

We learn enough to make a living.

Then we work our 40 years.

And then we retire.

What I've noticed is that some people do NOT retire.

They just keep on working.

For one reason or another.

Perhaps they blew their life savings during the Tech Bubble stock market hoping to get rich quick.

Or maybe they lost their nest egg when they refinanced their home and then the value split in half.

Or maybe they spent their savings on kids, life, etc. and never stored away retirement savings.

Or maybe their children returned home after college graduation because they can't find employment.

Or maybe some people need the insurance.

So some people are not in a position to retire into the sunset.

Which means that the positions that were supposed to open up and allow the next crop of indentured servants into the workforce are simply not there.

There was an article in the media questioning if the economy was wrecked on purpose.

Well, if you look at the all the factors, well then yes, our economy hit a perfect storm.

So you have to look at who benefits while others lose everything.

And I suppose that would point to the culprit(s) if in fact it were true.

I don't have all the answers and I don't judge or place blame.

I just watch the patterns and report back my findings.

Is the Business Divorcing IT?

We all talk of Business Intelligence and Business Analytics now a days.

So the business can make better decisions.

However, all the talk I hear is from the vendors perspective, or the analysts perspective or the consultant's perspective or the CEO's perspective.

I haven't heard much from the actual decision makers.

Why is that?

What improvements would they like to see?

How can they get more accurate data faster?

What are their goals at the end of the day?

Are things moving in the right direction?

Have they already by-passed IT?

Do they really want IT's help?

Is IT forcing a solution on them as to remain in the game?

Is the Business Divorcing IT?

Microsoft Big Data Symposium Orlando 2012

I attended the Symposium in Orlando from Microsoft.

The topic was Big Data.

However, they also discussed a lot of the new features in SQL-Server 2012.

I was really impressed with the way MS runs a show.

They had name badges pre-printed.

Offered snacks, water and COFFEE.

They had vendors to discuss some products.

The sound system was awesome.

The day started out with some cool uplifting videos.

And smoothly progressed from one topic to another.

The flow was transparent as each speaker took the stage.

Discussed their topic.

And so it went.

There was time in between for breaks and I asked some questions to the speakers.

Lunch was good, although I did pick one of the few boxes that had no cookie.

I sat with some ex-coworkers and some people from all over the country.

We chatted a bit.

Then more coffee, more speakers.

There is definitely a lot of new stuff in the new version of SQL-Server 2012.

They had an open discussion at the end, a panel run by Robert Skoglund from Microsoft.

Some good interaction and viewpoints based on differing industries and how they handle big data.

I asked a question, "what is the role of the data scientist going forward?  Is it a specific person or a group of people?  And do you see a new role of Chief Business Analytics Officer?"

Each panelist answered the question and with some of the new technology, and self service BI, they are hoping the user can get much of their data themselves.  Although some places will still have a need for the Data Scientist.

I Tweeted throughout the event.

While exiting the host of the day asked me if I was @SQLJon and he thanked me for tweeting.

We then headed to the bar and talked about data and such - good conversation (no alcohol of me by the way).

Then chatted a bit more and then headed home.

MS put on a good show, learned a lot and enjoyed the free lunch, validated parking and all you can drink coffee!

Coder Role vs. Manager Role

In IT you have a few career paths.

Stay technical, as a programming coder.

Or move into Management.

Some enjoy the progression.

More responsibility in management, more  money.  And more stress too.

And at some point you distance yourself from the day to day technology.

I remember a director at my old job, said going into supervisor position is a slippery slope.

Because once you head down that path, it's just a matter of time before you lose your tech skills.

However, I hope to prove him wrong.

I still have my skills, even though I've been supervising the past year.

It was a hands on role, I wrote code every day.

And I worked at my part time job writing code every night.

So I made the conscious decision to go back to the coder role - 100%.

Supervising is okay, I liked the authority in the sense that I could implement new procedures.

The day to day things like time off, I didn't get to involved in that.

Whatever they asked for they got.

I also enjoyed delegating the work based on available resources.

And meeting with customers and setting expectations.

All good skills going forward.

Most managers I've seen in IT were coders at some point.

And then they lost their programming skills.

And now they must be in management and can't go back.

It's a choice I suppose.

6/11/2012

Empower the Users - Bypass IT

Tradition Report Writers pull data from a relational database, into a report, for ad-hoc run by user, either on the web or client side, to consume data, to get a pulse of the business and eventually base a decision or action resulting into betting their position than before.

This takes time.

Time to gather specs.

Time to locate the data.

Time to write queries, often complex.

And create a report.

Etc.

This model works.

Although the process could be improved.

It could be streamlined, by removing the middleman.

The role of the Report Writer can be somewhat replaced by improved software.

Now data can be accessed via the web, by the end user, as long as the infrastructure is in place.

That means the data would have to be known ahead of time.

A layer of meta-data needs to be created.

And published.

And accessible for end users.

So security must be in place.

This is already a reality.

There are software vendors such as Tableau and QlikView  which have done a great job to empower the users.

However, Microsoft has improved their BI stack with the latest version of SQL-Server 2012.

Now users can gather disparate data from multiple sources including Public Data in a Microsoft Excel 2012 Plug-In called Power Pivot, join tables based on common keys, add Business Logic using DAX language, save the data to SharePoint, and create reports using PowerView.

If the data is good and used in many reports, IT can convert that dataset to SSAS Cubes for Enterprise level refreshes and consumption.

Users can run their reports, save them, schedule them without a ticket to the help desk or a phone call to IT Report Writer.

By empowering the users, you get faster access to the info, for quicker decisions, and that's the basis of Business Intelligence, get accurate data to right user in fast time, in order to take action and improve process / increase revenue.


6/10/2012

Big Data Symposium Orlando 6/10/2012 #sqlsymposium

Attending the Microsoft Big Data Symposium in Orlando June 10, 2012.

Event started at 11am, introduction by Bob Baker.

This is the 9th and final SQL Symposium provided by Microsoft with over 4k attendees so far.

Eugene Saburi gave a great presentation.

Said with the recent Data Explosion, an increase of 44% this decade while IT has grown at 4%.

Microsoft has a new World of Data.  Not just pull data from relational databases and OLAP cubes, there's a new push to think Holistically.

Intro of massive data or Big Data through Hadoop, need for Data Governance, variety and locations of data.  Become more diverse.

More data sources and mashing public information.

Need to do risk analysis to provide more accurate insight.

 Mobile has become more important, bring your own device.

Richer apps.

Any device.

Any time.

Microsoft's goal is from end to end.

With SQL-Server 2012.

3 things, Mission Critical, Insight and Cloud.

Cloud Ready.  175 new features.  Major product release.

Always On feature = high availability.

Proactive troubleshooting - watches patterns, sends alerts if DB running into trouble before major problems occur.

BI top 3 items of conversation among CIOs.

Been a low success track record.

Need for analytics, not just cubes/dashboards.

Need for front end self service.

Can't wait on IT with turnaround times of 3-4 months.

Intro of Power Pivot extend with PowerView.

Has stunning visualizations and discoverability.

Rich self service.

End user vs. IT

Managed self service.

Integrates Active Directory and Sharepoint.

Need the right user, right data, right time.

Credible consistent data.

Garbage in garbage out.

Need for High Data Quality.

Intro of BI semantic model.

Big Data - Microsoft  full Hadoop support.

More data = more accuracy.

Most used statistician tool = MS Excel

Data Scientist pulls data from Hadoop.

Into Power Pivot,  apply biz rules and Publish Model to Sharepoint.

Single version of the truth.

Once in Sharepoint, connect real time with PowerView.

Insight speed of thought.

Using Cloud can reduce confusion with licensing.

Faster time to market.

IT solves Biz problems.

Cloud handles app.

HYBRID IT:

Productivity
  T-SQL works on 2012 and Azure

Scale up or down - elastic

Immediately analyze and react.

Highly available.

Balance IT efficiency and reduce costs.

Compliance.

Traditional IT
Private Cloud - pool resources - virtualization
  Elasticity - scale up or down
  Built in Self Service
  Usage based model
Pubic Cloud

Cloud is NOT all or nothing proposition- a mix.

Private Cloud - for compliance reasons - keep data onsight

Economic reasons

Remove need to manage infrastructure - ie. OS patches

Easy for proof of concepts, quick development, scale up quickly, certified.

And a whole lot more.

More than I can type into a blog article.

However, the food was good, networking, conversations with the Vendors and asked a few questions to some of the speakers.

Good conversations with Robert Skoglund from Tampa Microsoft office and also host of the Vendor panel presentation.

Also, had lunch with some ex-coworkers from Pinellas County Government.

Overall good day, and they validated parking too.

Thanks to Microsoft for offering the free day of training / presenting.

6/07/2012

Swiss Army Programmer

When I was a child, my Grandfather gave me as a present a Swiss Army Knife.

If had all these cool features, like a toothpick, tweasers, big knife blade, small knife blade, scissors, a mini saw, etc.

I thought this was the greatest invention ever.

It had a tool for any project you needed.

A utility knife for every occasion.

And then there a knifes specifically for hunting, fish cleaning, digging, small knives for key chains.

These knives have one or two purposes and they work great for what they were intended.

As a programmer, I like to think of my skills as a Swiss Army Programmer.

I can troubleshoot an IIS server, I can write code in Java, I can program in other languages like .net and Visual Basic, I can write reports in a variety of platforms, I can speed up queries, interact with the Mainframe, etc.

I can do all sorts of things.

Then you have programmers who do one or two things really well, and that's about it.

So as you can see, both types serve a purpose, the utility and the specialized.

One is not better than the other.

Regarding Knives or Programmers.

And I thank my Grandfather for the great gift!

6/06/2012

Silos

In some organizations, territory is well defined.

Don't tread on my turf.

And sometimes information is kept secretive in the form of silos.

And when someone new comes along, and they don't know the secret handshake, well tough petewtie!

The masters of the silos have no intention of dispersing business rules to such lowly mortals.

And so people struggle to find information on their own.

Which is an uphill battle.

I take the opposite approach.

Let's liberate the data and the business rules.

Let's apply some transparency.

No need for the data to be held hostage.

Territoriality is a thing of the past.

Except in some places, it still exists.

I find this form of behavior to be a mild form of bullying.

And if the org's allow it, then that's not the best place to be working.

Now is it?

6/04/2012

Conventional Herd Thinking

We all like to think we are unique.

Just like the other bazillion people on this planet.

We all get our media from the same sources.

We all drive vehicles, own homes, have jobs, etc.

So we all tend to have the same conventional herd thinking.

So when things are good, everyone's out spending.

When times are tough, people retreat.

I tend to do the opposite of what the herd is doing.

When the housing market was booming, I said sell, this market is going to tank and bring a world wide depression with it.

While attending a wedding in Boston, Ma, my father had me speak with our great uncle who is very rich and plays in the stock market.

I forewarned them both of the coming crash.

They were hysterical.  Said buy the biggest house you can afford.

They like the REIT's at the time as they produced great wealth.

I took my lumps, but never backed down.

I sold two houses and netted a hefty sum.

Then the market took a dive.

And my father asked me how I predicted the crash.

Because society moves like a herd of sheep.

Whatever they're doing, I do the opposite.

Also, my old boss Wyly clued me in during 2001-2 daily lunches.

I'm not that smart, but I listen to people and watch things happen.

If I were actually smart, I would have invested my savings in Gold and I would have shorted the housing market.

Then I'd be writing this blog aboard my yacht in the Bahamas!


When Your New Position Ain't So New

We all love getting new jobs.

The thrill.

The excitement.

The feeling that you are needed.

And then you work at a place for a while.

You solve some good problems.

You create some process to improve the place.

And then it wears off.

Noone left to impress.

You don't find joy in the challenges.

People are preventing you from excelling.

This leads to ambivalence and discontent.

I'm sure there's a lot of IT people gritting their teeth right now.

Because they are stuck in dead end jobs.

And they need a paycheck.

And there are some jobs out there.

But the competition is fierce.

And the hiring people are looking for only diamonds in the ruff.

And willing to pay copper prices.

So where does that leave a great bulk of the IT professionals.

I believe they are all just waiting it out.

For the market to turn.

And then there will be some musical chairs job changing like never seen before.

And all the biz logic will be gone with the wind.

Employers should really take time to prune their employees, give them sense of value, give them challenging assignments, bump up their salaries.

Because when the tide turns, your IT people will be in your office submitting their 2 week resignation.

Just some forewarning!

Take on the Project Noone Else Wants

From my experience, most programmers want the cake walk.

They want to stroll in, get the plumb assignments, which is typically "new development", get a chance to learn new code on other people's dime, then ride off in the sunset armed with good experience.

I say do the opposite.

Take on the projects noone else wants.

That usually entails learning the business.

And learning all the systems that support it.

And understanding the users.

And becoming an asset to the organization.

My father gave me this advice early on in my IT career.

He said that's how he survived the tail end of his 34 year career with IBM.

When they were letting go all the hot shots, they kept him on because he knew how the critical applications ran.

I've basically supported a lot of the grunt applications which entailed mostly bug fixes and enhancements.

I wasn't concerned about new development per say.

I actually prefer to support other people code than to write new code.

Short enhancements to improve functionality / user experience.

You can hire the hot shots to write the quick application which I'll end up supporting.

I've done it for 16 years now.

I take on the projects that noone else wants.

Gain Skills, Salary Will Follow

I've always had the belief that you should gather all the skills you can.

And eventually, the high salary will follow.

I remember early on in my IT career, my father said to me, "Don't worry about the salary, you should actually be paying them for getting the skills."

And that has been my mantra ever since.

I would say to people entering the IT world, don't expect your dream salary out of the gate.

Pay your dues.

Learn the systems.

Learn the business.

Learn how to maneuver within an organization.

You may be the hottest thing since slice bread.

Except there may be 10 people lined up to keep you down.

Watch for turf wars.

Be thankful.

And pay your respect to those who have come before you.

And in time, you will be running the show.

And you will have the salary to show for it.

6/03/2012

Artificial Intelligence

Henry Ford automated the process of creating things.

Each aspect of the job was identified and separated into individual jobs.

Thus, the creation of the assembly production line.

And that began an era of expansive growth.

And so now, today, we have a similar phenomena with data.

The data stored in databases = the raw material.

The assembly line is the process of turning data into information.

And that's accomplished through querying of the database.

And creating a final product called information.

This is called Business Intelligence.

However, this too will grow by removing the need for humans to crunch the data.

The applications will soon process data and look for patterns, remember (brain) and learn, and then predict future behavior based on repetitious patterns.

And that is called Artificial Intelligence.

Just around the corner I imagine.

Programming languages that observe, remember, learn, predict as well as guide future behavior.

Without the assistance of human intervention.

6/01/2012

Holy Grail of BI

What is the holy grail of Business Intelligence?

A glimpse into the minds of each and every customer, their every thought process, past decision and potential behaviors, in a nutshell.

And the goal is extract that information based on any available resource.

And since Data now captures high volume of behavior over time, because the prices have dropped significantly for storage and extraction software, this is a hot item.

Perhaps it might be easier to discuss every person with the mother or father, you know, over tea.  What was Johnnie like as a kid, did he like to spend money are large ticket items?  Was he more likely to spend the extra money on upgrades on his 3 speed bike when he was 8 years old, or was he more practical and saved his money?

Or I suppose you could break into people's answering machines like the media outfit in England did, that would surely allow you to get into the minds of the people, nah, maybe not legitimate way.

Perhaps we should just continue down the path we are on and find ways to automate that extraction.

It's my belief that the reason we are struggling so much trying to define it, is because its actually a subset of artificial intelligence.

And that's my take on it.

Mountain Living