11/19/2012

Assembling Data Sources Into Single Query

I've been putting together the mother of all queries.

Joining data from 2 separate data sources, then combining that with SalesForce, then combining that with a final data source.

Let me say there's a lot of data.

And let me say the query is pulling from multiple servers and multiple databases.

And the queries take a long time to run.

So I've been piecing them together, slowly.

Mostly in Temp tables, with indexes.

So today I moved some of the data closer to the source, and now the query runs a lot faster.

So I can actually see the data results after a few minutes opposed to hours.

Which means tomorrow I should be able to dissect the relevant information to answer some questions we posed as hypothesis.

My goal is to answer the questions, and document the methods in which the data was gathered.

As there will always be exceptions, assumptions and work a rounds.

However, in the end, we should have a global view of the data mashed with multiple data sources to then find insight.

And give the uppers an ability to manage the business in a way they were never able before.

And put the data into a Tabular Model cube for interrogation.

That's really what it's all about.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Thoughts to Ponder