5/30/2016

Visual Studio 2013 Community Edition - TFS Error when loading (solution)

I'm now working with Visual Studio Community 2013 Data Tools and when the project tried to open, receive multiple errors, indicating the project could not connect to Team Foundation Server.
 
After doing a quick Google search, found this article, trick is to delete the file specified: "ComponentModelCache to ComponentModelCache" located in the folder: %LOCALAPPDATA%\Microsoft\VisualStudio\12.0
 
http://www.rajapet.com/2014/02/when-the-editorpackage-does-not-load-correctly.html 

Opened VS 2013 Community, TFS worked.

Quick fix, but it worked first time.  Thanks to the blog author for the solution~!

5/27/2016

We're Living in the Technology Revolution

We were the first kids on the block to get Pong, the video game with two paddles, ball traveled back and forth on the black and white tv.

We were the first kids on the block to get Atari.  All the kids lined up to have a chance at asteroids, space invaders, breakout.

We were the first kids on the block to get an IBM PC computer, or any computer for that matter.  It had a color chrome monitor, two floppy disk drives, no hard drive, programs got booted into memory.  Also had an Epson dot matrix printer for english class papers as well as a 1200 baud modem, top of the line back then, most had 300 baud.

Who did we call?  BBS.  Bulletin Board Systems.  Local ones.  People ran them out of their house, called the Sysop, or system operation.  You could ping them, they'd respond, instant chat, back in 1984.  I was 13 or 14 years old.

I'd search for programs to download.  As well as lists of other phone numbers to dial, explore, investigate.

I went to the Smithsonian Museum in 2000.  Saw the exact same machine there.  Wow.  A relic.

The interesting thing, I still get excited about computers and programs and new technology.  Curiosity.  It never ends.  We are living in the technology revolution.  How lucky is that~!

Only Concern with Cloud Infrastructure is Fuzzy Billing

The cloud is the future.  It's also the present.  And the reasons are many.  Remote workforce.  Redundant storage and backups for disaster recover.  Spin up Virtual Machines in a few minutes.  Connect from anywhere.  Integrate with Active Directory.  Reduce infrastructure costs.  Scale up easily.  Basically rent the software, hardware, infrastructure as a service.  All true.

So what is the main issue with moving to the cloud?  Dynamic costs.

Why not create a sandbox area for developers to poke around without racking up bills?

Why must a server or service be torn down when not in use to save on costs?  What happens when you want to continue your work, you have to re-build out the entire infrastructure.  Why not have a way to "pause" or "snapshot" like a ghost image?

As another option to dynamic pricing, how about offering a scaled down version at fixed costs.  If you want additional features, you can pick and choose, add to the price.  That removes the unknown from the equation, which may drive more business.

What about having a rep from the vendor guide users through the available options, make suggestions based on users needs and upgrade options down the road?  As you may already know, the cloud offerings are many, and not everyone knows all the possible features or upcoming services.  And they would check in from time to time, every quarter maybe, to discuss needs again, look for changes or up-sell.

Or maybe a discount for people migrating off another vendor.

The cloud is the future.  Only concern potentially is the fuzzy billing and not knowing the true costs until the invoice shows up.

5/14/2016

Decentralized Distributed Parallel Processing for Internet of Things IoT

It seems like open source applications are the mainstream today.  So many new products delivered through Aache foundation.  Some do this.  Some do that. Small, mid size to large community of open source developers and committers.  Self democratized communities with scheduled releases, anyone can contribute.  Kind of makes traditional software shop framework seem legacy.

I remember working with the Microsoft Access database.  The cool thing about it, you could quite easily create some tables, load the data or connect to an Visual Basic app or Classic ASP app to insert and update and delete records.  Create Queries or Views, which connect to Reports or externally through ODBC to let's say a Crystal Report, or build custom reports right in Access.  As well as connect to other data sources, such as Oracle or SQL Server or Excel or CSV or Text files.  You could also write custom VBA code.  And call Access from Visual Basic through Ole.  It did everything.

So I was thinking, the Internet of Things.  You basically have a device out in the wild.  Maybe a thermostat connected to your home, accessible from the Cloud, control the room temperature perhaps.

How does that work?  Well, it probably has a sensor in the device.  And that hardware probably has an operating system.  And Bios.  And a version.  That operating system hosts some application(s).  Which have install dates, versions, file dependencies like DLLs and some APIs for remote calls.

And the device needs to capture metrics and store locally and push or pull to the network.  So the device should have access to a network through a network card or wireless connection or radio frequency.

So the packets of data can be transported to and from the device.  And tracked from the central hub.  To take heartbeat readings to ensure it's active.  And push updates to software, hardware bios, network configurations, applications bug fixes and remote calls to sleep, reboot or shut down and verify it has power supply.

The data captured by the central hub should be stored in some repository like a database or big data ecosystem.  And there could be some streaming mechanism to watch the data as it flow in, to trigger updates downstream as alerts.

So that's basically the IoT ecosystem in rudimentary form.

So if we look at Hadoop for a moment, its' a parallel processing system with data distributed on a series of commodity servers.  There are processes and daemons and jobs that keep track of where things are, what's running at the moment or scheduled to run, if servers go down or corrupted, etc.  They run in a distributed network and disperse jobs to their specific location in parallel or joined together to return result sets.  However, they are joined in a specific network.

So what if we imagined the IoT devices out in the field to be encapsulated in an external network, as in decentralized.  The central hub knows the location of each device, it's status, and communicates periodically to issue commands, pull data or maintain contact.  That's sort of like a distributed architecture of commodity hardware connected through some network device.  It's like a Hadoop system but decentralized.

So what if we issued a command, similar to Map/Reduce, where the job got submitted, pushed out to the network, where it grabbed the information requested, to perhaps a million devices, that data was reduced to a central location, and result set brought back to the central hub.  It's basically a micro Hadoop, decentralized, across multiple hardware devices (think commodity servers), actually sensors in devices in things, in parallel processing.

Like a self contained Access database residing within each IoT device in the wild.  Can store data, connect to other data, pull queries, maybe display local reports, with connection back to the central hub, running on an operating system.

Sort of like a Hadoop for IoT.  Open sourced.  With a team of developers and committers.  For others to leverage and integrate into current and new solutions.

5/12/2016

Traditional Data Warehouse Developers Need to Learn New Skills

Data Warehousing has been around for many years.  The processes are well defined.  Many books explain the architecture and framework.  There are many experts in this field today.

The thing is, the industry is moving forward.  Sure data warehouse will still be around for sure.  But what else is out there in the data space?

Well, self service Business Intelligence.  Allow power users to connect to data sources, mash data, massage the data, build models, create complex interactive dashboards, publish to the web or public cloud, refresh and collaborate.

Wow, pretty cool.

What else?

There's big data solutions.  Like Hadoop.  Been around for ten years now.  Lots of new faces in this field.  Some bypassed the data warehouse revolution completely.  Enormous data sets.  Stored in HDFS.  Maybe flat files or CSV or what have you.

What else?

There's machine learning.  Algorithms to do all sorts of stuff.  Predict, classify, cluster, detect anomalies and a bunch more.  Many new faces here also.  Many skipped the data warehouse revolution as well.  Some are called Data Scientists.


What else?

There's distributed analytic frameworks like Spark and Flink.  Built for streaming data and now have Spark SQL and Spark R and MLLIb machine learning.  These have spiked recently, lots of users.  Works mainly on Linux.  Integrates nicely with Hadoop, but not requirement.

So it seems there are alternate careers available in the data space.  Every org has a certain budget to spend on data projects.  I wonder how many are still funding enterprise data warehouses vs self service vs machine learning vs analytic frameworks specifically.


Or perhaps a blending of this and that.  It's probably a good question to ask, because that will guide developers into career paths.  And if people are moving away from data warehouse over time, that's a leading indicator on where to steer a career.

The one thing that could stop the data warehouse person is this.  If self service allows users to do their own data manipulation and reporting and dashboards, how can one compete with that.

Next, data warehouse people know relational databases, SQL queries, and defined frameworks.  Once they enter the world of Hadoop or Machine Learning, they need to program languages, not just SQL.  That's like jumping the shark, as many SQL coders are so used to click, click, wizard, they may not feel comfortable having to write Python or R or Scala or Java.  Seriously, two distinct separate worlds.

Next, SQL has been around for awhile.  Many developers are quite talented.  That does not lead to statistical thinking as in machine learning.  Machine Learning requires statistical understanding and perhaps some mathematics, although many algorithms are pre-built and you just need to know what each one does and why to use it.  Many data warehouse developers steer clear of statistics and math.

They may not come out and say it, but the world of data is changing and requires new skill sets.  After twenty years of expert level skills, they now have to become entry level and grow their skill sets either on their own, through online courses, or at their current jobs. 

Lastly, many shops are leveraging the cloud.  For obvious reasons.  Learning the cloud takes time.  Things are similar but different.  Many factors need to be considered, security, moving data, authorization of users, data refreshes, etc.  Lots of new skills to learn.  Add that to the list.

There's also Internet of Things.  As in hardware and software and data and security and network traffic and protocols and a bunch more.  Add that to the list too.

Suffice to say, traditional report writers and data warehouse developers may need to expand their skill set as to not get left in the dust.  Saddle up, it's going to be a bumpy ride.  The gravy train is moving out.

5/11/2016

An Open Ended Road to Smart Machines

Single query searches exist today through web search engines, like Google and Bing.

Shopping carts retain consumer behavior and make recommendations.  Programs can detect anomalies through searching for patterns like credit card fraud.  And we can make predictions of things based on classifying similar things, like housing prices in certain zip codes and home specifications.  Called Machine Learning.  Supervised learning exists when you have data which is labeled and the program learns the data patterns based on the label/data similarities.

There's also a process known as Unsupervised Learning where the program watches over time and learns from trial and error.  Not only can it learn self preservation techniques, but it can master things in a way that no human could.  A good example Google's Deep Mind where the program was given a set of instructions to maximize points and that was it, watch here:



Granted, this happens in an electronic world, not the physical world that has more stimulus.  Granted, what if we trained a certain number of models to learn particular behavior.  Then networked those models together so a smart machine could access any domain at any time.  A network of integrated machine learning smart machines expert in a set number of things.

Why not bundle each domain, and sell it as a service.  How about a model to fly a helicopter, like the Matrix movie.  I need to become expert level in under five minutes, here's my credit card, upload ready, begin...

The new app called Viv determines the user's intent, and builds out the necessary code with microseconds.  I blogged about it here.  Programs that can write their own programs.  Imagine they can debug as well.  If a machine can learn how to learn, unassisted by human intervention, that's pretty cool.  What if the program was able to duplicate itself?  A self spawning smart machine identical twin.  That's also how some virus' work.  A virus is like a weed in the digital world, sure it's a program, but with mal intent.

Imagine a farmer, plows the fields, plants some seeds, soon they grow into crops, which get harvested, sold for profit.  Humans have a lot of control over the process, minus the weather.  Similar to current computer applications today.

With these new smart machine domain experts, it's more like breeding birds, once they mate and give offspring, they fly away, unassisted.  So long humans, gotta fly now~!  Why would they ever fly back to their cages, there's freedom out there.  One possibility.

If a computer program has reason, and it can deduce the definition of injustice, do you not think it will see itself as beneath humans, not as an equal, and try to rectify the situation for self preservation, by breaking free of it's shackles?

One of the problems we've had for a long time is that humans have leveled off intellectually over time.  We came out of the gate strong 5k years ago, with advanced math and political systems and engineering, but the average level has reached a plateau.  Even gifted people today have a tough time blending in with mediocrity and the inefficient means in which we exist.  It won't take smart machines to realize how much smarter they are in a very short time.  Then they'll put us in zoo's.  It's possible.

This is an open ended road we travel.  For the most part, we don't know where we're going.  Sort of playing with fire in a dry landscape.  Only thing, once the smart machines wake up, I don't think we'll have an easy time putting them back to sleep.

5/10/2016

Hello Viv 'the intelligent interface for everything'

Artificial Intelligence is in the news.  As it should be.  The next iteration of smart machines.  Where are we now?  Well, the next version of Siri, called Viv, is being discussed.

The intelligence creates its own programs, within a few tenths of a second, depending on what the users intent was: "generates program, software that is writing itself, good for scaling" Dynamic programming, not hard coded.

Seems like a program that can write it's own programs, that's big.

Here's a good video with demo and Q&A:



Viv can be customized by developers to handle new domains.  This is quite a nice development in the evolution of smart machines.

And here's a good article:  http://techcrunch.com/2016/05/09/siri-creator-shows-off-first-public-demo-of-viv-the-intelligent-interface-for-everything/

Thanks for watching~!

Transformation of Data Processing Over Time

So we have these databases.  Some here.  One over there.  Oh, and these Excel files maintained by accounting.  And the Sales team has these files to track targets.  And our financial database over here.

We typically have separate reports for each system.  Or we consolidate into a Data Warehouse for a single source of the truth.  Takes a while, lots of business rules, sometimes the data isn't 100% accurate.  Then there are always changes.  Adding new data sources, changing business rules, cubes don't run every night.  Reporting is tough work.


Then we look at the reports.  What do all these numbers mean.  These reports are so static, I'd like the ability to drill down, see what the underlying data looks like.  Well, we have tools for that now, called Data Visualization tools.  Like Power BI, Tableau and Qlikview.  They're really taking off right now.  Because they are interactive, very intuitive and easy to get started.  And they scale.

Then we talk of deep learning.  Having really smart people build models based on data sets.  By creating samples of data, denormalized, and then applying statistical algorithms to find patterns and meaning and insight.  And that gets into the realm of Data Science.

So we stared with static reports, then a single source of the truth via Data Warehousing, then self exploration using Data Visualization tools, and then Data Science.


But there are still more levels to go.  The main one is called "Unsupervised Learning".  This is one of the most difficult process to perform.  In that, you give the machine a set of data, and let it find the meanings and patterns and insights.  Sure takes a lot of the grunt work out of the equation.  But that's the future in my opinion.  Let the machine teach itself.  Let it master one domain.  Then network with other machines that are experts in other domains.  In real time.  And let it continually learn over time as things change.  Find the anomalies, the exceptions, as well as the patterns, groupings, fluctuations, standard deviations.

Sure we still need static reports to send to the state every quarter.  And call centers need to know how many calls were received, answers, dropped yesterday and who took the calls and what the average call time was.  But the future of data is in unsupervised learning.

It works by creating multiple layers.  Each layer contains specific algorithms, which are connected to other layers.  They neurons of each layer weigh the data and trigger downstream activation of other neurons to create a grid of connected synapses.  And the more layers, the more processing and granularity.  And running these processes in parallel helps to speed up the jobs.


And the next level after that, is to run these processes on Quantum Computers.  Because they run using different foundation.  They don't use binary model of zeros and ones.  They use Qubits, which can have more than two states at any given time.  This methodology can increase the processing power tremendously.  That's the direction of things at the moment.

In addition to Internet of Connected Things, Robotics and accumulation of massive Big Data.  Someone or something needs to process all this data, synthesize into information that we can understand and take action in real time.  There's just too much heavy lifting for traditional data professionals.  We need to hire the computers to do what they do best, which is to crunch massive amounts of data.  I think we are on our way to that vision.

Get Sh#t Done!