After getting carried away by the Big Data hype, I was more than skeptical about the Internet of Things.
Sensors in every piece of equipment. Sure, but how does that affect me? I only work with the data. I don't build washing machines or have access to sensors.
True, but from a data professional perspective, the data is where it at. And where is the data, it's going to be everywhere. From everything.
Little pulses of information. Sent continuously. From just about anything. How will the data get transmitted? Well, it sounds like radio frequencies connected to a hub that routes the messages to the central hub.
Could be millions of incoming messages per second. And there are tools to assist with the streaming of these messages. Actually called Streaming Analytics, in Azure, from Microsoft. An analytic platform to watch the data in flight. Look for anomalies, send alerts, etc.
And how will these messages find there way? Service Hub. Another Azure product. Similar to Service Broker, which is a messaging queue like application baked into SQL Server. Doesn't require source or destination to be in same network, guaranteed delivery, in the order in which it was sent. Event Hub is built on a concept called "partitions". This was designed to handle issues of blocking from Service Broker.
The data professional will set up the infrastructure on the Azure side and then, once the data starts flowing in, there's lots to identify, in the Streaming data. The incoming messages may not be verbose, it's like being eaten by little mosquito's, tiny, yet when combined with a swarm of others, potent.
Now I see the potential impact of the "Internet of Things". Monitoring millions of devices in real time 24/7. Processing each message and detecting anomalies, then basing some kind of action downstream.
So perhaps Big Data wasn't all hype after all. Because these little messages of streaming data from IoT will surely amplify our data, exponentially.
And there you have it~!
I signed up for the Hortonworks Certified Associate exam last Thursday. Figured if I sign up, I'd have to take the test. And if I tak...
This blog post is in no way an attempt to steal other people's work. It's basically an conglomeration of notes from research I did...
Saw a post today on Twitter, " Microsoft releases CNTK, its open source deep learning toolkit, on GitHub " This is big news. Be...