by Marcus Crast, Senior Consultant, 10th Magnitude
In my last blog post, I talked about Azure Event Hubs and Stream Analytics and how they enable real-time workflow triggering and alerts for Internet of Things (IoT) applications. This time I want to delve into Azure Machine Learning, a cloud service that Microsoft officially launched to the public in February 2015. Azure Machine Learning can kick your IoT efforts up a notch by incorporating predictive analytics. What if you could automatically be alerted to and take action on a problem or a business opportunity before it even happens? That’s the power of Machine Learning for IoT.
First let’s step back and talk about the larger concept of machine learning, specifically as it relates to IoT. There are two core skill areas required to implement machine learning:
1. The data science required to build predictive models. The rise of big data has also given rise to the data scientist, who can use huge volumes of available data to build models to predict things like customer churn, risk or equipment failure. Data scientists create models and the APIs that can be used to incorporate these predictive models into applications. With Azure Machine Learning, data scientists can publish in minutes what used to take days once they have a viable model.
2. The use of these predictive models in IoT applications. Solution architects and developers can then use the APIs built by data scientists as part of IoT applications that solve business problems or capture opportunities. With Azure Machine Learning, developers aren’t required to know much more than how to interface with an API in order to take advantage of the models created by data scientists.
Microsoft makes both of these areas easier for companies of all sizes to implement by giving them access to pre-built and pre-integrated machine learning resources in the cloud at a cost that’s much lower than for a similar competitive solution. When you combine these cost savings with the ability to store, sample, and process a huge amount of data in the cloud, Azure Machine Learning becomes quite appealing for organizations of all sizes.
One of Microsoft’s stated goals for Azure Machine Learning is to open up machine learning to non-data scientists. Microsoft accomplishes this by making pre-built predictive models available through its Azure Machine Learning Gallery and Azure Marketplace sites, and by allowing the community to do the same. You can visit these sites today and download an API for text analytics, product recommendations, fraud detection, risk analysis and many other scenarios. You can also download experiments, which are batch executions against an API that are used to make the API “smarter”—i.e., better at accurately predicting the desired outcome. There are both free and for-purchase API options available. If you’re interested in incorporating Machine Learning into an IoT application, there is likely an API that is already available or will be in the near future that will meet your needs with some customization and additional experimentation.
Going Beyond “Real Time”
Which brings me to the second component of machine learning—how to apply those APIs in an IoT scenario. To illustrate, I’ll expand on the pipe analogy I used in my last post to describe how Azure Event Hubs and Stream Analytics work together to enable IoT applications. In that analogy, the data flows like water through a pipe (i.e., Event Hubs) and the Stream Analytics query is a valve in the pipe. The query “valve” is constantly monitoring the data to see if it matches the conditions set by the query.
To add Machine Learning to this analogy, we can envision it as a branch off of the pipe that bumps the data against the machine learning API and then feeds the enhanced data back into the stream—for example, adding some type of score that predicts risk or churn. This additional information can be used to trigger a workflow in the IoT application. In this way, Machine Learning supplements the capabilities of Stream Analytics by enabling action to be taken based on predicted behavior in addition to current behavior.
For a real-world IoT scenario using Machine Learning, I’ll return to the HVAC example I used in my last post. In this example, a building management company for a residential high rise could use sensors to report suboptimal operating conditions in real time for their HVAC systems. This real-time reporting could help them save on repair costs by fixing issues as soon as possible and avoiding evening and weekend service calls. To go a step further, what if the company could use Machine Learning to predict which machines are more likely to fail before they hit a problem state?
The first step would be to create a viable API that returns a score correlating to likelihood of machine failure, perhaps based on machine vibrations per minute. The API would need to be trained with current and historical data on machine vibrations in order to accurately correlate vibration frequency with likelihood of failure and develop ranges for the vibration score. For example, the model might determine that 20 or more vibrations per minute correlates to the highest score, for example, while 15 to 19 per minute correlates to the second highest score, etc.
Once the API is in place, the score from Machine Learning would be added to the data that’s flowing through Event Hubs. Stream Analytics could then trigger a workflow or fire off an alert at a given score, notifying the building management company that an HVAC machine failure is imminent. By catching failures before they happen, the company could avoid major repairs or replacements for failed machines as well as improve tenant satisfaction. Machine Learning adds more context to the data and is more sensitive than Stream Analytics, which makes it a powerful tool as long as you invest in maintaining the predictive solution. In this scenario the building management company would still want to have Stream Analytics scan for and notify operators of real-time machine failures in addition to the predicted failures detected by Machine Learning.
I don’t want to understate the work that goes into building a successful IoT application that incorporates Machine Learning. The development of a viable API requires significant up-front effort to run experiments and train the model to make it smarter. The reason they’re called “experiments” is that they’re not guaranteed to always be successful; you start with a hypothesis to build the API, but you could disprove a lot of hypotheses before you discover one upon which you can build an accurate model. Machine Learning implementations also require substantial input from experts outside the IT organization to interpret outliers in the data—in the example above, this might be an HVAC engineer.
The good news is that Microsoft has made it easier than ever to get started on the road to a successful Machine Learning implementation. As more companies adopt Azure Machine Learning, creating their own APIs and running their own experiments, the resources available through the Gallery and Marketplace sites will only continue to grow. Expect to see some really cool Machine Learning case studies emerge over the next year, producing resources that other companies can use in their own IoT projects.