Accelerating the
Industrial Internet of Things

Internet of Things (IOT), Cloud, Big Data and Machine Learning

Published on 12/07/2016 | Use Cases

639 1

Chirag Patel, Bharath Kumar

Business Development - IoT. Hortonworks



These are our thoughts on the current status of IOTs, where it is headed and also a little history. Would love to engage the community in validating our thoughts and learning more about the Future of IOT

Chirag Patel and Bharath Kumar

Connected World and IOTs

It is estimated that by year 2020, approximately 44 zettabytes of data will be generated and over 90% of that will come from over 30 billion connected devices. Companies of varied sizes will look to leverage information from any device possible to ensure they are, either cutting cost, increasing revenue or improving customer experience. Companies will have figured out innovative ways to keep pulse on their customer in terms of movement, behavior, usage, preferences and more areas than currently known.

This brings us to a million-dollar question – how will IoT add value to a particular industry? What could translate into business value for one industry vertical may differ vastly for another. A few examples:

Can an airline operator reduce delays and improve customer satisfaction by capturing and streaming device data from aircraft to ground operations?

Can a Bank transform a traditional product centric business model to a customer centric model by examining data from IoT devices in ATM machines and elevate their service offerings to attract millennials?

Can a CPG company avoid losses on perishable goods by monitoring sensors during transportation or improve inventory planning by monitoring usage through devices in-store?

Can the Aerospace industry build a connected enterprise by optimizing system design to develop better integrated products and systems with higher quality and lower cost through data collected from embedded sensors?

Can an energy and utility company capture future market share by improving their distribution and transmission network performance through an integrated grid-management system assimilating device data from operational control systems, for outage management, distribution management, geographic information, and asset management?

Can an automobile manufacturer build a tighter supply chain with accurate anticipation of demand for parts based on sensor data reporting vehicle usage patters?

Can a healthcare institution achieve efficient allocation of resources while decreasing potential errors by leveraging devices to offer care remotely and improve quality of care and outcomes cost effectively?

Can a defense contractor optimize asset availability by leveraging the full value of in-service data collected from equipment in field?

These and many more industry value drivers would reveal that an IoT explosion is on its way at the periphery of current day IT infrastructure. In order to better understand this phenomenon, let’s venture into the past and understand the history of IoT.

When did it all start? Where are we today? And where are we heading?

When we think of connected devices and IOTs, one that comes to our minds is Nest Thermostat. Even though IOTs and connected devices have existed for more than a decade in one form or another, this generation of IOT awareness was started by Nest and its acquisition by Google. Prior to Nest, Connected Cars have been there and GM OnStar, being one of the early ones, even though we may argue that they were not connected in real-time have been around for a while. Most of the items that we interact with today have some level of IOT functionality such as:

Connected Homes – Appliances, door locks, Weather control, security systems, monitoring, lights, switches, moisture sensors.

Connected cars – This is already a reality with hundreds of sensors in newer cars sending data to the Internet or to the user, giving information about the car’s usage patterns and physical condition.

Manufacturing – One of the areas where sensors have played a major role in the manufacturing processes a few decades back, in status, automation, process control etc. Expected to evolve rapidly as more sensors become available that can connect to the internet

Police departments – Already use facial recognition, license plate recognition using visual sensing technologies.

Security and monitoring – Cameras scanning buildings for abnormalities

Farming – soil conditions, temperature monitoring

Hospitals – RFIDs have been in use in hospitals for a while but sensors connected to a patient reporting data back to the doctors and hospitals are already in use in a limited way and expected to grow fast.

Retail – One of the first industries to use RFIDs.

Speaking of RFIDs, they are one of the earliest technologies that enabled adoption of IOT. More than a decade ago, Walmart was using it for incoming inventory. Proctor and Gamble was using more than a billion RFIDs in 2002. These were passive RFIDs, uniquely identifying its presence to a RFID reader and was used by P&G and Walmart for inventory management. As the technology evolved, there were active RFIDs, connecting to a network and sending data at predetermined intervals.

With a rich history, the sensing technology is now ripe to become a primary source of measurement for IOTs like signal strengths, moisture, heat, chemical, weight, proximity and then on to sensing gestures in 2D and 3D, like Kinect, image and patter detection. Possibilities are endless and IOTs explosive growth is happening now, exponentially increasing in future. And we should be ready for it - both as the users of this technology as well as creators of architecture and business applications around it.

IOT Architectures and Challenges

Today, IOTs are a mix of standards, technologies and protocols. Many institutions are working on standardizing the architecture around a single set of technologies much like IP became the standard in Network technologies. But currently, RFIDs might work in different radio bands (though it was one of the earliest to standardize on it), some light switches work on Bluetooth Low Energy, a few thermostats connect using ZigBee etc. and these are all built into the hardware making it really difficult to be changed. More over the data is not exchanged in any standard format.

One way of making the IOTs interact with each other is to make all IOTs capable of connecting to cloud – their own cloud and then to the Cloud on the internet. With this, even though at the sensor level, they may be using different standards, they can exchange information between them and with the apps. And a set of sensors can have their own private cloud and send the data to the public cloud only as needed. This reduces loss of data. This is a simplistic way of creating a standard architecture and a lot of work will have to go into this. But when a public cloud on the internet is mentioned, first question is always about security, both of the data and IOTs themselves. Security is a major area of challenge and at this time, it is still not as secure as some of the other technologies. Sure, nothing is 100% secure but recent news about some connected medical devices being hacked or connected cars being hacked has made is only a bigger cause for concern.

We will do a deep dive in to IOT architectures and its challenges in the next installment of this blog

IOT and Cloud

Cloud technology is a natural fit for IOT. While major parts of IOTs, especially in Hospitals and Manufacturing could stay within its own private cloud, majority of IOT devices can be leveraged only when the data is stored at some place. And with the number of IOTs expected to grow into eleven figures in the next few years and expected to be located at widely distributed geographical areas many of which are even inaccessible, the current designs provide a way of connecting and uploading the data to the internet, and Cloud becomes a natural fit for such a distributed model. Many organizations would still expect that the data not go public and in such cases, private clouds becomes an answer.

All the usual benefits of using Cloud are applicable to IOTs too.

- Central storage, with redundancy built to prevent loss of data

- Ease of connectivity

- Unparalleled scalability

- Variety of tools to make use of data

- Distribute across continents

- Works as a standard architectural layer that will connect potentially multiple underlying technologies

IOT and Big Data

Where does Big Data fit in? In a way, the increased use of IOT in the recent years have followed a similar trajectory compared to evolution of big data technologies, because IOTs generate huge amount of data. In fact, device generated data is expected to outpace human generated data, if not already happening. There are different types of data that come in from IOT devices, some of them are below:

Passive Data –This is about the presence of an item and at a location. The life of the data in many cases is not beyond the “current value”. It is often collected on demand and by a RFID reader.

Active Data –There are some cases, like geo-coded data, location history may be very useful.

Sensor Data – These are readings from devices, like pressure, temperature, moisture etc. Apart from their utility for alerts and triggers based on defined thresholds, their stored will be valuable for Predictive Analytics models.

Streaming Data – While the purpose of storing and use of this data varies, the defining characteristic of this category is that image, audio and video data are voluminous and in many cases, continuously generated, thereby requiring high speed and high volume storage.

The above characteristics makes one thing clear. We need to store lots of data even with a small collection of IoT devices.

Take a case of 10000 IoT devices that generate data every 5 seconds, each message being 32 bytes in size. That is 5 TB a day. Just for 10000 devices. We can reduce the frequency of data collection or message size to reduce this but it is still going to be multiple TBs. Even though the usefulness of this collected data in the long term is questionable, ability to store 10s or 100s of Terabytes is a requisite for any viable IoT application.

IOT and Machine Learning

How do we utilize the IOT data to draw actionable insights and formulate viable business outcomes?

While there are many applications of data collected from IOT devices, one of the most useful applications is Predictive Analytics using Machine Learning techniques and algorithms. And these fall into two categories:

Streaming Real-time analytics – As the data flows in, apply the models against the data streaming in or Score the models and predict the behavior of one device or a collection of devices. A good example of it is in the Airline industry or more specifically devices that are part of aircraft engines. By evaluating the streaming data and its pattern, potential failure of specific components can be predicted and corrective action can be taken before the failure.

Analytics on landed data – BigData technologies and distributed computing play a major role here. To train a model on high volume of data, which in turn might help in increasing accuracy of models, a lot of computing power is required. This has been made easy with BigData technologies.

Major challenges (more to come in the upcoming blogs)

Device Economics – Adoption of IOT technologies is closely linked to the price of these sensors and devices. Often, these devices are embedded into the environment which is being monitored – for example, sensors measuring the stress levels of concrete are built into it with no way of even changing the batteries, with a limited life space – thus might become an expensive proposition.

Security – Security of the information coming out of the devices has been a major worry among the current adopters of IOT technologies and these are of particular concern in the healthcare industry. Moreover, hackers could take control of the devices if the devices are not secured. Imagine a self-driven car being taken control of by a hacker!

Connectivity – With the IOT devices expected to spread across vast geographic, areas out of reach of a Wi-Fi hub or wireless base stations, how and when the devices connect to an available network and transmit data, how much of data it can buffer, how the critical events are transmitted immediately are some of the issues being researched with solutions already in place to meet those challenges. 

Data – Transmission, storage and availability of data in-time and on demand are some of the primary expectations from a well-designed IOT system. These challenges have been mitigated to some extent by the availability of BigData technologies

We will come back with future blogs to dive deeper into each of these challenges. Please feel free to post questions or comments.

Feature New Record