Next Gen IIoT Platform and Distributed Computing

Over past few years with the recent enhancements to Internet of things (IoT) and sensors deployments, the generation of big data in Industrial Internet of things (IIoT) is increased. However, processing big data is challenging step due to limited computational, networking and storage resources at IoT device-end. Thereby accessing and processing of big data become a challenging issue due to the limited storage space, computational time, networking, and IoT devices end.

The power of big data in IoT are well thought-out to be the key concepts when describing new information architecture projects. The techniques, tools, and methods that help to provide better solutions for IoT and big data can have an important role to play in the architecture of business.

In this blog, we will discuss how IoT and big data analytics (BDA) go hand in hand and how various enterprises are obtaining benefits by using the industrial internet of things big data analytics together. And we will also learn how the recent BDA technologies and techniques that can lead to the development of intelligent IIoT systems.

Also, to understand the role of big data in IIoT, we will study an approach called Analytic Network Process (ANP). This technique shows that the proposed research works well for evaluating the role of big data in IIoT.

Big data analytics (BDA) is expected to provide operational- and customer-level intelligence in IIoT systems

What is Industrial Internet of Things (IIoT)

Industrial Internet of Things (IIoT) (also known as Industry 4.0) currently attributed as the fourth industrial revolution. The technology ecosystem underpinning IIoT is mainly the integration of cyberphysical systems (CPS), Internet of Things (IoT), cloud computing, automation (e.g. intelligent robots in product assembly lines), Internet of services, wireless technologies, augmented reality and concentric computing, amongst others. Advances in such related areas as IoT, big data analytics (BDA), cloud computing and CPS have fueled the formation of IIoT activities to deliver unprecedented flexibility, precision and efficiency to manufacturing processes. Given this cross-platform integration, IIoT systems need to ensure interoperability, virtualization, de-centralization, real-time capability, service orientation, modularity and security across all verticals.

The designs of IIoT systems involve seven principles [11], as depicted in Fig. 1


Figure 1: Seven design implementation principles for Industry 4.0 systems

Power of Big Data in IIoT

When enterprises are grabbing hold of the data for analysis purpose, IoT is acting as a major source for that data, and this is the point where the role of big data in IoT comes into the picture. Big data analytics is emerging as a key to analyzing IoT generated data from “connected devices” which helps to take the initiative to improve decision making.

With the power of big data in IoT is to process a large amount of data on a real-time basis and storing them using different storage technologies.

A typical IoT big data processing follows four sequential steps –

  1. A large amount of unstructured data is generated by IoT devices which are collected in the big data system. This IoT generated big data largely depends on their 3V factors that are volume, velocity, and variety
  2. In the big data system, which is basically a shared distributed database, the huge amount of data is stored in big data files
  3. Analyzing the stored IoT big data using analytic tools like Hadoop MapReduce or Spark
  4. Generating the reports of analyzed data

En route, IoT research and business models are accelerated by using big data technology. The following graphic illustrates the connection between IoT and big data.



Since in IoT data is source for both unstructured and semi-structured data and these are collected via the internet, hence, big data for the internet of things need lightning-fast analysis with large queries to gain rapid insights from data to make quick decisions. Thereby the need for big data in IoT is compelling.

Role of Big data Analytics in IIoT

Big data Analytics know as BDA refers to the process of collecting, managing, processing, analyzing and visualizing continuously evolving data in terms of volume, velocity, value, variety and veracity. Big data in IIoT systems arise due to unbounded internal and external activities relevant to customers, business operations, production and machines. BDA processes in IIoT systems manage the collected data using multiple transient and persistent storage systems that provide on-board, in-memory, in-network and large-scale distributed storage facilities across IIoT systems.

The granularity of data processing facilities for BDA processes in IIoT systems vary from resource-constrained IoT devices to resourceful large-scale distributed cloud computing systems. Similarly, analytic operations differ in terms of descriptive, prescriptive, predictive and preventive procedures. In addition, BDA processes must ensure real-time knowledge visualization across multiple IIoT systems. A proper integration of BDA processes into IIoT systems is perceived to maximize value creation to evolve business models for profit maximization.

Key concepts - BDA in IIoT systems: In this section we present a detailed discussion on different aspects of big data adoption in IIoT systems.

  1. Rise of Big Data in IIoT Systems
  2. Concentric Computing Model for BDA in IIoT
  3. Big Data Analytics for Delivering Intelligence in IIoT Systems a. Data Engineering b. Data Preparation c. Data Analytics d. Managing and Automating the Data Pipeline

Rise of Big Data in IIoT Systems

IoT devices in IIoT systems refer to devices that can remotely sense and actuate in industrial environments. IoT devices either work as stand-alone devices that roam around industrial environments or are attached with existing cyber-physical systems CPS to perform certain predefined actions. The on-board sensing facilities in IoT devices lead the generation of big data, which may become useful for value creation in enterprises. The integration of CPS and IoT devices results in massive back-end cloud service utilization for the execution of BDA processes. To achieve massively customized production, the number of cloud services can be grown immensely. Thus, BDA can facilitate in-service selection, service orchestration and real-time service provisioning.

Concentric Computing Model for BDA in IIoT

With recent evolution in sensing and computing technologies has opened new avenues for big data processing. Concentric computing refers to the large-scale highly distributed computing systems based on a wide range of devices and computing facilities in different form factors. Concentric computing offers big data processing at sensors levels, endpoints in IIoT systems, edge servers, and centralized and decentralized cloud computing systems, as illustrated in Fig. 2. Despite their small size and limited computational power, sensors and IoT devices have the ability to filter and reduce raw data streams by using on-board smart data reduction strategies. However, edge servers at gateways and centralized computing clusters have the ability to distribute the computing load for BDA applications. Multistage execution, automating, and management of BDA processes (i.e., data engineering, data preparation and data analytics) are necessary in concentric computing environments (such as sensors and wearable devices as endpoints, IoT devices, edge servers, and cloud computing servers).


Figure 2: Industrial IoTs and Multilayer Computing Resources

Big Data Analytics for Delivering Intelligence in IIoT Systems

BDA processes are executed as a result of multistage highly interdependent application components (Fig. 3). These components are categorized as follows.

Data Engineering

Data engineers can build computing and storage infrastructure to ingest, clean, conform, shape and transform data. Primarily, IIoT systems produce and ingest big data from inbound enterprise operations and outbound customer activities. To establish quality control and relevance with IIoT applications, the raw data at the earliest stage need further processing. Therefore, data wrangling and cleaning methodologies help select relevant datasets in case of historical data or data streams in case of streaming data. Data conformity procedures are applied to ensure relevant, correctly collected big data. Data shaping and transformation methodologies help improve data quality by reducing the number of attributes and converting data formats for uniform data processing.

Data Preparation

Production of Big data in raw form with large volume and enormous speed, and data scientists spend 70% − 80% of their time in data preparation activities. Big data are refined using statistical methods to handle unstructured, unbalanced and non-standardized data points efficiently. In addition, data refinement helps summarize voluminous data to reduce overall complexity. As a result, the spatiotemporal attributes of big data in IIoT systems vary. Ultimately, data locality is necessary to reduce in-network traffic and latency in big-data applications. Location-aware highly virtualized data infrastructure can address these issues. However, data blending, which is the process of combining data from multiple sources, becomes complex. Accordingly, further involvement by data scientists is required to perform data cleaning and noise removal. Detection methods for outliers and anomalies are also needed to prepare big data for further analysis.


Figure 3: Multistage execution, automating, and management of BDA processes (i.e., data engineering, data preparation and data analytics) in concentric computing environments (such as sensors and wearable devices as endpoints, IoT devices, edge servers, and cloud computing servers).

Data Analytics

The analytic processes in IIoT systems are executed in multiple phases. Data scientists generate learning models from high-quality well-prepared data. After the model is developed, model scoring operations are performed by giving sample datasets and finding and ranking the attributes in datasets/data streams. The correctly tuned models are deployed in production environments to find the knowledge patterns from future data.

Managing and Automating the Data Pipeline

Although existing literature still lacks the concept of automated data pipelines in IIoT systems, BDA processes are executed as a sequence of operations during data engineering, preparation and analytics. Therefore, a holistic approach is needed to execute and administer BDA processes across all layers of concentric computing systems. Life cycle management is needed for full process execution from raw data acquisition to knowledge visualization and actuation. Data provenance, that is, designating ownership of data to different stack holders, also needs serious attention to ensure system-wide control on data. The continuous evolution in data streams results in knowledge shift that enforces data pipelines to adaptively reconfigure analytic processes. The data pipelines need to be continuously monitored for change detection, and the entire BDA process needs to be re-executed to produce high-quality results. In security perspective, the cross-platform execution of BDA processes demands secure operations at IoT device, CPS and big data levels.


Figure 4 presents the taxonomy that is devised on the basis of data sources, analytics tools, analytics techniques, requirements, industrial analytics applications and analytics types.


Figure 4: Taxonomy of BDA in IIoT

IIoT and Big Data together can be a beneficial for companies

IoT big data analytics can be useful for a variety of IoT data to – • Examine • Reveal trends • Find unseen patterns • Find hidden correlations • Reveal new information

Hence, companies can benefit from analyzing large amounts of IoT big data and managing them to identify how they affect businesses. As a result, it assists business and other organizations to achieve an improved understanding of data, and thus, making efficient and well-informed decisions. Every segment of businesses and industries can achieve some benefits.

Advantages in manufacturing companies

If manufacturing companies install IoT sensors within its equipment, they can collect significant operational data on the machines. This helps them to have an in-depth look at how the business is performing and enable them to find out which equipment need repairing before much problems arise. This prevents them from more significant expenses by skipping the downtime or replacement of the equipment. Hence, investment in IoT and big data causes saving businesses money.

More benefits in Industrial internet of things (IIoT)

IIoT is related with various connected devices which help following tasks to control the behavior of the industrial devices – • Monitoring • Collecting • Exchanging • Analyzing • Instantly acting on information

Precision Manufacturing

BDA processes can help enrich precision manufacturing systems. The classification and categorisation of customers needs and behaviour-related data can lead towards innovative product designs. Enterprises will be able to offer the right products and services to the right customers. Precision manufacturing will considerably help in equal value creation for customers and enterprises. Early examples of precision manufacturing systems are available in the healthcare industry. However, these systems should be integrated into IIoT systems.

End-to-end Industrial Analytics

Big data in IIoT systems evolves from multiple inbound and outbound data sources, such as customer data and operational data from finance, marketing, human resources, IoT devices, CPS and manufacturing systems. Nevertheless, existing systems manage all these data sources separately to execute BDA processes. An opportunity exists to develop an end-to-end industrial analytics pipeline that can handle big data from various data sources in parallel and find highly correlated knowledge patterns that emerge across entire IIoT systems

Edge-Computing will be in high demand

Working on real-time data is a high priority today and a necessity as well. As IoT and Big data both enable on-demand and real-time action, the importance of deployment of these technologies is high. In this view, the popularity of edge computing is also becoming very high. As the IoT and big data are closely linked, there are many examples out there of organizational benefits to put them to good use.


The convergence of IoT and big data can provide new opportunities and applications in all the sectors. The vision of Industry 4.0 to connect manufacturing systems with distributors and consumers can only be achieved by adopting IIoT and BDA processes as core components for value creation. We conclude that the adoption of BDA in IIoT systems is still in its early stage. Research on complementary components of IIoT systems, such as IoT devices, augmented reality and CPS, is also in its infancy. Current BDA systems provide generic frameworks for data engineering, preparation and analysis. However, considerable effort is required to alter existing BDA processes to meet the demands of IIoT systems. . Future research should be conducted to devise new standards for interoperability among cross-Industry 4.0 BDA platforms and to provide capability for end-to-end reliable application processing by considering the anatomy of concentric computing systems


  1. The predictive power of big data analytics in the IIoT Era (
  2. Internet of Things and Big Data - Better Together - Whizlabs Blog
  3. Evaluating the Role of Big Data in IIOT-Industrial Internet of Things for Executing Ranks Using the Analytic Network Process Approach (
  4. The Role of Big Data Analytics in Industrial Internet of Things

Did you find this article valuable?

Support Harsh Vardhan by becoming a sponsor. Any amount is appreciated!