Looking into the Future of the Classic Data Historian

The speedy progress of the world’s industrial output has inspired ongoing advancements in operational technology. Digital transformation and Industry 4.0. are accelerating exponential demands for Industrial systems like DCS, SCADA, RTU’s, PLC’s, sensors, edge devices, data historian, industrial robots, 3D printers. All these systems are now generating unprecedented and escalating production volumes, velocities, and efficiencies. The data captured by these systems need to be properly managed, cleaned, processed, stored, routed, secured, and leveraged, and so on...

In the past, data historians, which are time-series databases, were located on the premise, next to the industrial system. They captured and stored all the sensor data. However, to take advantage of artificial intelligence and big data analytics applications, which are mostly available in the cloud environments, the data now needs to be moved, stored, and searchable in a cloud-based database.

The whole IIoT evolution is not new to the market. The concept of IIoT has been around in the industry for many years, however, the demand for the convergence of IT and OT is getting louder and louder. The time-series data historian plays a major role here in the context of IIoT. Industrial time series data will give way to complex adaptive systems and multi-processing. The future belongs to nanotech, cloud computing, wireless everything, artificial intelligence (AI)-based machine learning (ML), Big Data, and complex adaptive systems.

Time-series data is the data that changes with time (e.g., digital sensor readings). A time-series database keeps data values and timestamps which were collected over time with the unique ability to consistently store (ingest) large amount of data that is coming in with time. Time-Series data and related technologies are the fasted growing segment in the market. As a result, we have seen lots of investments and acquisitions recently e.g. AVEVA Acquire OSIsoft in $5 Billion Deal on August 25, 2020. Industrial time-series data has gravity and researchers anticipate it to grow with a healthy growth rate of more than 6.90% over the forecast period 2020-2025. Leading public and private cloud platforms, software startups, data lake vendors, control systems, SCADA companies, top tier visionary investors and venture capital firms are all rushing to be the vendor/partner/investor of record for this time-series data storage business of the world. In the coming year's competition will fuels more innovation and will grows Time-Series data historian market as a whole.

No alt text provided for this image


Future of Data Historian requires a different way of thinking

The Data historian has evolved from being just a place for storing data to becoming a data infrastructure. This means data collection or storage or visualization by themselves alone or even together doesn’t make a complete industrial data management system valuable. Industry4.0 evolution needs more with a complete infrastructure solution with the capability of integration, archiving, asset modeling, notifications, visualizing, analysis, and many more analyzing features. Data historian, MES, and ERP all might become part of DataLake (everything stored) which we can call the unified namespace, however, DataLake will still not be able to supply data at the right time and with the right context for time-stamped process data with proper data integrity.

The future of data historian is much more than a traditional operational data historian. The data historian capability is just a sub-function. A better name would be an “Industrial Data Management System” or something along those lines. Operational data historians are expensive, challenging to work with, and typically behind the times with limited analysis and visualization capability. Not all data historians are horizontally scalable and during a large amount of archive, data retrieval needs to face a performance penalty. It is also very difficult to contextualize sensors data to other metadata for data historians which is very lengthy and costly for customers to work with. Operational data historian provides the benefits of interfacing that sit with data collection points together with buffer capability (store and forward) with industrial system compatibility such as DCS, SCADA, OPC, etc.

Key technologies for the future of the Industrial Data Management System:

  • Digital Twin: Merging of the virtual and physical worlds - Digital twin capability to model either physical or logical objects with the view of assets or the data associated with them. Virtually replication for industries’ physical objects and processes with asset hierarchy.
  • Blockchain: IDMS (Industrial Data Management System) augmentation in conjunction with Blockchain technology to implement decentralization and robustness in the network to improve the asset lifecycle.
  • Blockchain Security: Protection, verification, and nonrepudiation of critical data for historian streams and connected applications.
  • Embedded cyber security: Ingress, egress, and historian integrations systems need to be extremely reliable with cybersecurity.
  • OPC UA adoption: More collaboration with the OPC Foundation for OPC UA to develop robust, redundant, and scalable UA connectors/interfaces with.
  • IoT Edge device integration: Interfacing with edge infrastructure and enabling intelligence at the edge level through ready to use interfaces for edge devices and intelligent gateways.
  • Deal volume: high-speed data collection and retrieval (Millions of inputs per second).
  • Efficient data storage: Industrial Data Management System requires efficient and easy techniques like a compression algorithm for time-series data archiving.
  • No SQL: Simple archival storage in blocks of time for easy access to data.
  • Sensor Level Security: Data security role down to individual data point granularity level together with Microsoft Windows Integrated Security and fine-grained access control.
  • Vertical and horizontal integration: Ready-to-use (Plug & Play) interfaces/connectors for a wide range of different input sources with 3rd party applications and business systems. Prebuild ingress solution called interface which can be deployed near to source system with buffering (Data Storage) capability and filtering noise capability (Exception Reporting).
  • Support to open-source connectivity: A wide range of data access technology support needed like OPC DA, HDA & UA, APIs, SQL, programmatic access via Software Development Kit (SDK), support to Microsoft's Component Object Model (COM), Data connectors, Web API interfacing, etc.
  • Speedy ingress and egress: Fast access for real-time analytics, machine learning, and AI as a One-Stop shop.
  • High availability and redundancy: Features that can mirrors the data across multiple nodes in the industry with high availability at each level and techniques.
  • Visualization and alert: High-quality querying, visualizing, and notifying and alerting capability.
  • Data enrichment and cleansing capability: Inbuilt data aggregation, auto data cleansing, interpolation, and data enrichment capability.
  • Data contextualization: The system should be smart enough to easily contextualize the time series data with metadata with the ability to combine different data types and different data sources.

And last but not least to have a solution for an extremely important challenge to solve human talent availability issues to help customers. There are multiple players in this race but still, no one has a complete solution for the future of the Industrial Data Management System.

Major market players in 2020 for time series data management market:

  •  AVEVA Group (OSIsoft PI, WonderWare InSQL, eDNA, and Citec historians)
  • Honeywell (PHD)
  • Aspen Technology (InfoPlus 21)
  • GE (Proficy & Predix)
  • Microsoft (Time Series Insight)
  • Amazon (AWS TimeStream)
  • Google (BigTable)
  • ABB Historian
  • InfluxDB
  • TimescaleDB
  • OpenTSDB
  • Graphite
  • Cassandra
  • MongoDB
  • Elasticsearch
  • Riak
  • kdb+ (Kx)

References –

Latest Posts