Real-time business intelligence

Last updated December 12, 2024

Real-time business intelligence (RTBI) is a concept describing the process of delivering business intelligence (BI) or information about business operations as they occur. Real time means near to zero latency and access to information whenever it is required.^[1]

The speed of today's processing systems has allowed typical data warehousing to work in real-time. The result is real-time business intelligence. Business transactions as they occur are fed to a real-time BI system that maintains the current state of the enterprise. The RTBI system not only supports the classic strategic functions of data warehousing for deriving information and knowledge from past enterprise activity, but it also provides real-time tactical support to drive enterprise actions that react immediately to events as they occur. As such, it replaces both the classic data warehouse and the enterprise application integration (EAI) functions. Such event-driven processing is a basic tenet of real-time business intelligence.

In this context, "real-time" means a range from milliseconds to a few seconds (5s) after the business event has occurred. While traditional BI presents historical data for manual analysis, RTBI compares current business events with historical patterns to detect problems or opportunities automatically. This automated analysis capability enables corrective actions to be initiated and/or business rules to be adjusted to optimize business processes.

RTBI is an approach in which up-to-a-minute data is analyzed, either directly from operational sources or feeding business transactions into a real time data warehouse and business intelligence system.

Latency

All real-time business intelligence systems have some latency, but the goal is to minimize the time from the business event happening to a corrective action or notification being initiated. Analyst Richard Hackathorn describes three types of latency:^[2]

Data latency; the time taken to collect and store the data
Analysis latency; the time taken to analyze the data and turn it into actionable information
Action latency; the time taken to react to the information and take action

Real-time business intelligence technologies are designed to reduce all three latencies to as close to zero as possible, whereas traditional business intelligence only seeks to reduce data latency and does not address analysis latency or action latency since both are governed by manual processes.

Some commentators have introduced the concept of right time business intelligence which proposes that information should be delivered just before it is required, and not necessarily in real-time.

Architectures

Event-based

Real-time Business Intelligence systems are event driven, and may use Complex Event Processing, Event Stream Processing and Mashup (web application hybrid) techniques to enable events to be analysed without being first transformed and stored in a database. These in-memory database techniques have the advantage that high rates of events can be monitored, and since data does not have to be written into databases data latency can be reduced to milliseconds.

Data warehouse

An alternative approach to event driven architectures is to increase the refresh cycle of an existing data warehouse to update the data more frequently. These real-time data warehouse systems can achieve near real-time update of data, where the data latency typically is in the range from minutes to hours. The analysis of the data is still usually manual, so the total latency is significantly different from event driven architectural approaches.

Server-less technology

The latest alternative innovation to "real-time" event driven and/or "real-time" data warehouse architectures is MSSO Technology (Multiple Source Simple Output) which removes the need for the data warehouse and intermediary servers altogether since it is able to access live data directly from the source (even from multiple, disparate sources). Because live data is accessed directly by server-less means, it provides the potential for zero-latency, real-time data in the truest sense.

Process-aware

This is sometimes considered a subset of operational intelligence and is also identified with Business Activity Monitoring. It allows entire processes (transactions, steps) to be monitored, metrics (latency, completion/failed ratios, etc.) to be viewed, compared with warehoused historic data, and trended in real-time. Advanced implementations allow threshold detection, alerting and providing feedback to the process execution systems themselves, thereby 'closing the loop'.

Technologies that support real-time analytics

Technologies that can be supported to enable real-time business intelligence are data visualization, data federation, enterprise information integration, enterprise application integration and service oriented architecture. Complex event processing tools can be used to analyze data streams in real time and either trigger automated actions or alert workers to patterns and trends.

Data warehouse appliance: Data warehouse appliance is a combination of hardware and software product which was designed exclusively for analytical processing. In data warehouse implementation, tasks that involve tuning, adding or editing structure around the data, data migration from other databases, reconciliation of data are done by DBA. Another task for DBA was to make the database to perform well for large sets of users. Whereas with data warehouse appliances, it is the vendor responsibility of the physical design and tuning the software as per hardware requirements. Data warehouse appliance package comes with its own operating system, storage, DBMS, software, and required hardware. If required data warehouse appliances can be easily integrated with other tools.

Mobile technology: There are very limited vendors for providing Mobile business intelligence; MBI is integrated with existing BI architecture. MBI is a package that uses existing BI applications so people can use on their mobile phone and make informed decision in real time.

Application areas

Fraud detection
Systems monitoring
Application performance monitoring
Customer Relationship Management
Demand sensing
Dynamic pricing and yield management
Data validation
Operational intelligence and risk management
Payments & cash monitoring
Data security monitoring
Supply chain optimization
RFID/sensor network data analysis
Workstreaming
Call center optimization
Enterprise Mashups and Mashup Dashboards
Transportation

Related Research Articles

In computing, a data warehouse, also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis and is a core component of business intelligence. Data warehouses are central repositories of data integrated from disparate sources. They store current and historical data organized so as to make it easy to create reports, query and get insights from the data. Unlike databases, they are intended to be used by analysts and managers to help make organizational decisions.

Business intelligence (BI) consists of strategies, methodologies, and technologies used by enterprises for data analysis and management of business information. Common functions of BI technologies include reporting, online analytical processing, analytics, dashboard development, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics.

A decision support system (DSS) is an information system that supports business or organizational decision-making activities. DSSs serve the management, operations and planning levels of an organization and help people make decisions about problems that may be rapidly changing and not easily specified in advance—i.e., unstructured and semi-structured decision problems. Decision support systems can be either fully computerized or human-powered, or a combination of both.

Event processing is a method of tracking and analyzing (processing) streams of information (data) about things that happen (events), and deriving a conclusion from them. Complex event processing (CEP) consists of a set of concepts and techniques developed in the early 1990s for processing real-time events and extracting information from event streams as they arrive. The goal of complex event processing is to identify meaningful events in real-time situations and respond to them as quickly as possible.

Online transaction processing (OLTP) is a type of database system used in transaction-oriented applications, such as many operational systems. "Online" refers to the fact that such systems are expected to respond to user requests and process them in real-time. The term is contrasted with online analytical processing (OLAP) which instead focuses on data analysis.

A mashup, in web development, is a web page or web application that uses content from more than one source to create a single new service displayed in a single graphical interface. For example, a user could combine the addresses and photographs of their library branches with a Google map to create a map mashup. The term implies easy, fast integration, frequently using open application programming interfaces and data sources to produce enriched results that were not necessarily the original reason for producing the raw source data. The term mashup originally comes from creating something by combining elements from two or more sources.

Business intelligence software is a type of application software designed to retrieve, analyze, transform and report data for business intelligence (BI). The applications generally read data that has been previously stored, often - though not necessarily - in a data warehouse or data mart.

Operational intelligence (OI) is a category of real-time dynamic, business analytics that delivers visibility and insight into data, streaming events and business operations. OI solutions run queries against streaming data feeds and event data to deliver analytic results as operational instructions. OI provides organizations the ability to make decisions and immediately act on these analytic insights, through manual or automated actions.

Business Intelligence 2.0 is a development of the existing business intelligence model that began in the mid-2000s, where data can be obtained from many sources. The process allows for querying real-time corporate data by employees but approaches the data with a web browser-based solution. This is in contrast to previous proprietary querying tools that characterized previous BI software.

Truviso is a continuous analytics, venture-backed, startup headquartered in Foster City, California developing and supporting its solution leveraging PostgreSQL, to deliver a proprietary analytics solutions for net-centric customers. Truviso was acquired by Cisco Systems, Inc. on May 4, 2012.

Event-driven SOA is a form of service-oriented architecture (SOA), combining the intelligence and proactiveness of event-driven architecture with the organizational capabilities found in service offerings. Before event-driven SOA, the typical SOA platform orchestrated services centrally, through pre-defined business processes, assuming that what should have already been triggered is defined in a business process. This older approach does not account for events that occur across, or outside of, specific business processes. Thus complex events, in which a pattern of activities—both non-scheduled and scheduled—should trigger a set of services is not accounted for in traditional SOA 1.0 architecture.

Web data services refers to service-oriented architecture (SOA) applied to data sourced from the World Wide Web and the Internet as a whole. Web data services enable maximal mashup, reuse, and sharing of structured data, semi-structured information, and unstructured information.

Collaborative decision-making (CDM) software is a software application or module that helps to coordinate and disseminate data and reach consensus among work groups.

The term is used for two different things:

In computer science, in-memory processing (PIM) is a computer architecture in which data operations are available directly on the data memory, rather than having to be transferred to CPU registers first. This may improve the power usage and performance of moving data between the processor and the main memory.
In software engineering, in-memory processing is a software architecture where a database is kept entirely in random-access memory (RAM) or flash memory so that usual accesses, in particular read or query operations, do not require access to disk storage. This may allow faster data operations such as "joins", and faster reporting and decision-making in business.

The Digital Firm is a kind of organization that has enabled core business relationships through digital networks In these digital networks are supported by enterprise class technology platforms that have been leveraged within an organization to support critical business functions and services. Some examples of these technology platforms are Customer Relationship Management (CRM), Supply Chain Management (SCM), Enterprise Resource Planning (ERP), Knowledge Management System (KMS), Enterprise Content Management (ECM), and Warehouse Management System (WMS) among others. The purpose of these technology platforms is to digitally enable seamless integration and information exchange within the organization to employees and outside the organization to customers, suppliers, and other business partners.

Esper is an open-source Java-based software product for Complex event processing (CEP) and Event stream processing (ESP), that analyzes series of events for deriving conclusions from them.

Threat Intelligence Platform (TIP) is an emerging technology discipline that helps organizations aggregate, correlate, and analyze threat data from multiple sources in real time to support defensive actions. TIPs have evolved to address the growing amount of data generated by a variety of internal and external resources (such as system logs and threat intelligence feeds) and help security teams identify the threats that are relevant to their organization. By importing threat data from multiple sources and formats, correlating that data, and then exporting it into an organization’s existing security systems or ticketing systems, a TIP automates proactive threat management and mitigation. A true TIP differs from typical enterprise security products in that it is a system that can be programmed by outside developers, in particular, users of the platform. TIPs can also use APIs to gather data to generate configuration analysis, Whois information, reverse IP lookup, website content analysis, name servers, and SSL certificates.

Apama is a complex event processing (CEP) and event stream processing (ESP) engine, developed by Software AG. Apama serves as a platform for performing streaming analytics over a range of high volume/low latency inputs and applications, such as IoT devices, financial exchanges, fraud detection, social media and similar. Users can define data patterns to listen for and actions to take when these patterns are found, which are defined in the provided domain-specific language called the Event Processing Language (EPL). The core Apama engine is written in C++; the process can also optionally contain a JVM for interacting with user created Java code. Apama focuses on high throughput, low latency and memory efficient performance; used in both Intel benchmarks and smaller machines such as the Raspberry Pi, routers and other Edge/IoT devices. It is particularly noteworthy within the CEP space as being one of the earliest projects, a long term market leader, and innovator of many patents.

A secure access service edge (SASE) is technology used to deliver wide area network (WAN) and security controls as a cloud computing service directly to the source of connection rather than a data center. It uses cloud and edge computing technologies to reduce the latency that results from backhauling all WAN traffic over long distances to one or a few corporate data centers, due to the increased movement off-premises of dispersed users and their applications. This also helps organizations support dispersed users.

References

↑ Azvine, B.; Cui, Z.; Nauck, D.D.; Majeed, B. (2006). "Real Time Business Intelligence for the Adaptive Enterprise". The 8th IEEE International Conference on E-Commerce Technology and the 3rd IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services (CEC/EEE'06). p. 29. doi:10.1109/CEC-EEE.2006.73. ISBN 0-7695-2511-3. S2CID 15523928.
↑ Richard Hackathorn (February 2004). "The BI Watch Real-Time to Real-Value".

External links

Complex Event Processing & Real Time Intelligence - Introduction to Real-Time Intelligence

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] Azvine, B.; Cui, Z.; Nauck, D.D.; Majeed, B. (2006). "Real Time Business Intelligence for the Adaptive Enterprise". The 8th IEEE International Conference on E-Commerce Technology and the 3rd IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services (CEC/EEE'06). p. 29. doi:10.1109/CEC-EEE.2006.73. ISBN 0-7695-2511-3. S2CID 15523928.

[2] Richard Hackathorn (February 2004). "The BI Watch Real-Time to Real-Value".

[1]

[2]