Enterprise application integration

Last updated

Enterprise application integration (EAI) is the use of software and computer systems' architectural principles to integrate a set of enterprise computer applications.[ citation needed ]

Overview

Enterprise application integration is an integration framework composed of a collection of technologies and services which form a middleware or "middleware framework" to enable integration of systems and applications across an enterprise [1] .

Many types of business software such as supply chain management applications, ERP systems, CRM applications for managing customers, business intelligence applications, payroll, and human resources systems typically cannot communicate with one another in order to share data or business rules. For this reason, such applications are sometimes referred to as islands of automation or information silos. This lack of communication leads to inefficiencies, wherein identical data are stored in multiple locations, or straightforward processes are unable to be automated.[ citation needed ]

Enterprise application integration is the process of linking such applications within a single organization together in order to simplify and automate business processes to the greatest extent possible, while at the same time avoiding having to make sweeping changes to the existing applications or data structures. Applications can be linked either at the back-end via APIs or (seldomly) the front-end (GUI).[ citation needed ]

In the words of research firm Gartner: "[EAI is] the unrestricted sharing of data and business processes among any connected application or data sources in the enterprise." [2]

The various systems that need to be linked together may reside on different operating systems, use different database solutions or computer languages, or different date and time formats, or could be legacy systems that are no longer supported by the vendor who originally created them. In some cases, such systems are dubbed "stovepipe systems" because they consist of components that have been jammed together in a way that makes it very hard to modify them in any way.[ citation needed ]

Improving connectivity

If integration is applied without following a structured EAI approach, point-to-point connections grow across an organization. Dependencies are added on an impromptu basis, resulting in a complex structure that is difficult to maintain. This is commonly referred to as spaghetti, an allusion to the programming equivalent of spaghetti code. For example:[ citation needed ]

The number of connections needed to have fully meshed point-to-point connections, with n points, is given by ${\displaystyle {\tbinom {n}{2}}={\tfrac {n(n-1)}{2}}}$ (see binomial coefficient). Thus, for ten applications to be fully integrated point-to-point, ${\displaystyle {\tfrac {10\times 9}{2}}=45}$ point-to-point connections are needed.

However, the number of connections within organizations does not grow according to the square of the number points. In general, the number of connections to any point is independent of the number of other points in an organization (Thought experiment: if an additional point is added to your organization, are you aware of it? Does it increase the number of connections other unrelated points have?). There are a small number of "collection" points for which this does not apply, but these do not require EAI patterns to manage.

EAI can also increase coupling between systems and therefore increase management overhead and costs.[ citation needed ]

EAI is not just about sharing data between applications, but also focuses on sharing both business data and business process. A middleware analyst attending to EAI will often look at the system of systems.[ citation needed ]

Purposes

EAI can be used for different purposes:[ citation needed ]

• Data integration: Ensures that information in multiple systems is kept consistent. This is also known as enterprise information integration (EII).
• Vendor independence: Extracts business policies or rules from applications and implements them in the EAI system, so that even if one of the business applications is replaced with a different vendor's application, the business rules do not have to be re-implemented.
• Common facade: An EAI system can front-end a cluster of applications, providing a single consistent access interface to these applications and shielding users from having to learn to use different software packages.

Patterns

This section describes common design patterns for implementing EAI, including integration, access and lifetime patterns. These are abstract patterns and can be implemented in many different ways. There are many other patterns commonly used in the industry, ranging from high-level abstract design patterns to highly specific implementation patterns. [3]

Integration patterns

EAI systems implement two patterns: [4]

Mediation (intra-communication)
Here, the EAI system acts as the go-between or broker between multiple applications. Whenever an interesting event occurs in an application (for instance, new information is created or a new transaction completed) an integration module in the EAI system is notified. The module then propagates the changes to other relevant applications.
Federation (inter-communication)
In this case, the EAI system acts as the overarching facade across multiple applications. All event calls from the 'outside world' to any of the applications are front-ended by the EAI system. The EAI system is configured to expose only the relevant information and interfaces of the underlying applications to the outside world, and performs all interactions with the underlying applications on behalf of the requester.

Both patterns are often used concurrently. The same EAI system could be keeping multiple applications in sync (mediation), while servicing requests from external users against these applications (federation).[ citation needed ]

Access patterns

EAI supports both asynchronous (fire and forget) and synchronous access patterns, the former being typical in the mediation case and the latter in the federation case.[ citation needed ]

An integration operation could be short-lived (e.g., keeping data in sync across two applications could be completed within a second) or long-lived (e.g., one of the steps could involve the EAI system interacting with a human work flow application for approval of a loan that takes hours or days to complete).[ citation needed ]

Topologies

There are two major topologies: hub-and-spoke, and bus. Each has its own advantages and disadvantages. In the hub-and-spoke model, the EAI system is at the center (the hub), and interacts with the applications via the spokes. In the bus model, the EAI system is the bus (or is implemented as a resident module in an already existing message bus or message-oriented middleware).[ citation needed ]

Most large enterprises use zoned network to create layered defense against network oriented threats. For example, an enterprise typically has a credit card processing (PCI-compliant) zone, a non-PCI zone, a data zone, a DMZ zone to proxy external user access, and an IWZ zone to proxy internal user access. Applications need to integrate across multiple zones. The Hub and spoke model would work better in this case.[ citation needed ]

Technologies

Multiple technologies are used in implementing each of the components of the EAI system:[ citation needed ]

Bus/hub
This is usually implemented by enhancing standard middleware products (application server, message bus) or implemented as a stand-alone program (i. e., does not use any middleware), acting as its own middleware.
Application connectivity
The bus/hub connects to applications through a set of adapters (also referred to as connectors). These are programs that know how to interact with an underlying business application. The adapter performs one-way communication(unidirectional), performing requests from the hub against the application, and notifying the hub when an event of interest occurs in the application (a new record inserted, a transaction completed, etc.). Adapters can be specific to an application (e. g., built against the application vendor's client libraries) or specific to a class of applications (e. g., can interact with any application through a standard communication protocol, such as SOAP, SMTP or Action Message Format (AMF)). The adapter could reside in the same process space as the bus/hub or execute in a remote location and interact with the hub/bus through industry standard protocols such as message queues, web services, or even use a proprietary protocol. In the Java world, standards such as JCA allow adapters to be created in a vendor-neutral manner.
Data format and transformation
To avoid every adapter having to convert data to/from every other applications' formats, EAI systems usually stipulate an application-independent (or common) data format. The EAI system usually provides a data transformation service as well to help convert between application-specific and common formats. This is done in two steps: the adapter converts information from the application's format to the bus's common format. Then, semantic transformations are applied on this (converting zip codes to city names, splitting/merging objects from one application into objects in the other applications, and so on).
Integration modules
An EAI system could be participating in multiple concurrent integration operations at any given time, each type of integration being processed by a different integration module. Integration modules subscribe to events of specific types and process notifications that they receive when these events occur. These modules could be implemented in different ways: on Java-based EAI systems, these could be web applications or EJBs or even POJOs that conform to the EAI system's specifications.
Support for transactions
When used for process integration, the EAI system also provides transactional consistency across applications by executing all integration operations across all applications in a single overarching distributed transaction (using two-phase commit protocols or compensating transactions).

Communication architectures

Currently, there are many variations of thought on what constitutes the best infrastructure, component model, and standards structure for Enterprise Application Integration. There seems to be consensus that four components are essential for a modern enterprise application integration architecture:[ citation needed ]

1. A centralized broker that handles security, access, and communication. This can be accomplished through integration servers (like the School Interoperability Framework (SIF) Zone Integration Servers) or through similar software like the enterprise service bus (ESB) model that acts as a services manager.
2. An independent data model based on a standard data structure, also known as a canonical data model. It appears that XML and the use of XML style sheets has become the de facto and in some cases de jure standard for this uniform business language.
3. A connector, or agent model where each vendor, application, or interface can build a single component that can speak natively to that application and communicate with the centralized broker.
4. A system model that defines the APIs, data flow and rules of engagement to the system such that components can be built to interface with it in a standardized way.

Although other approaches like connecting at the database or user-interface level have been explored, they have not been found to scale or be able to adjust. Individual applications can publish messages to the centralized broker and subscribe to receive certain messages from that broker. Each application only requires one connection to the broker. This central control approach can be extremely scalable and highly evolvable.[ citation needed ]

Enterprise Application Integration is related to middleware technologies such as message-oriented middleware (MOM), and data representation technologies such as XML or JSON. Other EAI technologies involve using web services as part of service-oriented architecture as a means of integration. Enterprise Application Integration tends to be data centric. In the near future, it will come to include content integration and business processes.[ citation needed ]

Implementation pitfalls

In 2003 it was reported that 70% of all EAI projects fail. Most of these failures are not due to the software itself or technical difficulties, but due to management issues. Integration Consortium European Chairman Steve Craggs has outlined the seven main pitfalls undertaken by companies using EAI systems and explains solutions to these problems. [5]

1. Constant change: The very nature of EAI is dynamic and requires dynamic project managers to manage their implementation.
2. Shortage of EAI experts: EAI requires knowledge of many issues and technical aspects.
3. Competing standards: Within the EAI field, the paradox is that EAI standards themselves are not universal.
4. EAI is a tool paradigm: EAI is not a tool, but rather a system and should be implemented as such.
5. Building interfaces is an art: Engineering the solution is not sufficient. Solutions need to be negotiated with user departments to reach a common consensus on the final outcome. A lack of consensus on interface designs leads to excessive effort to map between various systems data requirements.
6. Loss of detail: Information that seemed unimportant at an earlier stage may become crucial later.
7. Accountability: Since so many departments have many conflicting requirements, there should be clear accountability for the system's final structure.

Other potential problems may arise in these areas:[ citation needed ]

• Lack of centralized co-ordination of EAI work. [6]
• Emerging Requirements: EAI implementations should be extensible and modular to allow for future changes.
• Protectionism: The applications whose data is being integrated often belong to different departments that have technical, cultural, and political reasons for not wanting to share their data with other departments

Related Research Articles

The Common Object Request Broker Architecture (CORBA) is a standard defined by the Object Management Group (OMG) designed to facilitate the communication of systems that are deployed on diverse platforms. CORBA enables collaboration between systems on different operating systems, programming languages, and computing hardware. CORBA uses an object-oriented model although the systems that use the CORBA do not have to be object-oriented. CORBA is an example of the distributed object paradigm.

Middleware in the context of distributed applications is software that provides services beyond those provided by the operating system to enable the various components of a distributed system to communicate and manage data. Middleware supports and simplifies complex distributed applications. It includes web servers, application servers, messaging and similar tools that support application development and delivery. Middleware is especially integral to modern information technology based on XML, SOAP, Web services, and service-oriented architecture.

Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating systems and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.

An enterprise service bus (ESB) implements a communication system between mutually interacting software applications in a service-oriented architecture (SOA). It represents a software architecture for distributed computing, and is a special variant of the more general client-server model, wherein any application may behave as server or client. ESB promotes agility and flexibility with regard to high-level protocol communication between applications. Its primary use is in enterprise application integration (EAI) of heterogeneous and complex service landscapes.

Tuxedo is a middleware platform used to manage distributed transaction processing in distributed computing environments. Tuxedo is a transaction processing system or transaction-oriented middleware, or enterprise application server for a variety of systems and programming languages. Developed by AT&T in the 1980s, it became a software product of Oracle Corporation in 2008 when they acquired BEA Systems. Tuxedo is now part of the Oracle Fusion Middleware.

An enterprise messaging system (EMS) or messaging system in brief is a set of published enterprise-wide standards that allows organizations to send semantically precise messages between computer systems. EMS systems promote loosely coupled architectures that allow changes in the formats of messages to have minimum impact on message subscribers. EMS systems are facilitated by the use of structured messages, and appropriate protocols, such as DDS, MSMQ, AMQP or SOAP with web services.

In computing and systems design a loosely coupled system is one in which each of its components has, or makes use of, little or no knowledge of the definitions of other separate components. Subareas include the coupling of classes, interfaces, data, and services. Loose coupling is the opposite of tight coupling.

Microsoft BizTalk Server is an Inter-Organizational Middleware System (IOMS) that enables companies to automate business processes, through the use of adapters which are tailored to communicate with different software systems used in an enterprise. Created by Microsoft, it provides enterprise application integration, business process automation, business-to-business communication, message broker and business activity monitoring.

An XML appliance is a special-purpose network device used to secure, manage and mediate XML traffic. They are most popularly implemented in service-oriented architectures (SOA) to control XML-based web services traffic, and increasingly in cloud-oriented computing to help enterprises integrate on premises applications with off-premises cloud-hosted applications. XML appliances are also commonly referred to as SOA appliances, SOA gateways, XML gateways, and cloud brokers. Some have also been deployed for more specific applications like Message-oriented middleware. While the originators of the product category deployed exclusively as hardware, today most XML appliances are also available as software gateways and virtual appliances for environments like VMWare.

Mule is a lightweight enterprise service bus (ESB) and integration framework provided by MuleSoft. The platform is Java-based, but can broker interactions between other platforms such as .NET using web services or sockets.

Oracle Fusion Middleware consists of several software products from Oracle Corporation. FMW spans multiple services, including Java EE and developer tools, integration services, business intelligence, collaboration, and content management. FMW depends on open standards such as BPEL, SOAP, XML and JMS.

IBM APP Connect Enterprise is IBM's integration broker from the WebSphere product family that allows business information to flow between disparate applications across multiple hardware and software platforms. Rules can be applied to the data flowing through the message broker to route and transform the information. The product is an Enterprise Service Bus supplying a communication channel between applications and services in a service-oriented architecture.

A message broker is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally-defined messages. Message brokers are a building block of message-oriented middleware (MOM) but are typically not a replacement for traditional middleware like MOM and remote procedure call (RPC).

System integration is defined in engineering as the process of bringing together the component sub-systems into one system and ensuring that the subsystems function together as a system, and in information technology as the process of linking together different computing systems and software applications physically or functionally, to act as a coordinated whole.

Service-oriented architectures (SOA) are based on the notion of software services, which are high-level software components that include web services. Implementation of an SOA requires tools as well as run-time infrastructure software. This is collectively referred to as a service-oriented architecture implementation framework or (SOAIF). The SOAIF envisions a comprehensive framework that provides all the technology that an enterprise might need to build and run an SOA. An SOAIF includes both design-time and run-time capabilities as well as all the software functionality an enterprise needs to build and operate an SOA, including service-oriented:

Oracle Enterprise Service Bus, a fundamental component of Oracle's Services-Oriented Architecture suite of products, provides integration of data and enterprise applications within an organisation and their connected enterprises.

Enterprise Integration Patterns is a book by Gregor Hohpe and Bobby Woolf and describes 65 patterns for the use of enterprise application integration and message-oriented middleware in the form of a pattern language.

An integration platform is software which integrates different applications and services. It differentiates itself from the enterprise application integration which has a focus on supply chain management. It uses the idea of system integration to create an environment for engineers.

A canonical model is a design pattern used to communicate between different data formats. Essentially: create a data model which is a superset of all the others ("canonical"), and create a "translator" module or layer to/from which all existing modules exchange data with other modules. The individual modules can then be considered endpoints on an intelligent bus; the bus centralises all the data-translation intelligence.

The JBoss Enterprise SOA Platform is free software/open-source Java EE-based service-oriented architecture (SOA) software. The JBoss Enterprise SOA Platform is part of the JBoss Enterprise Middleware portfolio of software. The JBoss Enterprise SOA Platform enables enterprises to integrate services, handle business events, and automate business processes, linking IT resources, data, services and applications. Because it is Java-based, the JBoss application server operates cross-platform: usable on any operating system that supports Java. The JBoss SOA Platform was developed by JBoss, now a division of Red Hat.

References

1. Linthicum, David S. (2000). Enterprise Application Integration. Addison-Wesley Professional. ISBN   978-0-201-61583-8.
2. In its April 2001 report for AIIM International, "Enterprise Applications: Adoption of E-Business and Document Technologies, 2000–2001: Worldwide Industry Study," Gartner defines EAI as "the unrestricted sharing of data and business processes among any connected applications and data sources in the enterprise."
Gable, Julie (March–April 2002). "Enterprise application integration" (PDF). Information Management Journal. Retrieved 2008-01-22.
3. Hohpe, Gregor; Woolf, Bobby (2015). "Messaging Patterns Overview". Enterpriseintergationpatterns.com and Addison-Wesley. Retrieved 2016-05-19.
4. MSquare Systems (2014-05-21). "Types of EAI". Archived on 2014-05-21 at https://web.archive.org/web/20140521124430/http://www.msquaresystems.com/enterprise-application-2/eai. MSquare Systems Retrieved on 2014-05-28 from http://www.msquaresystems.com/enterprise-application-2/eai.
5. Trotta, Gian (2003-12-15). "Dancing Around EAI 'Bear Traps'" . Retrieved 2006-06-27.
6. Toivanen, Antti (2013-10-25). "Avoiding Pitfalls of Integration Competency Centers". Archived from the original on 2017-07-30. Retrieved 2013-10-26.