Integration is generally defined as the integration of a part into a whole. When we speak of system or IT integration, we mean the embedding of a module or subsystem in an existing system landscape. The correct connection of the systems is particularly important to avoid data loss, error messages or delays. The main tasks of system integration are to plan comprehensive technical system solutions and to create corresponding concepts as well as to integrate and put them into operation. It is usually implemented with the help of middleware or integration platforms.
Organizations are confronted with more and more systems and applications and often threaten to lose track of their system landscape and the programs they use. IT landscapes in particular, whose individual components are connected via a wide variety of interfaces, are often difficult to manage. The more extensive and heterogeneous these structures are, the less transparent they are. New systems can only be implemented with great effort and changes also require a lot of time and effort. If you manage to integrate new elements, you can also destabilize the entire system. Any integration can therefore be dangerous.
If such difficulties are to be avoided, it is advisable to implement a centrally fed integration layer. As a link within the system landscape, it compensates for irregularities between the subsystems.
This integration layer is called middleware. It regulates the communication between applications and IT systems and thus enables their cooperation. As a kind of interpreter, it controls the transfer of the required data. Differences and discrepancies between the protocols, languages and formats of the systems are overcome with the help of middleware. The complex structures and differences are hidden in this way.
An example from the retail industry should illustrate the performance of the middleware. Orders are transferred from a Magento online shop to the in-house SAP system using middleware. One advantage is that Magento is not obliged to map the system’s data formats and protocols. Instead, the middleware itself accepts new orders via a webshop SOAP interface. These are then converted and the saved key figures and information are forwarded as iDoc to the SAP system. Even if the receiving system is not permanently online, the transmission takes place in this form. If the goods are returned to a store, the middleware is used to record the reversal in the POS software. In addition, the online shop inventory is adjusted immediately.
As a rule, the basic element of middleware is an integration infrastructure, for example an ESB (Enterprise Service Bus). Based on this, interactive integration solutions can be created. In combination, the resulting solutions result in so-called middleware.
The decision for or against the purchase of middleware can be compared to the decision for the equipment of a conference room. Smaller groups often only need a distribution socket and a LAN hub to work effectively in their conference room. However, if there are more than two or three participants, additional equipment is usually required. So you need to organize, connect and install additional adapters, hubs or cables. The time required for this was usually planned as the actual speaking time. If 20 or more colleagues take part in our conference, the room no longer meets the requirements. The preparations and technical coordination would take so much time that the actual conference could only be started very late. However, if meetings of this magnitude take place more frequently, it is advisable to set up a room that has been equipped with the necessary technology.
The middleware can be imagined in a similar way. The introduction of middleware is advisable, especially for large system landscapes that comprise a large number of applications and software. However, if it is not necessary to link more than 2 or 3 technically identical systems (e.g. SQL and SOAP) with each other, the purchase could prove to be superfluous. However, if you regularly use different systems that have to be connected via changing interfaces (for example, REST, iDoc, SOAP, and SQL), you should decide to use middleware. After successful integration, it ensures that differences between your systems are technically obscured and smooth business processes are implemented. New modules are easier to integrate and adjustments are simplified.
There are three general dimensions which can be used to define your scenarios.
Endpoints – Which protocols do I have?
The systems you need to integrate run on-premise, in the cloud, on mobile end-devices or on smart devices. Typically in each case differing communication protocols will apply. SQL, iDOc, EDI and FTP run on-premise. In the cloud you will find SOAP, REST or ODATA and AMPQ for your smart devices. If middleware is now used, the complexity of your protocols is hidden. The systems to be integrated can simply be connected with your protocol in the middleware.
Deployment – Which network topology do I have?
If the systems you need integrating are distributed across a complex network – on-premise, cloud or mobile devices – the middleware creates transparency. It provides connectors that allow a “local” connection for your systems. The result is that aspects such as firewalls and DMZ are no concern for these systems.
The middleware must naturally be operated somewhere. Ideally, your integration platform should be located close to your systems. Appropriate solutions therefore are on-premise installations, operation in the cloud as IPaaS, embedded solutions or a mix of these models.
Integration patterns – what do I want to integrate?
Here we differentiate roughly between four variants: data integration, process integration, application integration and B2B scenarios.
Data integration is about the synchronization of data across multiple distributed systems. For example product master data between ERP and online shop or ETL scenarios with corresponding monitoring.
Process integration is about the integration of a customer order across the distributed systems: online shop, ERP, Logistics, payment, CRM and the monitoring of the overarching process.
Application integration is about the seamless integration of distributed applications. For example the provision of on-premise ERP functionality or data on demand for a tablet app used by technical field staff.
B2B scenarios such as the integration of systems of record with those from B2B partners by means of electronic data transfer (EDI).
Each of these scenarios carries within it differing challenges and each challenge justifies the use of middleware. Should you even have a crossover at multiple points with your scenarios, then middleware is a worthwhile investment for your company.