Data integration trends are continually changing, especially that today enterprises need to precisely plan how they are going to digitalize their operations and services to remain competitive. Data integration solutions are a critical element of digitalization. This is why this year again we wanted to look at the data integration trends of 2019.
In the last 30 years, a lot has been changing in the data integration scene. The first data integration system was designed in 1991 by the University of Minnesota using a data warehousing approach to extract, transform, and load data to attain data from multiple sources. It enabled large-scale data integration, however, the process had to be repeated frequently.
Companies have been using EDI since the early 70s to exchange information electronically with their trading partners to speed up the communication and enhance their processes.
In the 2000s, Service Oriented Architecture (SOA) was gaining popularity for developing integration solutions. This approach relied on data mappings between the mediated schema and the schema of the original source and by this adding new connections became a lot easier.
In the last five years, integration platform as a service (iPaaS) has been trending among data integration buyers, as it is simplifying and speeding up the way integrations can be developed, deployed, and maintained and it is all done in the cloud.
While many celebrate blockchain as the technology that will revolutionize connectivity management and data sharing among trading partners, it is important to start one’s digitalization journey with a bit of housekeeping. This means that once you have an integration strategy and you’ve executed all elements of it, you are well connected both internally and externally, then you’ll be able to implement such exciting, revolutionary technologies like blockchain.
In this blog, we have gathered seven trends that are going to shape the future of enterprises. Regardless of the size of your organization, you’ll perhaps find something that you’ll want to implement too in 2019.
Find our article on last year's trends here >>
I read an article recently in Harvard Business Review called ‘Digital Transformation is Racing Ahead, and No Industry Is Immune’. It stated the following:
“ Research shows that since 2000, 52 percent of companies in the Fortune 500 have either gone bankrupt, been acquired, or ceased to exist as a result of the digital disruption.”
This means one thing: those that fail to digitalize and innovate will go extinct very quickly.
While there are an emerging amount of new SaaS applications available that can help to make businesses more digitalized, these applications alone cannot tackle the integration challenges that enterprises face today. Let us explain these in more details in this article.
Enterprises have many systems and departments internally that lack connectivity, thus collaboration is ineffective as they lack real-time data sharing (see number 3 about EAI). Due to globalization, companies today have more trading partners, customers, manufacturers, and partners than ever before and they need to be able to connect with all of them quickly to improve cooperation.
It’s easier said than done though. Whether it’s an Enterprise Application Integration (EAI) case or a B2B integration case, the challenges are the same: technologies and applications were built in different decades, thus connecting them is not an easy task. Even when the systems are connected, there can be challenges such as heterogeneous data formats and issues with the quality of the data.
To overcome these challenges efficiently and quickly, enterprise integration platform as a service (EiPaaS) can be a relief for many. There are different kind of EiPaaS: you can find one for citizen integrators (e.g. for business people with no IT skills to quickly connect business applications), subscription-based iPaaS (when you pay for the iPaaS, but your IT team needs to learn to use it and develop the integration solutions on top of it), as well as fully managed iPaaS (in this case, your iPaaS vendor will take care of the development, deployment, and maintenance of your integration solutions, so you don’t have to have an IT team specialized in integrations.
iPaaS is gaining popularity compared to traditional integration solutions as its key capabilities make it a lot easier to develop and manage integrations. Additionally, iPaaS is always built on a private or public cloud so that buyers will see the common benefits of the cloud (such as scalability, high availability, etc.). As with iPaaS, developing integrations is a lot faster, it also helped to reduce data integration prices, thus it democratized integrations – once a prohibitive overhead cost is now in the reach of all.
For the last few years, Gartner has been emphasizing the strategic importance of hybrid integration platform (HIP). At the same time, Gartner also stresses that there are only a few iPaaS providers on the market that can develop genuinely hybrid integration solutions.
As companies are rushing to become more digitalized, connecting their legacy on premise systems is a prerequisite for success. Building interoperability between on-premise technologies and cloud-based modern applications cannot be achieved without a state-of-the-art iPaaS that acts like a HIP and can tackle the challenges different technologies and data formats mean.
Hybrid integrations will extend the lifecycle of your legacy technology investments. Instead of retiring your old systems (that are typically working well, they were just not designed for interconnectivity) and investing in new systems that can be risky, you can just add a layer of integration to your on-premise technologies and connect them to any more modern applications.
Enterprises have started to realize the advantages of Enterprise Application Integration (EAI). Internally, an organization may use hundreds if not thousand software and business application. Nevertheless, these applications are rarely capable of communicating with each other. Connecting all systems, software and applications can have a significant impact on one’s organizational efficiency. Connectivity can help enterprises to break down data silos and improve processes just by sharing data automatically in real-time.
Information sharing has an impact on how different departments can cooperate and how departments will be able to optimize their processes. Additionally, EAI allows companies to automate some of the processes that they have. By increasing connectivity and data sharing across the entire company, operations will become more agile.
In the last few years, we have seen an increasing need for B2B integration. Enterprises have realized the benefits of B2B integrations, as well as how data integration providers can help them to overcome some of the challenges that they have been facing in terms of B2B integrations.
While EDI integration has been around for decades, it doesn’t mean that communication with trading partners have been easy.
Just by reading this article until here, you can imagine how many challenges one may change when they need to connect with one of their partners to exchange data. Imagine, if you need to connect with hundreds of stakeholders.
You will encounter several roadblocks, such as technologies built in different decades, on-premise technologies and cloud-based applications, tens of different data formats (even proprietary ones), or poorly documented API that your partners cannot utilize without some help.
Once you need to connect with more than two partners, B2B integration can get extremely challenging, therefore it’s vital to have adequate data integration tools available.
During the last years, iPaaS has been gaining popularity among integration architects and it’s a great tool for challenging B2B integration scenarios. Most iPaaS vendors are selling simply the tool, so you have to have a team that can learn to use it, and develop integrations on top of that. Others may provide fully managed data integration solutions utilizing an iPaaS. This means that they will be responsible for building, testing, deploying, and managing your integration solutions.
I am convinced that we are living in an API economy. What does this mean? APIs help to connect to pieces of software so those two can talk to each other. There are no new software firms that wouldn’t have an API that their customers and partners can use for connecting with them. Also, many established companies are also releasing APIs to make it easier for their stakeholders to connect with them and share information.
APIs are great and I am convinced that in the future, we will use them even more for creating connectivity across two endpoints, as it can improve partnerships and even the customer experience.
If APIs have been around and Forbes declared 2017 to be the “Year of the API economy”, why do we talk here about APIs as a data integration trend of 2019?
Simply, because we have seen in quite a few customer cases that having an API available doesn’t mean that everyone can automatically use it and connect to your software. The API documentation cannot be 500 pages long. It has to be as simple as possible, so really anyone can connect with your software. If it’s not, then you better have a data integration partner that will help your trading partners and customers to connect with your API.
For sure, API will have a critical role in your integration strategy, but in 2019 focus on the ease-of-use of your API.
Investing in an iPaaS has become extremely popular in the last few years (just look at the chart of Gartner!). Nevertheless, most of the iPaaS providers only supply you with a tool. You are purchasing the rights to use the integration platform – nothing more. In addition to that, you either hire integration architects that got the experience of using the tool in question or you will also be paying for their training so they would become expert in using the tool. As integration experts are scarce, hiring the right people will be a challenge itself.
In the last few years, managed integration solutions have emerged. This means that instead of just buying the rights to use an integration platform, you will really get what the name claims: integration platform as a service. Your iPaaS vendor will provide you with fully managed integration services. This includes the usage of the platform, the development, testing, and deployment work, as well as the maintenance of the solutions once the integrations have been deployed. It’s beneficial because the vendors often have long experience in developing and deploying integrations faster, so you will be able to connect with your ecosystem and share data in real-time quickly. This also means that you don’t have to have an internal IT team that is responsible for integrations.
Often, when a company is looking to implement new data integration solutions, it’s a lot bigger project than simply connecting to endpoints and automating the data flow between them. More often, they are ready to connect tens or sometimes hundreds of stakeholders and send millions of messages monthly.
This means two things: you can’t implement such a large project without having an integration strategy and a chief digital officer (CDO) that will lead the changes that you are about to implement by utilizing data integration solutions.
As CDO is quite a new role, you need someone that has the business acumen and experience in change management, as well as an understanding of IT as well as how the digital change can have a serious impact on the company’s operations.
Depending on the size of your organization and on your integration solution needs, we are confident that everyone will find something on this list that can be applied to their business.
Whichever integration solutions you may need, make sure you have a plan in place, find the adequate tools to implement the plan and name a person within your company that will be responsible for driving the change.