Integration projects have a terrible reputation. They are costly, they take a lot of time, and, more often than enough, in the end, they may not deliver the desired outcome at all.
According to some studies, as many as 70% of all system integration projects fail to achieve the goals set for the project. In this blog, we discuss the various reasons that can lead to this situation and try to provide solutions on how to avoid the most common mistakes regarding data integration projects.
The list provided in this blog is by no means an exhaustive list. There are many other reasons as well that may lead to the failure of an integration project. Also, the various reasons listed below do not mean either that if any such circumstance exists, a project is automatically going to fail. In many cases, even successful integration projects may face many similar issues. Nevertheless, typically the knowledge and expertise of the people involved can help to save the project from a disaster.
Why does an integration project fail?
We have discussed various system integration methods earlier in our blog (check out the post “What is system integration?”). We concluded that there are plenty of different options to design system integration between various systems.
In the most straightforward scenarios, system integration projects can be relatively simple and straightforward, but that does not mean that even such projects would be completely free from failures and challenges.
The reason for this is that, in many cases, the failure of projects is actually not because of the integration software/tools used, the functionality of the different end systems that are integrated, or any other technical difficulties during the integration project build phase.
The real reasons for the failure are, in fact, related to management issues regarding the integration.
So, let’s have a look at the various management-related issues that may cause an integration project to fail. Some of these examples are based on our real-life experience working with prospects and customers.
1. Oversimplifying the integration project
The easiest way to get an integration project to start off down the wrong path is to ignore the complexities involved in the processes that the integrated systems handle. People who do not know or understand the business processes involved can very easily merely overlook extremely important details regarding the integration. This will, in turn, lead to incorrect planning and scoping of the project, which is a guaranteed recipe for failure.
2. Overestimating own capabilities
This is a very common issue in everything we all do in business and is by no means limited to integrations.
We all tend to overestimate our own capabilities in doing stuff that we are not really that familiar with. This, combined with the above-mentioned oversimplification of the project, will more than double the risk of failure.
How this behavior is demonstrated in the system and data integration field is that some companies tend to simply go out and buy integration tools with the assumption that they will be able to use them “out of the box” or “with minimal training”.
In reality, they may find themselves owning an expensive tool that they are not able to use. It’s kind of like buying a surfboard and expecting to catch the first six-foot wave that comes by. Reality will generally sink in pretty fast in these cases.
For integration projects, the first casualty is usually the project schedule. When a simple project starts to last for several months, someone has probably overestimated their capabilities.
3. Focusing too much on technical integration
The goal of an integration project shouldn’t be just a technical solution that enables the various IT systems to exchange information between them.
When this is the case, what happens is that the solution design will lead to an excessive mapping exercise and the most complex integration framework that is very rigid and difficult to develop further in the future.
When starting an integration project, a far better approach is first to endeavor to find a clear mutual understanding between all stakeholders on the outcome of the project, i.e., start with “what we all are trying to achieve here together”.
This lays a foundation for process changes that can be achieved as part of the integration project, which increases the value of the project outcome tremendously. Suddenly, you are not just integrating various systems, but you’re improving your fundamental business processes.
4. Lack of clear accountability
Integration projects typically have multiple stakeholders that may have various conflicting interests and requirements.
Sales & Marketing may have entirely different integration requirements than the finance department, for example, and the poor IT department usually sits in the middle of all this, trying to keep everyone happy without having a vested interest in the whole integration project (for them the project might be a nuisance more than a benefit).
Larger organizations may have a clear owner for all this (typically the CIO or CDO of the company), but small and medium-size companies seldom have such luxury.
However, it is vital that before an integration project is kicked off the person who will be ultimately responsible and accountable for the final structure and design of the system is clearly defined. This person also apparently needs to have the ability to take the final decisions between conflicting interests.
5. Constant changes in the environment
With complex system integration projects, the different applications that are integrated hardly ever stay the same. Constant change is more of a rule than an exception to it. In today’s application landscape, the pace of change seems to be ever-increasing, and absolutely nothing stays the same for extensive periods of time.
Yet, many integration projects are designed in a way where they are intended to resolve an integration challenge at a single point in time.
Using a waterfall model to design the integration in detail is not really a feasible delivery methodology as the rapid pace of change makes the original designs obsolete very quickly.
We have seen several integration projects where the traditional IT project model fails to keep up with the ever-changing application landscape. This is especially true for projects where the application to be integrated is under construction itself.
To succeed with the integration project in these cases, you need a very agile delivery methodology that allows for changes to happen instantly and dynamically throughout the project. Furthermore, even after the initial project is over, you need to ensure that the maintenance of the system integration is handled in the same manner.
6. "Build it and they will come” assumption
Today, organizations very often decide to tackle their business-to-business integration challenges by publishing an API so that their business partners or customers can connect with them more easily.
However, in most cases, this will not provide the desired outcome as you are not providing a solution to the integration problem. Instead, you are simply pushing the problem into another party’s lap.
As one of our prospects put it: “We have this great API, and our partners would like to use it, but they don’t know how to do it.”
This summarizes the whole issue with the APIs. If you do not know how to integrate with other parties, do not expect them to understand how to connect with you either. APIs are great, but only to professional people who understand integration processes and can work with the APIs (which brings us to our next point).
7. Shortage of integration experts
Excellent integration experts are a rare breed (here, at Youredi, we are lucky enough to have quite a few great ones!). The reason for this is that technical knowledge and expertise alone is typically not enough to make the integration project a success, you need an integration expert that also has knowledge and understanding about different business processes and can guide you through the complex requirements involved. This combination is not something you can easily find.
Integration experts are architects that help design the optimal integration framework. Finding these experts has proved to be tough even for IT companies, but it is even harder for companies whose core business is something entirely different. Moreover, as many large organizations have outsourced their IT department to third-party service providers, companies simply don’t have any integration expertise in-house anymore.
8. Protectionism
As strange as it may sound, often the integration projects run into trouble for the very same reason that they are initiated. Namely, the lack of access to data in other systems.
The ”owners” of different systems are unwilling to grant access and share the data in their system with other stakeholders in the same organization. There may be various reasons for this that stem from, e.g., fear of losing independence, fear of transparency of information, fear of losing one's importance within the organization, etc. None of these reasons should stop an integration process, but in practice, companies run into these issues all the time.
This phenomenon becomes even more apparent when integrating with third parties as some of their business may be based on the data they possess in their application, and they’re very reluctant to share it with others. For an integration project to overcome these issues, you need strong support the senior management of the company that can override any protectionist objections that may arise during the project.
9. "Not invented here” attitude
Probably one of the most common issues that we have faced regarding integration projects is the resistance of various internal stakeholders. The most typical one of these is the internal IT department.
While the business should be the driver for an integration project, the IT department sometimes tends to forget its role as a support organization, and it wants to act as the driver for the integration. This then easily leads to an argument by the IT department that “we can do this ourselves”.
Yet, the average IT department does not have the required skill set to take on a complex, enterprise-wide integration project. The typical IT department is also not very good at contacting third parties to engage in B2B integration.
So, what typically tends to happen in these situations is that the organization loses 6-12 months’ time (or sometimes even longer) before they realize that they cannot do the integration project by themselves.
One way to avoid this is to ask your IT department to provide a detailed project and resourcing plan and ask them to commit to it. If they still insist on doing it, you may consider giving them a chance.
10. Not starting at all.
When an organization is designing a large-scale B2B integration project, they often feel that they may have created a monster.
Suddenly there are so many aspects to take into consideration that people start questioning the feasibility of the whole project. In one big block, a system and data integration project can be quite overwhelming, and in the end, organizations end up doing nothing at all. They simply settle with the status quo and accept the fact that their systems and business processes will remain sub-optimal. However, a well-designed systems integration project can be executed in smaller sub-projects, keeping each individual sub-project short and enabling readjustment of their prioritization along the way.
This also reduces the overall risks related to the project, and each sub-project alone will deliver tangible benefits to the organization quickly and even if the entire systems integration project takes longer.
How to succeed?
Although integration projects are quite demanding, the benefits that they bring to organizations are unquestionable. Therefore, organizations should not be discouraged by the above-mentioned potential reasons for failure. They are simply matters that need to be taken into consideration before starting an integration project. All the above-listed challenges can be overcome by good design and competent management of the integration project. The rest is just hard work.
Our experience is that the most successful integration projects are conducted by open-minded organizations that are willing to engage an integration service provider that guides them through the project.
In some cases, our customers have been faced with tremendous pressure from their business partners (or internal stakeholders) and therefore have had no choice but to look for new innovative ways to solve their integration challenges. This has not always been easy for them, as the old saying goes, “When faced with the choice between changing one’s mind and proving that there is no need to do so, almost everyone opts for the latter”, but in the end, their system integration project has delivered the desired outcomes in record time and under budget.
If you want to learn how to succeed in developing integration projects and how an iPaaS can support your integration strategy, download our new Inside the iPaaS eBook.
Also, we recommend you to read our article on "How To Excel In Your Next Integration Project?"
