I did my first data integrations in the late 90’s. It was just some simple scripts for an IRC bot, that pulled data from the internet request and sent it as a message to its users. They weren’t called integrations then, but that’s what it was. There wasn’t any fancy frameworks or services to use, it was just low-level code that did everything.
With the 2000’s some web APIs started to pop up, but still integrating to them required a lot of work. We implemented a lot of Windows and Unix services that were usually scheduled to run at certain intervals and checking for new data, transforming it and pushing it onwards to some database, FTP server or an API. They weren’t great pieces of software, but they got the job done, orders and invoices were moving, and not a lot of errors were happening.
When system integrations became a commodity
As 2010 approached, the logistics industry really woke up. Suddenly, everyone wanted their systems to be integrated. There were some standards that were proposed to be used, but most ignored them and instead created their own message formats. What we noticed while implementing these projects was that there was a lot of work in building and maintaining everything else in these solutions, except the actual business logic related things.
Integrations became more complex
As the amount of data being moved through the integrations grew, monitoring and logging error situations weren’t as trivial anymore as it was before. External APIs and message formats kept changing all the time, users were producing poor quality data, there were bugs in different systems and all kinds of unexpected things started happening that broke the integrations every now and then. Often the integration services were running on customers’ servers that were behind some firewall, required different VPNs or weren’t accessible at all.
The result of all this was that integration services started to be quite complex and large software projects. As they were being implemented at an increasing pace, this meant that each new service was a bit different as before, since the software evolved along the way. So, maintaining and doing changes to them got difficult too, as you had to change between contexts that were similar, but not similar enough.
iPaaS: Instant love
The first time I implemented an integration using an iPaaS system, I was immediately sold. I realized that these systems allowed us to concentrate building the business logic and not needing to worry that much about the architecture and infrastructure behind the integration, therefore we can develop integrations faster.
The iPaaS system takes care of the trivial tasks, which required more perspiration than inspiration, than when they were built by hand. The best example for this was logging. I haven’t met a single developer that likes to implement logging. Everyone knows you need it, but it’s just a pain in the ass to build, and it’s never in those places where you end up needing it when exceptions occur. When it’s already built into the iPaaS, it’s there for all your projects and works the same way in each of them.
Deploying integrations is easy
Another great improvement is how easy deploying is. Since the integration logic is running in the service, you can do live development, without having to deploy each time you make a change, but just edit, save, test, and repeat. It takes like a minute to set up a base for an integration for you to work on, and you need to deploy locally only those parts of the system that absolutely need a local resource to work on.
Error handling simplified
And then there’s error handling. Whether you’re using HTTP, FTP, SFTP, AS2, its implementation in code is never as easy as it sounds. You end up spending too much time in implementing a simple file transfer, handling all the different errors, defining retry policies and stuff like that when you should be using time on creating value for your customers. Setting up schedules, installing software, maintaining and updating server software, or even worse, the hardware is something that you just had to do before, but it’s not productive work in any way, it’s just overhead.
With an iPaaS, you don’t need to worry about this stuff. You have a working system running in the cloud and what you can do is just pick ready-made tools for the trivial stuff and then spend your time more wisely.
Integrations are easier to maintain when built with iPaaS
The system encourages you to build your integrations using similar implementations each time, so it’s easier to document and maintain them. You get a centralized management using a web browser, so no need to have multiple VPN clients installed anymore. Protocol implementations take care of the common errors and retrying. Scheduling is just a matter of a few clicks and as easy to monitor, as everything else running through the system. There are in-built tools for you to log and manage error situations. And you can even let your customers’ support team into the same system to handle those error situations, if applicable.
Scaling, scaling, and scaling your integrations – couldn’t be easier!
And then there’s of course scaling. No need to worry that the server you got your services running on will eventually run out of resources, you’re in the cloud now. So, if you haven’t yet tried out implementing integrations in an iPaaS system, give it a try. It will totally change your way of working!
For more information and specs, download our iPaaS product sheet:
