As part of its ‘All Cloud’ push that began to take shape in 2019, Connect for Health Colorado (C4) selected Matillion ETL to be its cloud ETL/ELT tool. At that time Matillion had surfaced as a leader in data movement and transformation in the cloud. This could have been seen as a huge leap of faith for the organization; C4 had been using the Informatica powerhouse product as its on-prem solution since its inception in 2013.
But replacing Informatica with Matillion as they progressed to AWS a difficult choice. Matillion has a data movement and transformation process that is engineered in a way similar to other ETL tools, as shown in the diagram. And while the general concept of transforming data is no different between ETL products, Matillion makes use of the Snowflake engine to do its work instead of processing it in its own workspace. The move to Matillion made sense for these and many other reasons, the key ones being:
1. Matillion would be less expensive to run.
Even after throwing out the sunk costs of the initial licensing fees paid to Informatica back in 2013, the annual support costs paid to Informatica would be more than the expense to run Matillion for the year, which has no licensing or startup costs.
2. Matillion is simpler for data engineers to learn and maintain.
Matillion has excellent online documentation which supports quick learning and troubleshooting. C4 found that engineers new to Matillion could be productive faster than with Informatica.
3. Matillion was built for the cloud.
And so includes, at no additional charge, more than 100 connectors to other cloud-based systems, along with most popular database management systems. Informatica charged a steep price just for adding a connector to another common database such as PostgreSQL. That meant C4 would have to spend money licensing additional Informatica products just to get up and running in AWS.
The plan to do this was hatched in 2019 and conversion work began in earnest in the 1st quarter of 2020. With a deadline to go live before open enrollment beginning November 1, C4 was able to make it to AWS on September 23, 2020 as a ‘big bang’ migration.
How has Matillion performed? It has met and exceeded expectations. The engineers who develop Matillion jobs believe it to be a more productive tool than Informatica, the cost has been predictable and not surprised anyone, and its use has already been expanded at C4. With Matillion, C4 is now able to ingest data from its cloud-based CRM solution, and will soon be getting data from Box and other cloud-based sources.
The post Matillion: strategic to modern at Connect for Health Colorado appeared first on Datalere.