ETL using Python: Build advanced data workflows on Rivery with Python

The highly anticipated Python integration is here. For data engineers using Rivery, this was one of the most requested features, especially among advanced data teams that need programming-based solutions for complex data analysis and workflows. 

While we’re officially announcing this new capability today, we have already been working with amazing design partners and customers to test and optimize it. In addition to improving our product, we have gained a deeper understanding of what our customers want, need – and which are some of the most important use cases that by adding Python to the mix, helps users take their data processes to the next level.

With most data platforms choosing “no-code,” why Python ETL tools?

Within the realm of tools and platforms to build data pipelines and workflows, it can be said that the overall industry trend gravitates towards simple, easy-to-use, no-code solutions that can fit most use cases and needs. In fact, Rivery is no different and we take pride in offering the ecosystem to “plug-and-play” when it comes to connecting, transforming, and orchestrating data. However, working with the world’s most advanced data teams means that addressing most use cases isn’t enough. 

Teams who work on edge cases that need powerful customization are welcoming this new launch with open arms. Especially for companies that need complex data analysis for machine learning, this new integration is essential. Ultimately, we want to offer Rivery users the best of both worlds. Data teams and analysts can manage and orchestrate their data without the need from dev and engineering teams – which covers 80-90% of Rivery users’ needs. For the additional 10-20%, by adding Python as a native source/target in Rivery’s ETL workflows, data teams can now handle any of their complex data pipeline needs.

Python for ETL in Rivery

In essence, we have simplified the process of leveraging Python. With our new interaction, we’ve reduced the process to 3 steps on Rivery. 

First, just pull the data directly into your DataFrame. Secondly, execute your python script using the data you just pulled into your DataFrame. Finally, simply push the transformed data into your data warehouse. 

This new feature is available to all users of Rivery to facilitate custom connectivity, complex transformations, AI/machine learning, and data enrichment.

Want to see the Python for ETL in action?

If you want to see what Rivery’s new Python integration looks like, don’t miss this 10-minute video introduction. With help from one of our super-talented solutions engineers, Ujjwal “JJ” Tamhankar, you can quickly understand what the capability looks like in action. In addition, we briefly discuss how data teams can fully harness the power of Python, as well as some of the key use cases for which data engineers are now using this highly requested feature.

Want to learn more? Check out our on-demand Python webinar, where you can discover new Python use cases and see how your organization can leverage Python to create optimal data workflows.

See how you can run custom Python code directly within a no-code ETL platform and easily get your data into (or out of) Python without the need to write any connectivity code. Watch today!