Yotpo leverages Rivery’s Databricks integration to boost end-to-end data flow and cost savings

About

Yotpo is an eCommerce marketing platform that integrates with leading marketing tech stack and data-driven solutions for reviews, loyalty, SMS marketing, subscriptions, and more. The Forbes cloud 100 company powers forward-thinking global brands to create smarter, higher-converting experiences that spark and sustain customer relationships.

Before Rivery, we’d load data to S3, then run a Spark job using another tool to prep parquet files to load to Databricks. It was a painful process. As soon as we started using Rivery, the frictionless onboarding and ease-of-use made us realize we made the right choice.
 
Assaf Liebstein Data Platform Team Leader, Yotpo

Challenge

Yotpo has over 100 internal data users from R&D and customer success to support and solution engineers managing behavioral data from embedded products as well as B2B and B2C data from their customers’ data lake. 

Yotpo needs to move data quickly between systems and transform it for analysis and reporting using Looker and Tableau.  

To enable consumption and analysis of the data in the BI tools, Yotpo needed a solution to easily manage all the integration from business apps  to Databricks. 
 

Three different tools to solve a single problem

We tried out three different tools to solve a single problem – data ingestion. Every time, we ended up with more issues than we started out with.

Initially Yotpo used a low-touch tool for data integration. Due to the tool’s limitations, they could only load data from business applications such as NetSuite and Jira to AWS S3 instead of directly to Databricks. To alleviate the problem, they started using another tool to query data from S3 and rewrite parquet files to load to Databricks. But schema issues kept cropping up. Yotpo tested out another tool to ingest data to Delta tables in Databricks but it turned out to be very expensive. The functionality was broken and the Delta tables couldn’t be written properly. 

You can’t rely on the premise that all integrations are the same. You need one solution that lets you custom build your integrations. It takes so much time to build and without proper support, it’s just too painful 

To maximize their chance of success they tried a different route. One that offered more advanced data integration capabilities to answer their more complex use cases.
 

Solution

Yotpo turned to Rivery to provide the solution they were looking for to power up their data flow from source to Databricks. It quickly became apparent that Rivery was the right match. 

Rivery fit in with their current ecosystem,  communicating their compute language and making table formats in the data lake easily accessible. By switching to Rivery, Yotpo could leverage Rivery’s pool of data connectors with Databricks, significantly reducing the complexity involved in managing a data pipeline across multiple tools by different vendors.

Yotpo has expanded its solution use case and can now create data pipelines for other apps even without a native connector in Rivery such as Opsgenie. This is all due to Rivery’s capability with custom connection and pipeline against any REST API. 

Beyond this use case, Rivery’s integration with Snowflake has also allowed Yotpo’s HR team to access sensitive data without the need of an admin. They’ve built an autonomous system that only an HR analyst can manage and have adopted a successful data mesh strategy.

Rivery has enabled our HR unit to manage their data product without the need to go through a centralized data team.

Yotpo has achieved coherent data democratization. The data team can access the data they need and model it in the data warehouse without wasting precious data engineering time. 

Yotpo has completely transformed their DataOps process. Their BI and Business app team now have the freedom to handle different integrations and add tables on their own. They can easily ensure the data in the data lake is on par with what’s in the integration. That saves Yotpo a ton of time and resources every single month. 

Thanks to Rivery’s UX, less technical users are interacting directly with the platform – removing a lot of the maintenance and operational burden from data engineers. 

Rivery is a great option to support different integrations. It aligns with our mission in creating a more self-service data infrastructure. It is a cost-effective solution – and you can’t beat that in the current economic climate.