How Armstrong saved 1,500 data analyst hours with Rivery

Rivery has been a force multiplier for us. You don’t have to be a data engineer to get up and running. If you have experience as a data analyst you can get started quickly and figure it out as you go.
Sr. Manager of Data Analytics at Armstrong Transport Group 

Results

• Achieved a staggering 7500% reduction in the time taken to extract and consolidate data from a single source, slashing it from 10 hours to 8 minutes.
• Streamlined connectivity to any data source including niche freight applications within minutes rather than months.
• Expanded data accessibility organization-wide, transitioning from exclusive access for the finance department to a minimum of 300 users across the company in Sigma by year-end.

Armstrong Transport Group is a non-asset-based logistics provider headquartered in Charlotte, NC. Their proprietary Transportation Management System (TMS) supports a network of independent freight agent offices across the US and Canada, offering comprehensive logistics solutions. By connecting shippers to a vast multi-modal carrier network, Armstrong helps optimize shipping costs while enhancing visibility and control of shipments for businesses of all sizes across various industries. 

Armstrong’s emphasis on data utilization distinguishes it from its logistics sector counterparts. Armstrong stands at the forefront of this movement in the freight industry, leveraging its 19-year data repository to provide informed strategies and solutions

Challenge

The company’s data infrastructure consisted of Power Automate for integration, and Power BI for data analysis.

Historically, data utilization was primarily restricted to the accounting and finance departments, which heavily relied on pre-generated reports for tasks like reconciliation and gap analysis. Integrating essential data sources, like DAT’s truckload freight marketplace, proved challenging because of outdated technology and the absence of native connections for seamless integration. As a result, individuals outside executive, accounting, or finance roles lacked interaction with data and access for decision-making purposes, underscoring a significant limitation of the outdated data infrastructure.

Consequently, the marketing department, responsible for campaigns across platforms like Facebook, LinkedIn, Google, and Instagram, operated in the dark regarding the ROI of their ad endeavors. With the data team already stretched thin, Armstrong estimated it could require a senior data analyst months to address the marketing department’s data needs.

Looking at the backend of the social media APIs is really complex, and as a lean data team, we couldn’t afford a senior data analyst going down this rabbit hole to connect to those sources and showcase the ROI for each platform.
– Senior Manager of Data Analytics, Armstrong Transport Group. 

Even though individuals in the finance and accounting department had access to information, they still relied on the data team to generate reports and update data for month-end reporting. For example, a senior-level data/IT employee dedicated approximately 1500 hours annually to extracting and consolidating data from their Sage ERP system to finalize month-end book closures. 

Solution

The data team at Armstrong was searching for a modern data integration platform that was flexible to address the company’s evolving data needs.

The Armstrong team put together a list of 94 requirements they were looking for in an ELT solution and conducted POCs with six vendors.They landed on Rivery after they completed the following during the two days they allotted to test out each platform.

2. Connect to any data source – As technology adoption increases within the freight industry, readily available connectors for essential data sources are becoming more prevalent. With over 37 providers, the data team needed to establish connections to all data sources and automate that process. The data team could easily create custom connections via Rivery’s actions or serverless Python. Rivery then handles upserting that data into Snowflake without requiring extra coding.“Rivery offers way more flexibility than other providers when it comes to connecting to custom sources that are unique to our line of business.”
3. Transactional database to data warehouse migration – This was a non-negotiable for Armstrong’s team as they needed to quickly establish a consistent database connection. With their application data stored in SQL server databases, Armstrong needed the flexibility to pull with CDC or standard SQL. Rivery provided both options for Armstrong: CDC to reduce the application database load and daily snapshots through standard SQL query extraction.
4. Easily recreate pipelines – Crunched for time, the data team needed a solution that allowed them to easily recreate pipelines without having to learn a complex platform. Rivery’s platform was intuitive enough to perform tasks and offered the flexibility Armstrong needed to serve any use case.

Outcome

The modernized approach to data architecture allowed the Armstrong data team to easily democratize data across their organization. Their new data stack consisted of the following tools:

• Rivery: For building end-to-end data pipelines
• Snowflake: For their cloud data warehouse
• Sigma: For self-service data analysis

By year-end, Armstrong anticipates having a minimum of 300 users leveraging Sigma for data analysis autonomously, eliminating the need for data team intervention.

Our leadership has been inherently data-driven, leading to a strategic approach in their adoption of technology. With Rivery and Sigma, we easily created a self-service approach to working with data at Armstrong.

The realization of a self-service approach materialized when the data team automated the extraction of custom Sage data triggered by business stakeholders, reducing the finance and accounting departments’ reliance on the data team. What was once a time-consuming process of extracting and consolidating data into a single file, previously taking 10 hours per cycle around five times per month, now only takes 8 minutes. 

With the custom connection to DAT, their team could optimize operations by accessing load-to-truck ratios for every key market area for every Van, Reefer, and Flatbed. Since moving to Rivery, the data team has been able to handle an influx of requests from departments looking to utilize data, like the marketing team. The data team used Rivery’s pre-built kits to ingest ad data from various sources like Facebook, LinkedIn, Google, and Instagram. The pre-made kits combine data from multiple entities in the respective APIs and provide a full data model for tracking each ad platform’s ROI.

The hardest part of this process was figuring out what credentials the service accounts needed. Rivery’s pre-made kits saved months of our team’s time.

Utilizing Rivery, Snowflake, and Sigma, the Armstrong data team facilitated the democratization of data for business stakeholders. This initiative not only empowered Armstrong to maintain a competitive edge in the freight industry but also offered users a self-service platform for seamless data interaction.

Sources:
Facebook Social, Google Ads, Google Sheets, Linkedin Ads, Linkedin Social, SQL Server, REST, Amazon S3, Custom Python (DAT and Highway)

Targets:
Snowflake, Sigma