Building a Top-Performing DataOps Team

In my recent presentation at DataOps.rocks 2021, I outlined what it takes to build a top-performing DataOps team. The following is an overview of all the key points.

DataOps is a flexible framework. By design, teams will include temporary stakeholders during the sprint process. However, a permanent group of data professionals must power every DataOps team.

Each organization will form a different DataOps unit. But in the DataOps teams that Rivery worked with, certain skill sets are clearly in-demand. Here are some of the personnel that often play key roles.

Executive

Profile

Focus: Building a team, measuring the business (true view of ARR), build IT infrastructure, support the product
through technology
Experience: 10 years in IT management, MS/MBA, product development, internal
IT ops
Common Tools: Python, SQL, Google Presentations, business dashboards
Average Salary: $250,000 (Source: Built-In)

Solves What Pain Points for Organization?

Provides Easy Access To Data

• Builds data stack that can serve business users across company
• Creates access data on the fly for immediate usage & experimentation

Enhances ROI via Technology

• Maximizes budget to hire optimal data engineering teams
• Optimizes subscriptions to multiple tools & technologies

Empower Technical Teams to Focus on Main Priorities

• Streamlines the DataOps process so teams can focus on product & customers – not data integration
• Enables teams across organization to focus on high-priority tasks

Scale That Doesn’t Fail

• Contingencies to avoid system breakage – scale is unpredictable, but could happen at any second
• Allows company to avoid disruptions & scale up in the long-term

Data Architect

Profile

Focus: Translate business requirements into technology requirements and define data standards and principles.
Experience: 5 years, network management, distributed databases and processing, application architecture and performance management
Common Tools: Python, Perl, Java, SQL, relational databases, ETL
Average Salary: $148,000 (Source: Built-In)

Solves What Pain Points for Organization?

Translates Business Needs to Data Infrastructure

• Translates business needs into technical specifications
• Produces data streams, integrations, transformations, and other to power business users

Empower the Data Team

• Generates reference architecture for organization
• Enables others to create and improve data systems

Deliver on Data Agility

• Builds the frameworks, standards, and principles of data architecture
• Defines modeling, metadata, security, reference data, and master data

Reduce Data Workflow Friction

• Defines data flows for organization, including who generates data and what data is is used for
• Increases data efficiency, security, & control

Data Quality Analyst

Profile

Focus: Identify, compare, and resolve data quality problems in an organization.
Experience: Entry-level, data interpretation & implementation, review & analysis, business priorities
Common Tools: ETL, SAS, JavaScript, SQL, Excel, MySQL, Unix/Linux
Average Salary: $78,000 (Source: Salary.com)

Solves What Pain Points for Organization?

Ensures Data Quality

• Deploys QA testing to guarantee data quality
• All data is accurate & usable for organizational stakeholders.

Improves Business Decisions

• Provides business users with data that influences better decision-making
• Improves business’s bottom line

Owns Trust in Data

• Enables business users to believe the data they rely on.
• Gives business users trust & confidence in data-driven initiatives.

Data Engineer

Profile

Focus: Building & maintaining pipelines. Migrating to the cloud from legacy systems.
Experience: 5 years, architecture design, data pipeline maintenance/testing, managing data & metadata.
Common Tools: ETL/ELT tools, SQL, NoSQL, Python, CDW, database architecture.
Average Salary: $124,000 (Source: Built-In)

Solves What Pain Points for Organization?

Converts Raw Data for Business Use

• Builds the data pipelines/infrastructure that deliver business users the data they need.
• Extracts, cleans, prepares, and transforms data for business usage.

On Call Support for Data Infrastructure

• Fixes breakages and other data infrastructure issues on call.
• Urgent API updates and pipeline maintenance

Assures Access to All Relevant Data Flows

• Organizes and makes relevant data accessible to business users.
• Manages data streams to maintain usability of system.

Speeds time-to-insight

• Automates & streamlines manual data workflows.
• Eliminates inefficient bottlenecks that delay data for stakeholders.

Supports Custom Integrations

• Builds custom data pipelines, custom API connectors.
• Creates data connectors for business apps & products.

Data/BI Analyst

Profile

Focus: Delivering dashboards and insights. Understanding the metrics that drive the company. Finding areas where insights can drive ROI.
Experience: 2 years, B.A., data management, reporting, some programming proficiency
Common Tools: Python, SQL, Looker, Tableau, SiSense, Qlik, Excel
Average Salary: $83,000 (Source: Built-In)

Solves What Pain Points for Organization?

Develops & Measures Business KPIs

• Develop core understanding of business objectives, KPIs, & projects to drive meaningful insights
• Deliver insights to relevant stakeholders across the company.

Ensure Timely Delivery of Insights

• Leverage agile development and other DataOps principles to provide insights
  within agreed timeframes.

Bridges Gap Between IT/Business Users

• Liaise between IT (data engineers) and business users to provide business users with the data they need.

Internal Communication & Promotion of Insights

• Communicate & present insights in a way stakeholders can understand.
• Leverage the right mediums for reports (dashboards, spreadsheets, etc.)
  Data Scientist

Data Scientist

Profile

Focus: Apply scientific methods, processes, algorithms and systems to data to generate advanced analytics and predictive insights.
Experience: 5 years, computer science, modeling, statistics, analytics, and math skills.
Common Tools: R, Python, SQL, SAS, Apache Spark, ML tools, D3.js
Average Salary: $132,000 (Source: Built-In)

Solves What Pain Points for Organization?

Enhances Business Decisions

• Employs CS, statistics, and advanced math to produce predictive insights.
• Enables business users to make more incisive and strategic decisions.

Harnesses AI/Machine Learning

• Builds and deploys AI/machine learning models to unlock new layer of insights.
• Enhances competitive edge of business.

Tames Massive Data Volumes

• Develop algorithms and data models that can activate the value in massive volumes
  of data.
• Navigate size & complexity of company data to create insights.

Decreases Uncertainty

• Tackle future initiatives with confidence through the predictive power of data science.
• Grow all facets of the business – from acquisition to product – with more certainty.

Citizen DataOps

Profile

Focus: Use tools & platforms to perform DataOps tasks, including building pipelines & preparing data.
Experience: Sales, marketing, Customer Success, analysts
Common Tools: ETL/ELT platform, pre-built data pipelines, orchestration & transformation
Average Salary: Varied

Solves What Pain Points for Organization?

Save Time & Resources

• Empower line-of-business (LOB) users to execute DataOps tasks with the right tools.
• Saves engineers & analysts time and resources by handling tasks themselves.

Unlock Untapped Value

• Enable LOB users to generate insights from untapped data.
• Turn unused data into new streams of opportunity & profitability.

Maximize Data-Driven Decisions

• Power decision-making at every level of your organization with data.
• Equip any team member with the data they need, at the moment.

Top-Performing DataOps Teams: Beyond Technical Skills

Beyond technical skill sets, DataOps team members must also possess critical personal qualities, such as leadership, communication, and the ability to work with employees outside of the data team. But there is still a last piece of the puzzle. DataOps teams must also have the right technology in place. To learn about the ideal tech stack, read our blog on DataOps toolchains.

And thanks to DataOps.rocks for giving me a forum — I’ll see you again next year!