About the Role

We’re seeking a Senior Data Engineer to take ownership of a complex affiliate and content-driven data ecosystem. You’ll work closely with our existing Data Engineer and Data Scientist to optimize and modernize data pipelines, enhance observability, and implement an orchestration framework. This position is contract-to-full-time, with the expectation that you’ll transition into a permanent leadership role on successful completion of the contract.

Key Responsibilities

1- Data Pipeline Ownership

Manage and optimize existing ETL pipelines that ingest data from affiliate management systems (Rakuten, Impact), web/CDN logs, and various ad networks.
Maintain and enhance custom Python scripts, Ayrbyte, Big Query Table sync, etc for data ingestion, transformations, and reverse ETL processes.
Oversee and expand our composable CDP (Hightouch) to enable identity resolution and audience segmentation, including deeper email integration.

2- Orchestration & Observability

Drive the decision, architecture, and implementation of an orchestration framework (currently leaning toward Dagster or Prefect).
Develop robust observability, monitoring, and alerting strategies to reduce downtime and simplify troubleshooting.

3- Data Modeling & Warehouse Architecture

Refactor and expand core data models to accommodate new business lines and future growth.
Collaborate with stakeholders to define dimensional models and fact tables for reporting, analytics, and campaign optimization.

4- Reporting & Analytics

Support existing reporting efforts (currently in Hex, with R/Python) to ensure data is accurate, timely, and easily accessible.
Work alongside our Data Scientist on advanced analytics initiatives, such as modeling, experimentation & AB testing.

5- AI Integration

Incorporate AI-driven tools and methods into day-to-day tasks where sensible, leveraging emerging technologies to enhance pipeline performance and scalability.
Stay current on AI trends and innovations, ensuring our architecture evolves alongside cutting-edge best practices.

6- Collaboration & Leadership

Partner with engineering, analytics, full stack web devs and business teams to align data strategies with company objectives.
Mentor junior engineers, share best practices, and foster a culture of continuous improvement.

Required Qualifications

Expert Python Skills: Proven experience in building and maintaining data pipelines, custom transforms, and scripts.
SQL Mastery: Expert-level proficiency in SQL for complex transformations (using DBT), analytics queries, and performance optimization. Must be comfortable debugging intricate queries and designing maintainable, scalable SQL-based workflows.
GCP Familiarity: Competence with Google Cloud Functions, BigQuery, and related GCP services; advanced GCP knowledge is a plus.
ETL/ELT Experience: Hands-on experience with DBT, Airbyte, or similar tools for data ingestion, transformation, and management.
Orchestration & Observability: Exposure to frameworks like Airflow, Dagster, or Prefect, with a strong desire to drive architectural decisions.
CDP Experience: Familiarity with Hightouch, Segment, RudderStack, or similar composable CDPs.
Data Modeling: Deep understanding of data warehousing, dimensional modeling, and best practices.
AI Enthusiasm: Active use of AI tools or methods in daily workflows and an eagerness to apply AI pragmatically to improve systems and processes.

Soft Skills

Strong problem-solving skills and ability to thrive in a fast-paced environment.
Excellent communication and collaboration skills, with an ability to explain complex data concepts simply.
Self-starter who can work independently and as part of a team.
Batteries Included: Capable of taking high-level business and technical requirements from concept to production with minimal oversight. Proactively seek team input and resources as needed to ensure success.
Resilience in the Face of Changing Priorities: Adapt quickly and maintain composure when business needs shift. Manage changing scopes or timelines gracefully while preserving quality and momentum.

Job Types: Full-time, Contract

Pay: $126,444.00 - $137,483.00 per year

Benefits:

Dental insurance
Health insurance
Paid time off
Vision insurance

Compensation Package:

1099 contract
Hourly pay
Yearly pay

Schedule:

8 hour shift
No nights
On call

Application Question(s):

When hiring for full time positions, our company starts with a contract project of 1-3 month duration to make sure we're a mutual excellent fit. Please confirm you understand this requirement.
Please describe your experience with implementing and managing orchestration tools.

Work Location: Remote

Throttle Up Technologies, LLC

New Jobs

Senior Data Engineer