About The Role
The role designs, builds, and maintains the data pipelines and infrastructure that move data from raw sources to clean, analytical-ready tables clients rely on daily.
You will work across the modern data stack - Python, Airflow, dbt, Snowflake, Kafka, and Spark - and will be expected to own your pipelines end-to-end.
Key Responsibilities
• Build and maintain ETL/ELT pipelines using Airflow or Prefect to move and transform data from operational databases, APIs, event streams, and SaaS sources
• Develop dbt data models following dimensional modeling and OBT patterns; implement dbt tests, documentation, and data lineage
• Administer and optimize Snowflake or Databricks environments: clustering keys, warehouse sizing, cost management, and query performance
• Build real-time streaming pipelines using Kafka and Spark Structured Streaming for event-driven data products
• Implement data quality monitoring using Great Expectations or dbt tests; build alerting pipelines for data SLA violations
• Collaborate with analytics engineers, data scientists, and ML engineers to ensure their data access and quality requirements are met
• Write clear technical documentation for pipeline architecture, data dictionaries, and on-call runbooks
What We Are Looking For
• 2–6 years of data engineering experience with evidence of production pipeline ownership
• Python proficiency: pandas, PySpark, and scripting for data processing and orchestration
• Airflow or Prefect for workflow orchestration; hands-on experience designing DAGs in production
• SQL expertise: complex queries, window functions, performance tuning across Snowflake, BigQuery, or Redshift
• dbt experience (any version): model development, testing, and documentation
• Familiarity with at least one cloud data platform: Snowflake, BigQuery, or Databricks
• Bonus: Kafka/Flink for streaming, Delta Lake, dbt Semantic Layer, or Fivetran/Airbyte connector management
Location
San Francisco Bay Area (Hybrid)
• New York City
• Chicago
• Dallas
• Seattle
• Boston
• Remote considered
Apply tot his job
Apply To this Job