Note: The job is a remote job and is open to candidates in USA. McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. The Sr. Data Engineer will assist in the design, development, and maintenance of scalable data solutions that support analytics and business intelligence, while collaborating with cross-functional teams to optimize data processes.
Responsibilities
- Assist and lead the design, development, and maintenance of scalable, high-performance data solutions that support analytics, AI, and business intelligence across the organization
- Assist in designing and documenting technical requirements for data flows between diverse operational systems and the data warehouse and support the end-to-end development of ETL/ELT processes using Apache Spark on Databricks
- Contribute to the ingestion of data into data lakes and warehouses, implement Delta Lake for transactional integrity, and help orchestrate both batch and streaming data pipelines
- Collaborate with cross-functional technology teams, on extract, transform, and load data from a variety of sources, build and optimize data models, and maintain software applications aligned with business needs
- Explore and integrate emerging AWS technologies to enhance data engineering capabilities, write complex SQL queries for data validation and transformation, and manage Databricks workspaces, clusters, and jobs
- Support data quality initiatives, ensure compliance with governance and security policies, and participate in the development and operationalization of machine learning and generative AI applications
- Assist in troubleshooting production workflows, refactoring legacy systems, participating in code reviews, and mentoring junior engineers
Skills
- Bachelor's Degree in Computer Science or related field of study
- Five (5) years of experience in the job offered or a related occupation
- Experience in data engineering with Databricks and AWS
- Experience with Apache Spark, SQL, and Python or Scala
- Experience with Delta Lake, Unity Catalog, and cloud data warehouses (e.g., Redshift)
- Experience with data modeling, ETL/ELT processes, and data governance
- Experience with orchestration tools including Gitlab or Bitbucket and CI/CD practices
- Experience in collaborating with cross-functional teams to deliver data solutions using Python and Databricks in Agile environments
- Experience with design and optimizing scalable data pipelines with Python and Spark in Databricks
- Experience with developing modular, reusable components in Databricks notebooks and workflows
- Experience in implementing data validation frameworks to ensure pipeline accuracy and consistency
Benefits
- Competitive compensation package at McKesson as part of our Total Rewards
- Annual bonus or long-term incentive opportunities may be offered
- 100% telecommuting allowed from a home office anywhere in the U.S.
Company Overview
Company H1B Sponsorship