Sr. Data Engineer

Job Description:

The Sr. Data Engineer in our innovative team will focus on revolutionizing healthcare through AI and generative AI technologies. This role will be instrumental in designing and implementing robust data pipelines to support the development of a Healthcare analytics data platform for a variety of healthcare applications and AI models. This is a critical role in transforming healthcare data into actionable insights, ensuring compliance with industry standards while leveraging cutting-edge technologies.

Data Engineering & Pipeline Development:

• Design, implement, and optimize ETL/ELT pipelines to support data ingestion and transformation from diverse healthcare data sources.
• Develop and maintain scalable data pipelines for real-time and batch processing to meet meet traditional BI applications and AI/ML needs.
• Work with structured, semi-structured, and unstructured data to create usable datasets for AI model training and deployment.

Database Management:

• Manage and optimize a variety of databases, including Postges, NoSQL, graph databases, and cloud-based databases like Snowflake and Redshift.
• Ensure efficient storage, retrieval, and integration of data across different systems.

AI Integration:

• Collaborate with data scientists and AI engineers to create datasets for a variety of use cases.
• Implement data annotation, curation, and augmentation techniques to enhance data readiness for AI/ML applications.

Healthcare Data Expertise:

• Apply knowledge of healthcare data standards such as HL7 and FHIR, ensuring adherence to HIPAA compliance and other regulatory requirements.
• Address challenges related to unstructured data extraction and large-scale data ingestion in EMR/EHR systems.

Collaboration & Innovation:

• Work closely with cross-functional teams, including healthcare specialists, business analysts and data architects to understand and address data needs.
• Ensure data quality, integrity, and security, especially when dealing with sensitive healthcare data.

Continuous Improvement:

• Stay updated with emerging technologies and frameworks in data engineering, AI, and healthcare.
• Contribute to the continuous improvement of processes and tools within the data engineering team.

Perform other duties that support the overall objective of the position.

Education Required:

• Bachelor’s degree (or higher) in Computer Science, Data Science, Artificial Intelligence, or a related field.
• Or, any combination of education and experience which would provide the required qualifications for the position.

Experience Required:

• 10+ years of proven experience in designing and building enterprise-grade ETL/ELT pipelines following modern architectural paradigms such as Data Lake, Lakehouse, and data mesh for high-volume analytics..
• Experience working with cloud platforms like AWS, Azure, or Google Cloud.
• Hands-on experience implementing star schema data models and orchestrating data pipelines to support scalable AI/ML model training and deployment.
• Hands on experience with ETL/ELT development using tools such as dbt, Fivetran, Spark, Snowpark including orchestration via Airflow or similar.
• Experience handling diverse data types and ensuring scalability in a complex environment.

Knowledge, Skills & Abilities:

• Knowledge of: Familiarity with big data frameworks such as Apache Spark, Kafka, and Hadoop. Understanding of data processing for voice, image, and text-based AI solutions. Familiarity with healthcare data standards and interoperability protocols. Knowledge of EMR/EHR systems and healthcare data management challenges along with healthcare data standards and coding systems (FHIR, HL7, ICD-10, CPT, etc.).
• Skill in: Strong programming skills in Python, and/or Scala, Java, and SQL. Excellent problem-solving and debugging skills. Strong communication.
• Ability to: Collaboration abilities to work with diverse stakeholders.

The company has reviewed this job description to ensure that essential functions and basic duties have been included. It is intended to provide guidelines for job expectations and the employee's ability to perform the position described. It is not intended to be construed as an exhaustive list of all functions, responsibilities, skills and abilities. Additional functions and requirements may be assigned by supervisors as deemed appropriate. This document does not represent a contract of employment, and the company reserves the right to change this job description and/or assign tasks for the employee to perform, as the company may deem appropriate.

is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

Originally posted on