Note: The job is a remote job and is open to candidates in USA. Empower Retirement, LLC is committed to transforming financial lives and fostering an inclusive environment. The Data Scientist will support the Responsible AI strategy by developing and validating methods for evaluating AI applications, ensuring alignment with business and regulatory needs.
Responsibilities
- Apply statistical analysis and experimentation to evaluate generative AI, agentic AI, and other AI-enabled applications
- Develop and maintain Responsible AI measures, benchmark datasets, test suites, and evaluation criteria
- Evaluate AI systems for fairness, bias, explainability, transparency, hallucination, safety, reliability, and robustness
- Use evaluation results to support the design, testing, selection, and ongoing assessment of AI-enabled business applications
- Establish performance baselines and help identify changes in model or application behavior over time
- Design and execute proofs of concept for new Responsible AI evaluation approaches
- Assess emerging Responsible AI tools, frameworks, and methodologies for validity, limitations, and business applicability
- Support the integration of Responsible AI testing into application development, quality assurance, and governance processes
- Partner with engineering, architecture, quality assurance, governance, and business teams to align evaluation methods with use case requirements
- Create and maintain model cards, evaluation reports, methodology documentation, and related guidance
- Communicate findings, limitations, risks, and recommendations to technical and nontechnical stakeholders
- Contribute to Responsible AI training, internal education, applied research, and industry engagement
- Monitor emerging technologies and evaluation methods related to generative AI, agentic AI, and Responsible AI
Skills
- Bachelor's degree in Data Science, Computer Science, Statistics, Mathematics, Psychology, Economics, Operations Research, or another quantitative discipline
- 2 to 5 years of experience in data science, statistical analysis, AI evaluation, applied research, or analytics
- Strong Python programming skills and experience working with data analysis and AI evaluation tools
- Strong SQL skills and experience working with large, complex datasets
- Experience designing experiments, selecting appropriate measures, and interpreting statistical results
- Experience evaluating generative AI applications, large language models, agentic systems, or other AI-enabled solutions
- Knowledge of Responsible AI concepts, including fairness, bias, explainability, transparency, hallucination, safety, reliability, and robustness
- Experience developing evaluation datasets, benchmarks, test cases, scorecards, or performance measures
- Experience translating evaluation results into recommendations for business applications and technical teams
- Experience with source control, automated testing, technical documentation, and basic software engineering practices
- Ability to assess technical methodologies, identify limitations, and communicate findings clearly
- Strong written and verbal communication skills
- Ability to work independently and collaborate across technical, governance, and business teams
- Advanced degree in a quantitative, behavioral science, computer science, AI, or Responsible AI-related discipline
- Experience evaluating large language models, generative AI applications, retrieval-augmented generation systems, or agentic AI solutions
- Experience with model cards, system cards, AI evaluation frameworks, or AI assurance documentation
- Familiarity with Responsible AI standards and frameworks, including NIST or ISO guidance
- Experience with fairness metrics, explainability techniques, hallucination and grounding evaluation, human evaluation, LLM-as-judge methods, or adversarial testing
- Experience applying Responsible AI evaluation methods within production or business-facing applications
- Experience in financial services or another regulated industry
- Experience contributing to technical publications, conferences, research collaborations, or industry working groups
Benefits
- Medical, dental, vision and life insurance
- Retirement savings – 401(k) plan with generous company matching contributions (up to 6%), financial advisory services, potential company discretionary contribution, and a broad investment lineup
- Tuition reimbursement up to $5,250/year
- Business-casual environment that includes the option to wear jeans
- Generous paid time off upon hire – including a paid time off program plus ten paid company holidays and three floating holidays each calendar year
- Paid volunteer time — 16 hours per calendar year
- Leave of absence programs – including paid parental leave, paid short- and long-term disability, and Family and Medical Leave (FMLA)
- Business Resource Groups (BRGs) – BRGs facilitate inclusion and collaboration across our business internally and throughout the communities where we live, work and play. BRGs are open to all.
- Other necessary computer equipment, will be provided.
Company Overview