AI Fellow, AI for Health

Location: United Kingdom, London Project Office

PATH current employees - please log in and apply Here

PATH is a global nonprofit dedicated to achieving health equity. With more than 40 years of experience forging multisector partnerships and with expertise in science, economics, technology, advocacy, and dozens of other specialties, PATH develops and scales up innovative solutions to the world’s most pressing health challenges.

About AI for Health

PATH’s Artificial Intelligence (AI) Initiative is our flagship technical team working across divisions to deliver best-in-class evaluation and implementation of AI in health, spanning model evaluation to product development and service innovation. We aim to use AI to address the world’s largest health inequities.

We’re looking for an early career researcher to help deliver a multi-country programme evaluating whether AI can provide safe and effective mental health/wellbeing support for those most in need. You will contribute to research to test and improve how automated evaluation methods (e.g., patient simulation, LLM-judging) reflect people and languages in different settings, and to make effective use of local clinicians and lived experience input.

A key part of the role will be undertaking a fully-funded PhD (supervised by PATH’s Chief AI Officer and Professor at University College London, Dr. Bilal Mateen, and co-supervised by the relevant technical leadership at PATH and UCL), investigating how we can most effectively and efficiently use human expertise and local data to deliver accurate and localised evaluation of AI performance across different languages and contexts.

The role is a strong fit for an evaluation science or computational researcher who wants to shape how AI is assessed across settings and earn a PhD through that work as a route into AI safety and global (mental) health.

Responsibilities

There won’t be a typical day in this role. You could be doing anything from drafting a protocol for a new study to organising a consensus workshop with partners from around the world or running in-silico experiments and analysing data generated by our partners. If you thrive on tinkering with tools, independently playing around with a task/problem, and having a team that supports you in failing fast and learning a lot, then this is the role for you. Support Research Delivery and Operations across the Programme

We’ll carve out a specific set of partners and tasks (aligned with your PhD topic) whom you will be the primary point of contact for, spanning operational and technical needs (as summarised below). Your job will be to make sure those elements of the grant-funded projects we’re collaborating on with our partners happen as planned.

Support the AI for Health team’s research agenda: contribute technical input as the team and its partners design, run, and analyse evaluation studies, including developing and testing simulated users/patients and LLM-judges and running evaluations of AI models and products.

Work with the Deputy Director of Operations and Finance and their team on the operational, administrative, and project management components of research delivery, including ethics and governance submissions, partner reporting, and workplan tracking.

Develop Your Research Skills

Complete a fully funded PhD at University College London as the core vehicle for developing your research skills. Under the guidance of your supervisors, conceive, design, and carry out the studies that make up your thesis, which will address how human expertise and local data can most effectively and efficiently be used to deliver accurate and localised evaluation of AI performance across different languages and contexts. Topics could include:

Analysing what clinicians and lived experience experts in different countries consider to be helpful and harmful AI behaviour: what is shared between contexts, what is local to individual settings, how much different experts agree/disagree, and how plurality of opinion can be represented in our evaluations.

Developing and adapting automated evaluation methods to reflect how people in each setting talk about their mental health and seek support from AI, including in local languages and where people mix languages.

Studying how to make the most of scarce expert time and local data: which ways of collecting expert feedback and data improve our evaluation tools most efficiently, and how much ongoing human input we need to stay reliable.

Contribute to studies led by other members of the AI for Health team, and to the development of the team’s wider research agenda, whilst building your own methodological and technical skills over the course of the post, with the support of your supervisors.

This is a full-time post based in London, combining research delivery at PATH with registration for a PhD at University College London, fully funded for three years. You’ll report to PATH’s Deputy Director for AI for Health, and your PhD will be jointly supervised by leadership at PATH and UCL. The role includes occasional international travel to partner sites (around 5-10% of time).

Required Skills and Experience

MSc or BSc in data science, psychology, cognitive science, statistics, computer science, human–computer interaction, computational linguistics, epidemiology, or a related quantitative discipline.

At least two years’ research experience beyond your degree, in academia, industry, or a research organisation.

Strong grounding in experimental design and measurement—reliability, validity, inter-rater agreement, calibration—and in statistical modelling (e.g., mixed-effects or Bayesian models).

Hands-on experience working with large language models and/or large text datasets, and familiarity with relevant programming languages.

Ability to work well with expert panels and partners from diverse backgrounds, and to handle sensitive mental health content with care.

Excellent written English, with evidence of scientific