The Health Gym: An open platform with health-related benchmark problems for the development of reinforcement learning algorithms

Reinforcement learning (RL) algorithms hold tremendous promise for personalising healthcare but their development has been hampered by a lack of openly available data with sufficient clinical detail. We will create The Health Gym, a set of publicly accessible healthcare-related ‘benchmark problems’ (tasks with examples from patient records) for developing, testing and comparing RL algorithms. The first two RL problems distributed as part of The Health Gym will concern the management of sepsis patients in the intensive care unit (ICU) and the optimisation of anti-retroviral therapy for people with HIV. We will create synthetic, but realistic, patient records using privacy-preserving generative adversarial networks (GANs) and publicly available datasets (MIMIC-III and EuResist). This will allow us to freely distribute the detailed clinical data required to solve the proposed problems.

If these datasets are sufficiently non-disclosive, we will generate further ICU and HIV synthetic datasets using our own clinical data. The benchmark problems will be distributed as a free and open source package for the Python programming language, a website with suggested ‘proof-of-concept’ solutions and a wiki where researchers can share their solutions. We will also make the GANs and related software publicly accessible through an online software repository, so it can be used to create further synthetic patient records and for educational purposes. 

At the end of the project, we will hold a two-day DataThon for teams of clinicians and data scientists, to help popularise The Health Gym platform and accelerate the development of robust and reproducible RL algorithms to be used in healthcare.