In an open letter to MR Mehra, SS Desai, F Ruschitzka, and AN Patel, authors of
“Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis”. Lancet. 2020 May 22:S0140-6736(20)31180-6. doi: 10.1016/S0140-6736(20)31180-6. PMID: 32450107,
and to Richard Horton (editor of The Lancet):
Concerns regarding the statistical analysis
and data integrity
The retrospective, observational study of 96,032 hospitalized COVID-19 patients from six continents reported substantially increased mortality (~30% excess deaths) and occurrence of cardiac arrhythmias associated with the use of the 4-aminoquinoline drugs hydroxychloroquine and chloroquine. These results have had a considerable impact on public health practice and research.
The WHO has paused recruitment to the hydroxychloroquine arm in their SOLIDARITY trial. The UK regulatory body, MHRA, requested the temporary pausing of recruitment into all hydroxychloroquine trials in the UK (treatment and prevention), and France has changed its national recommendation for the use of hydroxychloroquine in COVID-19 treatment and also halted trials.
The subsequent media headlines have caused considerable concern to participants and patients enrolled in randomized controlled trials (RCTs) seeking to characterise the potential benefits and risks of these drugs in the treatment and prevention ofCOVID-19 infections.
There is uniform agreement that well conducted RCTs are needed to inform policies and practices. This impact has led many researchers around the world to scrutinise in detail the publication in question. This scrutiny has raised both methodological and data integrity concerns. The main concerns are listed as follows:
1.There was inadequate adjustment for known and measured confounders (disease severity, temporal effects, site effects, dose used).
2.The authors have not adhered to standard practices in the machine learning and statistics community. They have not released their code or data. There is no data/code sharing and availability statement in the paper. The Lancet was among the many signatories on the Wellcome statement on data sharing for COVID-19 studies.
3.There was no ethics review.
4.There was no mention of the countries or hospitals that contributed to the data source and no acknowledgments to their contributions. A request to the authors for information on the contributing centres was denied.
5.Data from Australia are not compatible with government reports (too many cases for just five hospitals, more in-hospital deaths than had occurred in the entire country during the study period). Surgisphere (the data company) have since stated this was an error of classification of one hospital from Asia. This indicates the need for further error checking throughout the database.
6.Data from Africa indicate that nearly 25% of all COVID-19 cases and 40% of all deaths in the continent occurred in Surgisphere-associated hospitals which had sophisticated electronic patient data recording, and patient monitoring able to detect and record “non sustained [at least 6 secs] or sustained ventricular tachycardia or ventricular fibrillation”. Both the numbers of cases and deaths, and the detailed data collection, seem unlikely.
7.Unusually small reported variances in baseline variables, interventions and outcomes between continents (Table S3).
8.Mean daily doses of hydroxychloroquine that are 100 mg higher than FDA recommendations, whereas 66% of the data are from North American hospitals.
9.Implausible ratios of chloroquine to hydroxychloroquine use in some continents.
10.The tight 95% confidence intervals reported for the hazard ratios are unlikely. For instance, for the Australian data this would need about double the numbers of recorded death sas were reported in the paper.
The patient data have been obtained through electronic patient records and are held by the US company Surgisphere. In response to a request for the data Professor Mehra has replied; “Our data sharing agreements with the various governments, countries and hospitals do not allow us to share data unfortunately.” Given the enormous importance and influence of these results, we believe it is imperative that:
1.The company Surgisphere provides details on data provenance. At the very minimum, this means sharing the aggregated patient data at the hospital level (for all covariates and outcomes).
2.Independent validation of the analysis is performed by a group convened by the World Health Organisation, or at least one other independent and respected institution. This would entail additional analyses (e.g. determining if there is a dose-effect) to assess the validity of the conclusions.
3. There is open access to all the data sharing agreements cited above to ensure that, in each jurisdiction, any mined data was legally and ethically collected and patient privacy aspects respected.
In the interests of transparency, we also ask The Lancet to make openly available the peer review comments that led to this manuscript to be accepted for publication.
This open letter is signed by clinicians, medical researchers, statisticians, and ethicists from across the world. The full list of signatories and affiliations can be found below.
List of Signatories
Dr James Watson (Statistician, Mahidol Oxford Tropical Medicine Research Unit, Thailand)1
Professor Amanda Adler (Trialist & Clinician, Director of the Diabetes Trials Unit, UK)
DrRavi Amaravadi (Researcher,University of Pennsylvania, USA)
Dr Ambrose Agweyu (Medical researcher, KEMRI-Wellcome Trust Research Programme, Kenya)
Professor MichaelAvidan(Clinician, Washington University in St Louis, USA)
Professor Nicholas Anstey (Clinician, Menzies School of Health Research, Australia)
Dr Yaseen Arabi (Clinician, King Saud Bin Abdulaziz University for Health Sciences, Saudi Arabia)
Dr Elizabeth Ashley (Clinician, Director of the Lao-Oxford-Mahosot Hospital-Wellcome Trust Research Unit, Laos)
Professor Kevin Baird (Researcher, Headof the Eijkman-Oxford Clinical Research Unit, Indonesia)
Professor Francois Balloux (Researcher, Director of the UCL Genetics Institute, UK)
Dr Clifford George Banda (Clinician, University of Cape Town, South Africa)
Dr Edwine Barasa(Health economist, KEMRI-Wellcome Trust Research Programme, Kenya)
Professor Karen Barnes (Clinical Pharmacology, University of Cape Town, South Africa)
Professor David Boulware (Researcher& Triallist, University of Minnesota, USA)
Professor Buddha Basnyat (Clinician, Head of the Oxford University Clinical Research Unit -Nepal, Nepal)
Professor Philip Bejon (Medical researcher, Director of the KEMRI-Wellcome Trust Research Programme, Kenya)
Professor Mohammad Asim Beg(Clinician/Researcher, Aga Khan University,Pakistan)
Professor Emmanuel Bottieau (Clinician, Institute of Tropical Medicine, Antwerp, Belgium)
Dr Sabine Braat (Statistician, University of Melbourne, Australia)
Professor Frank Brunkhorst (Clinician, Jena University Hospital, Germany)
Dr Todd Campbell Lee (Researcher, McGill University, Canada)Professor Caroline Buckee (Epidemiologist, Harvard TH Chan School of Public Health, USA)
Dr James Callery (Clinician, Mahidol Oxford Tropical Medicine Research Unit, Thailand)
Professor John Carlin (Statistician, University of Melbourne & Murdoch Children’s Research Institute, Australia)
Dr Nomathemba Chandiwana (Research Clinician, University of the Witwatersrand, South Africa)
Dr Arjun Chandna (Clinician, Cambodia Oxford Medical Research Unit, Cambodia)
Professor PhaikYeong Cheah (Ethicist/Pharmacist, Mahidol Oxford Tropical Medicine Research Unit, Thailand)
Professor Allen Cheng (Clinician, Monash University, Australia)
Professor Leonid Churilov (Statistician, University of Melbourne, Australia)
Professor Ben Cooper (Epidemiologist, University of Oxford, UK)
Dr Cintia Cruz (PaediatricianMahidol Oxford Tropical Medicine Research Unit, Thailand)
Professor Bart Currie (Director, HOT NORTH, Menzies School of Health Research, Australia)
Professor Joshua Davis (Clinician, President of the Australasian Society for Infectious Diseases, Australia)
Dr Jeremy Day (Clinician, Oxford University Clinical Research Unit, Vietnam)
Professor Nicholas Day (Clinician,Director of the Mahidol Oxford Tropical Medicine Research Unit, Thailand)
Dr Hakim-Moulay Dehbi (Statistician, University College London, UK)
Dr Justin Denholm (Clinician, Researcher, Ethicist, Doherty Institute, Australia)
DrLennie Derde (Intensivist/Researcher, University Medical Center Utrecht, The Netherlands)
Professor Keertan Dheda (Clinician/Researcher, University of Cape Town,& Groote Schuur Hospital, South Africa)
Dr Mehul Dhorda (Clinical Researcher, Mahidol Oxford Tropical Medicine Research Unit, Thailand)
Professor Annane Djillali (Dean of the School of Medicine,Simone Veil Université,France)
Professor Arjen Dondorp (Clinician, Mahidol Oxford Tropical Medicine Research Unit, Thailand)
Dr Joseph Doyle (Clinician, Monash University and Burnet Institute, Australia)
Dr Anthony Etyang (Medical Researcher, KEMRI-Wellcome Trust Research Programme, Kenya)
Dr Caterina Fanello (Epidemiologist, University of Oxford, UK)
Professor Neil Ferguson (Epidemiologist, Imperial College London, UK)
ProfessorAndrew Forbes (Statistician, Monash University, Melbourne, Australia)
Professor Oumar Gaye (Clinical Researcher, University Cheikh Anta Diop, Senegal)
Dr Ronald Geskus (Head of Statistics at theOxford University Clinical Research Unit, Vietnam)
Professor Dave Glidden(Biostatistics, University of California, USA)
Professor Azra Ghani (Epidemiologist, Imperial College London, UK)
Prof Philippe Guerin (Medical researcher, University of Oxford, UK)
Dr. Raph Hamers (Clinician/Trialist, Eijkman-OxfordClinical Research Unit, Indonesia)
Professor Peter Horby (Clinical Researcher, Centre for Tropical Medicine and Global Health, University of Oxford)
DrJens-Ulrik Jensen (Clinician/Trialist, University of Copenhagen, Denmark)
Dr Hilary Johnstone (Clinical Research Physician, Independent)
Professor Kevin Kain (Clinical Researcher, University of Toronto, Canada)
Dr Sharon Kaur (Ethicist, University of Malaya, Malaysia)
For correspondence: firstname.lastname@example.org