ICO: DeepMind-NHS deal broke data laws

The Royal Free NHS Foundation Trust failed to comply with data protection law when it provided patients’ details to Google AI firm DeepMind, according to the Information Commissioner’s Office (ICO).

As part of a trial to test an alert, diagnosis and detection system for acute kidney injury, the trust provided personal data of around 1.6 million patients to DeepMind back in September 2015, revealed seven months later by the New Scientist.

However, an investigation by the ICO discovered several shortcomings in how the data was handled, including that patients were not adequately informed that their data would be used as part of the test.

DeepMind had a deal with the Royal Free where the former would process partial patient records containing personally identifiable information (PII) held by the trust.

The PII in question included data on people who had presented for treatment in the previous five years for tests together with data from the trust’s existing radiology electronic patient record system. Under the terms of the agreement, DeepMind would process approximately 1.6 million such partial records for clinical safety testing.

But Information Commissioner Elizabeth Denham determined that these records were processed for the purpose of clinical safety testing without patients being informed of this processing.

“The Commissioner was not satisfied that the Royal Free had properly evidenced a condition for processing that would otherwise remove the need to obtain the informed consent of the patients involved and our concerns in this regard remain,” the ICO said in a letter to the trust.

It added that the mechanisms to inform those patients that their data would be used in the clinical safety testing of the Streams application were inadequate.

“In short, the evidence presented to date leads the Commissioner to conclude that data subjects were not adequately informed that the processing was taking place and that as result, the processing was neither fair nor transparent,” said the ICO.

“Patients would not have reasonably expected their information to have been used in this way, and the trust could and should have been far more transparent with patients as to what was happening,” Denham said in a statement.

“We’ve asked the trust to commit to making changes that will address those shortcomings, and their co-operation is welcome. The Data Protection Act is not a barrier to innovation, but it does need to be considered wherever people’s data is being used.”

The ICO won’t fine the NHS trust or DeepMind. Instead, the trust has been asked to establish a proper legal basis under the Data Protection Act for the DeepMind project – which was and for any future trials, and to set out how it will comply with its duty of confidence to patients in any future trial involving personal data.

Plus, it must complete a privacy impact assessment, including specific steps to ensure transparency, and commission an audit of the trial, the results of which will be shared with the Information Commissioner, and which the Commissioner will have the right to publish as she sees appropriate.

“We welcome the ICO’s thoughtful resolution of this case, which we hope will guarantee the ongoing safe and legal handling of patient data for Streams,” said DeepMind co-founder Mustafa Suleyman, and DeepMind Health clinical lead, Dominic King.

“In our determination to achieve quick impact when this work started in 2015, we underestimated the complexity of the NHS and of the rules around patient data, as well as the potential fears about a well-known tech company working in health,” they added.

“We were almost exclusively focused on building tools that nurses and doctors wanted, and thought of our work as technology for clinicians rather than something that needed to be accountable to and shaped by patients, the public and the NHS as a whole. We got that wrong, and we need to do better.”

The pair said DeepMind has published all NHS contracts since its “mistake” in failing to publicise the Streams contract, which it also replaced with “a far more comprehensive contract” in November 2016.

DeepMind said it’s also since developed a patient and public engagement strategy, and is currently awaiting the published findings of nine “independent reviewers” it tasked with scrutinising DeepMind Health.