Sent: 15 August 2016 11:55
Professor Hardaker,
I wonder if you could help, please. I am writing to you in your capacity as Chairman of the Board of Trustees of Sense About Science (SAS).
I am trying to clarify SAS's position regarding the PACE trial, in particular concerning the analysis done by the trial Principal Investigators (PI) and their refusal to release the trial data. SAS say that they want to 'make a fuss' when something is wrong. A number of independent statisticians and scientists have said that the analyses done by the PACE PIs are wrong. The PIs claim they are restrained by the Data Protection Act (DPA). SAS have publicly supported data-sharing and this interpretation of the DPA could have a stifling effect on all research involving human subjects.
I have sent an email a couple of times to enquiries@senseaboutscience.org and received the automated response confirming receipt. I have also sent a letter to Tracey Brown. I have not had any reply.
I enclose a copy of the letter below.
I would be grateful if you could forward my email to someone who will reply.
Thank you for your help,
John Peters.
Dear Ms. Brown,
I wonder if you would please help me with a number of points concerning Sense About Science and the PACE trial.
First, an article by director of stats.org, Rebecca Goldin, concludes ' the flaws in this design were enough to doom [PACE's] results from the start'. An accompanying editorial by Trevor Butterworth, of Sense About Science USA, finds 'the way PACE was designed and redesigned means it cannot provide reliable answers to the questions it asked'. Does Sense About Science accept the criticism by Goldin as valid? Does Sense About Science agree with the editorial by the organization described as its 'sister', Sense About Science USA? If not, could you say why not, please.
Second, despite these criticisms of the trial from neutral parties, Sense About Science has not made any statement or alerted the media to the trial flaws, to make a fuss about something considered wrong by so many scientists and statisticians. Could you say why not, please? Will you do so?
Third, in discussion of the stats.org articles on Twitter, you said that PACE and the follow-up studies continued to have the support of Sense About Science and All Trials, because the trial results had been made available. Could you please therefore tell me where I can find the following results:
1. The fitness/conditioning, 'step', tests (a graph was included in the trial paper, but not the underlying results).
2. SF-36 physical function scores (range 0-100 points) [baseline and 52-week followup].
3. CFQ fatigue Likert scores (range 0-33 points) [baseline and 52-week followup].
4. CFQ fatigue bimodal scores (range 0-11 points) [baseline and 52-week followup].
5. Oxford criteria CFS caseness (does participant meet criteria, yes or no) [52-week followup only].
6. Participant-rated CGI scores (range 1-7) [52-week followup only].
7. Doctor-rated CGI scores (range 1-7) [52-week followup only].
8. 6MWT walking distances (in meters) [baseline and 52-week followup].
If these results are not available, will Sense About Science and All Trials call on the PACE trial Principal Investigators to release them? If not, could you say why not, please?
Fourth, the PACE trial PIs continue to refuse to release data. They accept that requests made by some at least, such as James Coyne, are not vexatious. They allow that the data can be anonymized. They argue, however, that they are unable to make the data available because of the Data Protection Act (DPA). As I'm sure you are aware, if this interpretation of the DPA is established, then a precedent will have been set for all trials involving human subjects. Does Sense About Science agree with this interpretation of the DPA? Does Sense About Science support the trial PIs in their use of the Act to block sharing of data?
Fifth, in October last year you gave Michael Sharpe the opportunity to claim on your website that 'the benefit of these treatments is still apparent two years later'. Since then, the trial itself and the follow-up study on which Sharpe bases his claim have been criticized by many, including, for example, Rebecca Goldin, Trevor Butterworth, James Coyne and Keith Laws. Sharpe's column remains unchallenged. In order to make sense of the evidence and to safeguard sound science and in the interests of scientific inquiry, will you agree to carry an article by James Coyne addressing the flaws in the trial and the need for caution regarding Sharpe's conclusions?
Thank you for your help in this matter. I look forward to hearing from you,
Yours sincerely,
John Peters.
From: Paul Hardaker
Sent: 16 August 2016 15:19
Dear Mr Peters
Many thanks for your note. Apologies that no one has had a chance to come back to you yet. I’ve forwarded this on today, coming back from leave myself, so hopefully you will hear back shortly.
Kind regards
Paul Hardaker
as Chair of Sense about Science
On 24/08/2016 14:22, Julia Wilson wrote:
Dear Mr Peters
Thank you very much for your letter to Tracey Brown. She has just returned from holiday and has asked me to reply so that it is not delayed by her catching up. In the order of your numbered points:
We were very pleased that SaS USA, which has a specialist stats unit, took a look at the PACE trial. They kept us in the loop and we tweeted their findings as soon as they were available a few months ago.
Our twitter is followed by the media and many commentators. If you follow us you’ll have noticed that we used it for this and to retweet other similar stories that we think people should be aware of.
These are your words not ours. In relation to the call for patient level data, we asked for clarification about what was not available.
We explained, in response to questions linking the quest for patient level data to AllTrials, that AllTrials is not calling for patient level data, but for the CONSORT reporting items which are more meaningful (that’s the acronym for the agree list of 24 or so items that a trial should report). You can assess whether a trial is any good without individual patient data (hence sas usa’s review). That being said we don’t object at all to patient level data being shared and have applauded efforts to pool it, and we agree with comments that have been made elsewhere that claims about patient identification are often exaggerated or unnecessarily fearful. We asked via twitter whether the full CONSORT list had not been reported but we didn’t get a reply. We also tried to call the researchers at this point both with concerns about whether that they had reported according to CONSORT and to ask them whether there was some problem with sharing their data in response to the requests. We left phone messages but did not get a response.
We absolutely do not agree with the use of the data protection act in this kind of situation. We are pleased that the researchers haven’t succeeded with this interpretation of the data protection act. We have campaigned to establish European regulations on trial reporting, which included the liberal data protection interpretation that sits behind the court’s refusal to accept Queen Mary’s argument. We see this case as confirming that the work of us and others on this has been successful. I appreciate that you would have liked us to weigh in on this particular case, but a lot of people already were and we have had to focus our small resources on changing this law itself so that data protection is not used against trial transparency in this or any other case.
The For the Record response was published at an earlier stage. It was requested in response to the headlines which referred to illness as ‘all in the mind’. As a For the Record response it should have just responded to the misleading headline. It shouldn’t have included the researcher’s comment about the benefit of the treatments. That was not picked up before it was posted. I will add a clarification to the post to make this clear.
Thank you for taking the time to write to us about this and for your patience. There have been wrong statements on twitter about what SaS thinks on this subject and we gave up trying to correct them, but we are grateful to you for taking the trouble to find out.
Best wishes
Julia
Date: Sun, 28 Aug 2016 11:30:00 +0100
From: John Peters
Julia,
Thank you for your email and the reply to my questions. I understand not everyone is about in August.
I too welcome the opportunity to engage and think it is important that your views are not misrepresented. In that spirit, then, could you clarify your response to my first point, please? You say you tweeted about the stats.org findings, which I know is true, but you don't say: Do you accept the criticism by Goldin as valid? Do you agree with the editorial by Sense About Science USA? If not, could you say why not, please.
Second, I saw you sent one retweet and tweeted once about a 'long read' on PACE. As you know, PACE was a publicly funded trial which was described as definitive by the investigators. It has had massive media coverage. Claims have been made for the interventions based on the trial's reported findings. There are implications for NICE guidelines and for public policy, not just here but around the world. Hundreds of thousands of patients are affected in this country alone. The analysis has been described as flawed and its results doomed by stats.org and SASUSA. It would seem that something is wrong and that a fuss needs to be made. I don't think many people would consider a RT and a tweet to be a 'fuss' as is normally understood by the word, especially not in comparison, for example, with a dozen or so tweets on the misuse of weather terms (interesting though they were).
Your website says you welcome applications from the public, so would you please accept this email as a request to make a fuss about how wrong PACE is? You say you have only small resources, so I would welcome the chance to work with you. I would be happy to get an independent scientist, such as James Coyne, and the former science correspondent for 'The Washington Post', Brian Vastag, to help. I am sure they would draft a press briefing and a press release.
Third, I seem to recall Tracey Brown used those words in a tweet, though I can no longer find her account. Perhaps I am mistaken. You did however tweet me: 'Results and protocols are far more important than data. Do you have those yet?' Are you able to provide me with those results?
It's true that SASUSA has been able to criticize the analysis done in the PACE paper. However, what we don't know is what the data do actually say. Perhaps I could quote back at you some of your own tweets:
“We need ALL available info to make informed decisions about health care.” Please donate & help us get it http://bit.ly/1TxWmqY #AllTrials
“Hiding half the data is how magicians do coin tricks and shell games” - not trials of lifesaving drugs #AllTrials http://bit.ly/1TxWmqY
“With incomplete data we can only get an incomplete picture. #AllTrials need to be published” - donor Robert Virkus http://bit.ly/1TxWmqY
“Outcome switching is like choosing lottery numbers after watching the draw” http://bzfd.it/1PmIKBs #AllTrials
Since the Principal Investigators and others are making claims for the efficacy of the interventions based on their analysis, one which involved outcome switching, this would seem to me to be a classic case where patients should #AskforEvidence. We are doing so. Will you support us? Will you call for the data to be released so that an informed decision can be made?
Fourth, I confess I was surprised, given you accept the importance of the decision re the Data Protection Act, that you did not speak out against QMUL's attempt to extend the interpretation to cover properly anonymized trial data. In any case, QMUL are currently considering the ruling and have not yet decided whether to appeal. Will you call on them not to appeal, and to release the data?
Fifth, I note you have now added a comment saying the final paragraph of Sharpe's column should not have been posted. Nevertheless the claims by Sharpe remain. Since his paper has been criticized by independent scientists and is itself a follow-up of a trial analysis described as flawed by stats.org, and since you say that the comments should never have been included in the first place, could you say please why you have not simply removed the offending paragraph? Will you do so? If not, since the assertions remain, will you allow Professor Coyne to explain why Professor Sharpe's claims should be treated with caution?
I do think your campaigns for openness, accountability and integrity are very important and fully support them. I'm sure you would agree we should see that those principles are adhered to in all instances.
Thanks again for all your help,
John.
From: John Peters
Sent: 07 September 2016 12:09
Hi, Julia, I don't seem to have received a reply to my email. I would appreciate a response when you have a chance. Thanks, John.
On 09/09/2016 10:08, Julia Wilson wrote:
Hi John
Apologies, it’s incredibly busy here at the moment. I will get back to you as soon as I can.
Best wishes
Julia
Sent: 09 September 2016 12:19
OK, thanks, Julia. I look forward to your response. John.
On 19/09/2016 11:29, Julia Wilson wrote:
Dear John
Thank you for your further email. As you may have seen, now that I've had a chance to run it past a colleague, I’ve taken down the final paragraph of Sharpe’s quote.
Thank you for your offer but we wouldn't outsource press briefings and releases and I can't imagine many organisations would.
We mostly don't repeat what Sense about Science USA, STATS.org or SaS EU do. We'd like an international division of labour on issues. Our tweets also represent whether someone's in the office to do it and what they know about.
You asked whether we had the results and protocols. I think we were asking you the same. We found it hard to establish ourselves. Once we knew Rebecca was doing work on this, we left her to it and she did her usual clear analysis.
There are tens of thousands of clinical trials each year. Many, many of these are not reported even at a basic level. Historically it’s been just half or worse. It is good that some are challenged by patients. Most are not and we need to focus on the system, culture and regulation change that would address all trials.
Best wishes
Julia
Sent: 20 September 2016 11:38
Julia,
Thanks for your reply. Yes, I have seen that the final paragraph of Sharpe's briefing has now been removed.
First, we both agree we don't want the views of Sense About Science to be misrepresented, but once again you haven't said whether you accept the criticism by Goldin as valid or whether you agree with the editorial by Sense About Science USA. Am I to infer from your unwillingness to say so, that SaS do not? If that inference is inaccurate, please could you could say exactly what the position of SaS is.
Second, you do not say whether you will accept my request to make a fuss about PACE, which has been said to be wrong by STATS and SaS USA. Will you? If not, could you say why not, please? I do of course understand that you would not outsource briefings and releases. I meant only that we would be happy to help with the work, given your limited resources.
Third, no, you clearly stated the results were available. In any case, as I'm sure you are aware, QMUL are now releasing the data.
I agree it is good that patients took the initiative. Perhaps, though, I could express my disappointment and surprise that at no time did SaS help the patients, despite requests, including on Twitter, for you to do so. Not once did SaS call for the release of the data or support those asking for it.
As far as I'm aware, SaS have not explicitly welcomed the release. Does Sense About Science welcome the Tribunal decision? Will you be saying so publicly?
Thanks again,
John.
On 20/09/2016 17:11, Julia Wilson wrote:
Dear John
No, that is not the inference at all. We have always found Rebecca’s analyses of trials to be excellent.
As I said, we have the demanding goal of getting all clinical trials reported from the thousands of studies related to medicines in use today, hundreds of which are being lost and destroyed as I write. We are also engaged in setting standards of all evidence reporting in the UK and in European administrations, from transport to prisons, and working on evidence in energy, agriculture, forensics, nutrition and other subjects. I’m afraid we do not want to set aside any of those commitments and want to put all the time and resources possible into those. That does not mean other things are not important; many of the thousands of people who support AllTrials are, like you, pursuing particular trial information, on cancer, implants etc.
I’m delighted to hear that QMUL is releasing the data.
Best wishes
Julia
Sent: 21 September 2016 11:44
Julia,
Thank you for your reply.
You say: 'We have always found Rebecca’s analyses of trials to be excellent.' But yet again you do not answer my questions:
Do Sense About Science accept the criticism of PACE by Goldin, that ' the flaws in this design were enough to doom its results from the start', as valid?
Do SaS agree with the editorial which says 'the way PACE was designed and redesigned means it cannot provide reliable answers to the questions it asked'?
As I said, the clear inference of your refusal to answer these questions is that in each case you do not. Why else would you not reply?
You say you are delighted to hear that QMUL are releasing the data, but will you please state whether SaS welcome the decision by the Tribunal and whether you will be making a public statement to that effect.
I understand that SaS are busy, but I am disappointed to hear you will not help us in any way. You say on your website people should contact you to make a fuss about something when it is wrong. Stats.org and SaS USA say that a major trial which cost millions of pounds of public money, which has implications for hundreds of thousands of patients and which contradicts NICE guidelines, is wrong. I have contacted you to ask you to make a fuss. You have declined.
Could you say then, please, what criteria you use to determine whether you will intervene, and why PACE does not qualify.
Thanks again,
John.
On 21/09/2016 13:10, Julia Wilson wrote:
Dear John
I could not have been clearer in saying no that is not the inference at all. I cannot though add to Rebecca’s analysis - she is an expert in assessing the trial design, I am not. You should rely on her analysis.
I have also explained to you that our approach to trial reporting is to deal with the huge and urgent issue of non-reporting of thousands of trials, which amount to billions of dollars and hundreds of thousands of patients. Our decision is based on the urgent necessity of that.
Thanks
Julia
Date: Wed, 21 Sep 2016 14:25:04 +0100
Julia,
Thank you for your reply.
I note once again you do not answer my questions.
As I say, I don't want to misrepresent the views of Sense About Science.
Could you therefore confirm that the position of SaS is that you recognize Goldin's analyses of trials to be excellent, but you refuse to say whether you accept the criticism of PACE to be valid or not. You also refuse to say whether you agree with the editorial. Is that correct? If not, will you please give an unequivocal answer. If it is correct, could you say why you refuse to say, please.
Is it also correct that SaS refuse to say whether they welcome the decision by the Tribunal and will not be making a public statement welcoming it? Could you say why not, please?
On your website you ask people to contact you when something is wrong so that you can make a fuss. You now say you are too busy to do so. Is this temporary or permanent? Does this mean that you no longer want people to contact you when something is wrong? Will you be removing the request from the website? If you are still accepting some references, could you say why you accept some and not that for PACE, please?
Thanks,
John.
On 28/09/2016 11:43, John Peters wrote:
Hi, Julia, I'm still looking forward to your response. Thanks, John.
Date: Tue, 4 Oct 2016 12:09:07 +0100
From: John Peters
To: Julia Wilson
Hi, Julia, I wonder if I could get a reply to my questions, please. Thanks, John.
From: John Peters
Sent: 06 October 2016 14:10
To: Julia Wilson; Paul Hardaker
Julia,
Should I expect a reply? Or have you reverted to the discourteous and unprofessional behaviour of the summer and are now simply ignoring me again?
I think I've been very patient. I've been trying to get a proper response for almost four months. I'm not asking trick questions. I'm trying to get a clear answer on where SaS stand. The problem, though, is that you're in a trap of your own making. As you say, Rebecca Goldin's analysis is excellent. You cannot disavow it. But your prejudice on ME prevents you from accepting her conclusion as valid.
Since your foundation SaS have taken a position in support of ME as a psychogenic illness, treated by psychotherapy. This position has never been supported by evidence and is simply a product of bias. Now that respected, independent scientists in the USA have criticized the proponents of the position you support, you are stuck in a quandary.
I do genuinely support everything you say you stand for: openness, rigour, science-based policy. It's something I and other ME patients have been calling for for almost 30 years. It seems, though, that you do not. When we call for evidence, when we ask you to make a fuss about something which people you claim to respect say is wrong, you ignore us. Worse, you were prepared to allow without any criticism the PACE PIs to extend the scope of the Data Protection Act across all research involving humans. You betrayed everything you claim to support rather than face an inconvenient truth, that you have been wrong about ME all along.
You have damaged science.
Underlying this exchange is another question: do you, personally and as an institution, have integrity or not? It seems you do not.
John.
On 06/10/2016 15:20, Julia Wilson wrote:
Dear John
I had begun a response to you but not found the time to get back to it.
Regarding the STATS review, I really would not have advocated that you rely on an analysis and called it excellent if I or anyone else here had some other interpretation of it.
Sense about Science does not have a position on ME. We generally don’t take positions. Scientific knowledge evolves and changes all the time, so we work from the best available knowledge we have and we have no attachment to particular theories.
You also seem to be under the apprehension that we sit down at Sense about Science and decide a position on a trial. We don’t work like that. We haven’t had a position on the PACE trial. It was a trial, it turned out to be flawed in its design. We would like trials to be well designed but the specifics of whether that was the case are something that STATS and others specialise in, not us, and we point people to their analyses of trials just as we point people to good sources on nuclear power or prison reform. We applaud individual patient data (IPD) sharing because it looks like a good thing and if researchers and patients want it, I’m sure it is, but again, that is something that others specialise in, not us. AllTrials has insisted that IPD is a separate issue and should not hold up compliance with trial registration and reporting.
We will have to differ over an interpretation of helping and involving people. People coming in to develop positions and statements on their issues under our auspices, on subjects we are not working on, would require just the same time and attention as adopting an issue for ourselves and isn’t what we, or probably any organisation, mean by help or involvement. As I have explained, there are thousands of trials and among them many hundreds of people who have specific issues to raise, and the fact that this doesn’t fall under our remit is no comment on the merits or struggles of those people. I am sure there are thousands of flawed trials. Our focus regarding trials is to get them published - that is the CONSORT 24 items plus adverse event data - so that others can analyse, identify flaws and learn from them.
I can’t think of anything more that I can add on this subject so I hope that concludes our correspondence.
Best wishes
Julia
Date: Fri, 7 Oct 2016 11:44:36 +0100
From: John Peters
To: Julia Wilson
CC: Paul Hardaker
Julia,
Thank you for your email and your explanation.
I welcome your acceptance of Rebecca Goldin's criticism as valid and your acknowledgement that the design of the PACE trial was flawed. I'm not quite sure why you didn't say so at the outset.
I understand your position on involving others. It was just a suggestion since you had said you had limited resources.
I would say I remain disappointed:
— You allowed Michael Sharpe to use your website, contrary to your editorial policy, to promote his interpretation of his study of a trial you agree was flawed.
— You rejected requests to support patients when they were asking for evidence and calling for the release of the trial data.
— You refused to speak out against QMUL's attempts to extend the Data Protection Act.
— You still have not welcomed the Tribunal decision.
— You will not make even the slightest fuss about PACE because you are 'too busy' despite your stated aim to make a fuss when something is wrong and the importance of the trial, including for NICE guidelines.
I am sure you will disagree, but for me, and I know for others, this gives the impression you have different standards when it comes to ME generally and PACE in particular.
I recognize I have taken up a lot of your time and you would prefer to leave matters there.
Thank you for your help,
John.