The Duke "deception": what should be done?

There is no cause more dear to me than the advancement of Pompe research. It is heartbreaking, plain heartbreaking, to see that cause tarnished in the way that it has been by the Duke paper. Science is about truth and evidence. There is very little room for short cuts and certainly none for spin.

The 2001 Duke paper in Genetics in Medicine stands exposed as containing false information. It is now clear that this was a deliberate falsehood, for which an inadequate explanation has been given.

Worse, that deliberate falsehood was used in the paper to argue against the work of another Pompe researchers, who used enzyme manufactured using a different method.

There is an established course of action in such cases: the paper must be withdrawn. It would be best if the authors, recognising the enormity of their error, asked for this to be done themselves. Failing that, the editor of Genetics in Medicine should do so to protect the integrity of the journal.

However, that is not the end of the matter. As we know, different companies were involved at this stage and the production method of alpha-glucosidase was itself a contentious subject. It is no exaggeration to say that fortunes were made or lost on that decision, on which the Duke paper may have had a bearing.

My questions now are:

Was anyone at Genzyme aware of the incorrect data reported in the Duke paper?

Was anyone within the senior management at Duke University aware that data was being misreported in this way?

Time to get digging, I guess!

The Duke response

In the 9 August  advance online publication section of Genetics in Medicine, two letters appear . One is from Arnold Reuser  Inconsistent reporting about dosing, dosing regimen, and immunomodulation therapy in Pompe disease and a response from Andy Amalfitano, Priya Kishnani and YT Chen: Response to Reuser.

Once again, these are behind a paywall but let me precis.

The letter from Arnold Reuser makes the observation that the dosage data reported in the 2001 genetics in Medicine  paper on the Duke ERT trial is inconsistent with that reported in the 2012 paper from Banugaria et al as described in the last few blog entries. For an editor to publish this means that they consider that there is a case to answer. As indeed there is.

Now, I have been awaiting the publication of the Duke response with some impatience, hoping that, somehow, there would be a reasonable explanation that would restore my belief in the integrity of researchers that I had admired.

Unfortunately, it does not. Here is what it says.

Dr Reuser is correct in noting that in our original report,
we had not disclosed that in latter portions of the first clinical
trial for patients 1 and 2 (who had declining AIMS scores and
increasing antibody titers), that we attempted to treat these
patients with increased doses of the enzyme,...

Going on to explain that:

These attempts were ongoing during submission and review of our original
manuscript, and were well beyond the scope of that manuscript,
requiring further more detailed review and reporting as noted in subsequent publications

And finally concluding that their statement
:

“Three patients with
infantile Pompe disease have been receiving twice-weekly
intravenous infusions of rhGAA for 14 to 17 months,”

 Might be better changed to:

Three patients with infantile Pompe disease have been receiving
at least twice-weekly intravenous infusions of rhGAA for
14–17 months

And that actually appears as a correction to the 2001 paper in the current issue of genetics in Medicine. This is actually pretty jaw-dropping. Let me explain why.

Firstly, the implication that the initial 5mg/kg twice weekly dose was changed so late in the course of the trial that it was just too complicated to include. Hardly worth mentioning, even. Well, the reported period was 14 -17 months. The dosage was changed after 20 weeks for Patient 1 and after 15 weeks for Patient 2. So for two thirds of the trial, the dose was much higher. The explanation given by Duke simply does not wash its face.

Secondly, there is that section in the 2001 Duke paper still uncorrected:

Similar cardiac improvements in four infants treated with rhGAA from rabbit milk for 9 months have recently been reported¹⁵; however, the doses of rhGAA from rabbit milk necessary for the therapeutic effects were 4 times higher than the present study of rhGAA from CHO cells.

 The conclusion is pretty clear. Incorrect dosage rates were deliberately reported in the 2001 paper, in order to put forward an argument against a rival group's work - a group which used enzyme manufactured via a different production method. This is deception, pure and simple, and has no place in science.

Duke author admits discrepancy

A bit of an update.

Firstly, a letter from Dr Arnold Reuser pointing out the discrepancy in the data has been published in Molecular Genetics and Metabolism, as a 'Letter to the Editor':
Enzyme therapy in Pompe disease: questions remain

This was published online (link above) but unfortunately is behind their paywall, if you don't have a ScienceDirect subscription. The key sentence is:

While Drs Y.-T. Chen, P.S. Kishnani and A. Amalfitano are authors
on both publications, the information contained in Banugaria et al.,
Mol. Genet. Metab. 2012;105:677–680 and Amalfitano et al., Genet.
Med. 2001;3:132–138 is mutually inconsistent.

This has been followed by a response from Dr Kishnani at the other authors of the 2012 paper. Again, it is a Letter to the Editor of Molecular Genetics and Metabolism. And again it is behind a paywall - sorry.  However, the key part is:

The details of the ERT dosing and immunomodulatory regimens
used in the management of this patient are accurate as described in
our recent article in Molecular Genetics and Metabolism [2]. The details
of the increased rhGAA dosing and immunomodulation were
not included in the 2001 Genetics in Medicine paper for reasons outlined
in a recently accepted letter to the editor submitted in Genetics
in Medicine by Drs. Amalfitano, Kishnani and Chen [3].

So it is accepted that the 2001 data were incorrect and that the 2012 data are correct. They go on to say:

We regret any confusion this may have caused.

So, we await the publication of the letter in genetics in Medicine which will explain why the data was withheld in 2001. I'm struggling to see what could be in this letter than will somehow make things OK. For the record though, I would be absolutely delighted to be surprised.

Why does the Duke error matter?

Right. As you know, I like to keep things as objective as possible. So I give fair warning that following one more excursion into the land of facts and evidence, I will give my subjective opinion.

First, more evidence. Let's return to that 2001 paper. As I've shown, it repeatedly describes the dosage regime used in the trial as 5mg/kg. Now, as you know, an earlier trial in Rotterdam had already been published in The Lancet in 2000. Here's how the Duke paper refers to the Rotterdam trial:

Similar cardiac improvements in four infants treated with rhGAA from rabbit milk for 9 months have recently been reported¹⁵; however, the doses of rhGAA from rabbit milk necessary for the therapeutic effects were 4 times higher than the present study of rhGAA from CHO cells.

To be clear, the two trials each used enzyme produced by a different method. Rotterdam used enzyme produced in the milk of transgenic animals, whereas Duke used enzyme produced by CHO cells in culture.  The Duke paper is explicitly stating above that their method was better because the dosage used was lower.

In other words they were making an argument based on dosage data that they presumably knew to be false.

And now the subjective part:

This wasn't a case of careless error-checking. It's hard to avoid the conclusion that this was a deliberate omission of data, designed to support the argument that a specific product required a lower dose - an argument now shown to be false.

As a scientist, I find this absolutely shocking. It is a violation of the standards of openness and transparency on which good science relies.

The question is why this occurred. It's probably worth bearing in mind that, once ERT had been shown to work, there was obviously money to be made. And where that money went would depend on the method chosen to produce the enzyme. As you know, the CHO cell method described in the 2001 paper was the production method chosen.

There's also the fact that credit for ERT for Pompe disease is a big thing - perhaps having published last, the temptation to try to gain priority over the previous publication was too much to resist.

Or perhaps, in some way and for some as yet unknown reason, the authors managed to convince themselves that it was OK to withhold the data in the service of some greater good. A human failing - politicians do it all the time - but not one allowable in science.

I have some thoughts on what should happen next but time for a breather.

What is wrong with the Duke publication?

As regular readers will know, I'm all for keeping things simple. However this is serious and in order to explain why it is serious I'm afraid that there is no option but to delve into the detail. You might want a coffee beside you at this point. OK? Right, here we go.

First stop, 2001 in that original Duke paper in Genetics in Medicine. This paper reported the results on the 3 patients in the Duke ERT trial. Now, the actual end results - the evidence that ERT works - are not in question. The problem lies in the 'dusty end' of the paper, the Materials and Methods. Here the enzyme dosage given to the patients is described as 5mg/kg, twice weekly. Under Results, we are told that the enzyme was given for 14-17 months and again the twice-weekly 5mg/kg dosage is mentioned. There is one further reference to the dosage that I'll come to later, however the message is clear: the paper describes the results of 3 patients treated with a dosage regime of 5mg/kg, twice-weekly.

On to 2004, and the publication of a paper by Hunley et al:

Nephrotic Syndrome Complicating α-Glucosidase Replacement Therapy for Pompe Disease

Pediatrics Vol. 114 No. 4 October 1, 2004

(Full text available free online)

This shares 3 authors with the 2001 paper, Drs Amalfitano, Chen and Kishnani and reports the case of a child on ERT who developed complications. However, the dosage rate is rather different from that described in the 2001 paper. This paper describes a dosage of 5mg/kg, twice-weekly (20mg every 2 weeks), for 15 weeks, followed by 10mg/kg ,twice-weekly (40mg/kg every 2 weeks)for 27 weeks, followed by 5x10mg/kg weekly (100mg per 2 weeks)for 11 weeks.

However, at no point in this paper is it explicitly stated that this patient was one of the three described in the 2001 results. So we might give Duke the benefit of the doubt (at this point, I don't think I do).

Fast forward to 2012 and the following publication from Banugaria et al:

Persistence of high sustained antibodies to enzyme replacement therapy despite extensive immunomodulatory therapy in an infant with Pompe disease: Need for agents to target antibody-secreting plasma cells

Molecular Genetics and Metabolism

Volume 105, Issue 4, April 2012, Pages 677–680

(abstract only available free)

This papers shares 3 authors with the 2001 paper, Drs Amalfitano, Chen and Kishnani, and it refers back explicitly to one patient in that study, however the results reported are rather different. Rather than a dosage of 5mg/kg for 17 months, this paper tells us that patient 1 received:  5mg/kg twice weekly (20mg every 2 weeks)for only the first 5 months, followed by 10mg/kg twice-weekly (40mg every 2 weeks) for 8 months and then 5 X 10mg/kg weekly (100mg every 2 weeks) for 4 months.

It is fair to say then that the 2001 paper which purported to describe a treatment regime of 5/mg/kg, twice-weekly, does no such thing. On the evidence of the 2012 paper, at least one of those patients had a much higher dosage for most of the study.

Apologies once again for all the detail. The point is to allow you to check what I've said for yourself.

Now at this point you may be thinking "OK, so the 2001 paper had wrong information in it. A simple error that any of us could have made - does it matter?"  Yes, it does matter. It matters a great deal. And I shall explain why in my next post.

One last twist in the tale

Well, here's a story that I wasn't expecting to have to write. If you've followed the Pompe story through this blog (and if not, why not?), then you'll know that there have been occasional patches of...murkiness. Times when things have happened that the reader may have had to stretch to see in a good, or even neutral, light.

Nearly all of those have been concerned with commercial interests and, well, I think you kind of expect it from that quarter, where the bottom line reigns supreme. At least, I thought, the same could not be said of the world of academic science. A world where truth and objectivity were tirelessly in the service of the scientific ideal. You know, the good guys.

Turns out, I was wrong. There are significant errors - errors which are difficult to explain - in the original publication of the ERT trial results from Duke University, one of the most important publications in the development of ERT.

The publication in question is:

Recombinant human acid a-glucosidase enzyme
therapy for infantile glycogen storage disease
type II: Results of a phase I/II clinical trial

Amalfitano et al Genetics in Medicine, 2001:3(2):132–138.

The full text is available online.

Over the next few posts, I will spell out exactly what the issue is, why it matters and what I think should be done.