Square One

Rather curiously, as I was going to give a Work-in-Progress presentation on a Systematic Review and Meta-Analysis of prognosis studies in epilepsy, I looked at this week’s edition of the BMJ, a regular weekly practice, and in it were three articles discussing the quality of prognosis studies, with broader implications for observational research generally. The first, Ten steps towards improving prognosis research (free full text) by seasoned researchers of prognosis – Harry Hemingway, Richard Riley and Doug Altman – does just what its title says in what Richard Lehman described on his blog as “rather angry in tone, but not angry enough for my taste.”

In a linked editorial, Sørensen and Rothman in their interestingly titled The prognosis for research disagree with Hemingway et al on their suggestion that there should be a register for prognosis studies, and observational research generally as it is presently required by law for clinical trials:

We suspect that historians and philosophers of science would recoil at the notion that advance registration of all scientific studies in a publicly accessible database would produce better science. How much room would this policy leave for exploration, serendipity, or pursuit of unpopular theories?

If the rules precluded easy registration, that might create an undesirable drag on the end of the research spectrum that constitutes the quirky, brilliant work that is not enterprise driven. Moreover, registration would not prevent publication bias among the many studies conducted with secondary data, because researchers could still selectively register study ideas after the data have been explored.

They also disagree with Hemingway et al on their suggestion of developing guidelines for reporting prognosis research:

Reporting guidelines do have advantages, but the disadvantages are generally overlooked. On the positive side, guidelines increase uniformity and can improve the average quality of reporting. But guidelines also promote rigidity and can enshrine misconceptions, because they are merely compiled from the consensus of a few opinion leaders and form a common denominator of current beliefs. If all science throughout human history had been filtered through reporting guidelines, we suspect we would live in a very different world, one in which the science had lagged far behind what actually has been achieved.

They end their article by placing the responsibility and the blame for the quality of prognosis research at the feet of journal editors:

Consider the crucial role of the gatekeepers of published research. Any published research, including the low quality work … has survived the scrutiny of peer reviewers and of the ultimate gatekeepers, journal editors. Perhaps the priority should be continuing education efforts focused on journal editors.

Then comes the third article this week on the same topic as BMJ editors Elizabeth Loder, Trish Groves, and Domhnall MacAuley respond to Sørensen and Rothman in another editorial, Registration of observational studies. They defend the need for protocol driven observational studies:

At present, consumers of observational research cannot easily distinguish hypothesis driven studies from exploratory, post hoc data analyses. Researchers do not routinely disclose the number of additional analyses performed. Nor is there any satisfactory way to know whether the research questions or methods of statistical analysis diverged from those initially planned.

We agree that exploratory observational research is important. Many new ideas arise from unexpected findings in observational research, and many researchers learn their skills from examining available datasets. However, that is not the sort of research the BMJ usually aims to publish…

I thought that was a rather weak argument though, and while they are quiet about the need to train journal editors, they go ahead to state a series of not necessarily insurmountable hurdles to get your observational study published in the BMJ:

We will now ask authors of papers reporting observational studies submitted to the BMJ to tell us more about the origins, motivations, and data interrogation methods of that work.

We will be asking authors to report in their papers a clear statement of whether the study hypothesis arose before or after inspection of the data…

We will ask to see study protocols if they exist; and we will add to the papers’ abstracts their registration details, if they have been registered…

The Systematic Review and Meta-Analysis I’m presently conducting though shows that apart from the thorny issue of inconsistent definitions of disease state and classification, and the conceptualisation of outcome measures – which none of these hurdles addresses or could possibly address – there seems to be relatively better quality of prognosis research at least in epilepsy than these articles generally suggest, which may at least partly be due to the fact that clinical trials in epilepsy are rather tricky.

I still remember the shock when I realised that the practice of homeopathy was funded within the National Health Service (NHS) in England. I was taking a lunchtime stroll when I came across the Royal London Homeopathic Hospital, which as it turned out was one of the hospitals within the UCL Hospitals NHS trust, which also included the hospital to which the Institute of Neurology where I was studying at the time was affiliated. I almost felt ashamed.

It was particularly shocking because the United Kingdom does have a reputation for science and rationality that is hardly equalled elsewhere in the world. The experience however made it less shocking for me when I got to Sydney, Australia and found lots and lots of alternative practices, with all sorts of interesting and grand names, structured to deceive and confuse with legitimate science. However, I doubt that like in the UK, any is funded from the public purse of Australians.

Two weeks ago, I met a black woman from Jamaica (although born in London and raised in Auckland). She was the first black person I walked up to and spoke with in Sydney, and the first and only person ever to have referred to me as ‘brother’ in the restricted black folk sense of the word. It was rather strange. I had never been a ‘brother’. There are not many in Australia.

We got on the bus together. She had studied ‘alternative/complimentary medicine’ in her youth and had also received training in homeopathy. Much of the rest of the trip was spent discussing the claims of homeopathy, during which she argued that homeopathy was much like vaccination. I was so mad, but out of courtesy I had to change the topic immediately. It reminded me of Raymond Tallis‘ 2007 Sense about Science annual lecture where he said:

…and this is how it is with junk science that borrows the terminology of science, without any sense of its true meaning, and of the massive interconnected hinterland of facts and concepts and even uncertainties behind them.

…and so we have treatments such as ‘reflexology’ which expropriates a well-established, indeed central, concept in biological science, and uses it to label treatments that have no biological foundation whatsoever.

…and ‘homoeopathy’ which, being in Greek, one of the languages of science, sounds very scientific but is based on magic thinking that would shame a six year old child.

…they domesticate terms by uprooting them from a complicated nexus of hard-won concepts.

Whenever I see those Sydney shops or offices, what comes to mind is how successful alternative medicine practice/movement is in Nigeria as well, and how they feed on pretty much the same sentiments. I reckon it must indeed be a universal phenomenon. Thankfully, we are not yet at the stage where the Nigerian government will fund an alternative medical practice, but I bet we are not that far either. I am almost certain of it that the movement will soon have a ‘scientific’ arm made up of people trained as much as to be able to throw terminologies around but not quite as much as to have any deep, nuanced understanding of them.

Here is an excerpt from an advertorial on the popular Doctor Akintunde Ayeni of Yem-Kem International Nigeria Limited:

…he [has] invested resources – time and money to visit renowned herbal homes in India, China, Australia, Japan and Pakistan. In similar vein, [practitioners] of alternative therapy in those countries visit him, here in Nigeria, to exchange notes. The result of these research efforts is manifested in the emergence of our three products namely (1) Blood Cure, which a blood purifier and immune boosting herbal medicine (2) M & T Capsule which is an effective herbal medicine for all chronic fever and (3) Energy 2000 which is a powerful herbal medicine for sex ability deficient patients.

The words again: ‘research’, ‘immune’, ‘capsule’, ‘doctor’, ‘discover’, et cetera. It is also interesting that Australia has its place among the visibly oriental countries that Akintunde Ayeni has visited.

John Diamond, who before his eventual death had his hopes of cure from cancer falsely raised by several alternative medicine practitioners did put what would be my summation very beautifully: “There is in reality no such thing as alternative medicine, just medicine that works and medicine that doesn’t…There isn’t an ‘alternative’ physiology or anatomy or nervous system any more than there’s an alternative map of London which lets you get to Battersea from Chelsea without crossing the Thames.”

So how do you define medicine that works? Well, the same way that Artemisinin made its way from the fields of central China to clinics everywhere chloroquine resistant Malaria is treated.

Jo Stephenson writes in this week’s BMJ about different approaches to equipping doctors for management and leadership roles. This is how we need to start thinking in Nigeria now. We cannot continue to leave the management of our hospitals and health care institutions and system in the hands of those who are not trained to manage them, no matter how senior or clinically experienced.

I sent this article to Dayo Osholowu, a sports physician in Nigeria who replied by saying: “We have to embrace the responsibility that is falling on us or someone else will. The ownership and management of health service insitutions will fall on an elite corp of financially sophisticated management experts. Lets join this corp with open arms and the venture will reward the few.”

He went ahead to paraphrase a quote from Dr Strangeglove, where General Jack D. Ripper says: “Clemenceau (himself a medical doctor who later became French PM) once said that war is too important to be left to the generals. When he said that, 50 years ago, he may have been right, but now, war is too important to be left to the politicians. They have neither the time, the training, nor the inclination for strategic thought.”

Health care management indeed in the same vein, is too important to be left to doctors/clincians. Here are excerpts from the BMJ article:

However, many people would like to see management skills introduced earlier in doctors’ training. In America there are a growing number of dual MD/MBA courses on offer. Harvard set out to create a truly integrated programme, explains Richard Bohmer, medical professor at Harvard Business School.

Students concentrate on medicine for the first two years but do a management internship in a hospital or healthcare firm between the two years. In their third year they continue medical training as well as doing several management courses, including looking at management problems on the wards.

“Students are getting clinical experience. At the same time we’re teaching them about some of the organisational issues they’re seeing,” says Professor Bohmer.
During the fourth year they do an MBA with a clinical rotation and in the fifth and final year they complete their MBA and clinical rotations.

………………..
Meanwhile other US medical schools are introducing leadership elements into graduate medical education, such as Dartmouth Medical School’s leadership preventive medicine residency. It is a two year programme on top of a traditional three year clinical residency, offering the chance to earn a masters degree in public health and gain skills to lead change and improvement in health care.

The programme includes clinical leadership rotations and a major practical assignment in which trainees are expected to apply what they have learnt to a real health service situation.

………………..
Many doctors combine management with clinical work, but in Australia and New Zealand it is possible to be a registered specialist in medical administration by becoming a fellow of the Royal Australasian College of Medical Administrators. Doctors with three years’ postgraduate clinical experience can go straight into a three year training programme providing supervised medical management experience in posts such as deputy medical director.

Alternatively, senior doctors trained in other specialties can gain fellowships in a minimum of 18 months depending on previous experience.
This approach has boosted the status of medical managers. Fellows can command higher salaries, and in some states fellowship is a requirement for directors of medical services.

“There is that feeling that a doctor who has gone into administration has gone over to the dark side. Making it a recognised vocational skill certainly goes a long way to dispelling that myth,” says college president David Rankin

The full article is here.