“I Just Can’t Get Enough”: Beliefs vs. Good Evidence in Public Health?
By Rida Bilgrami and Fiorenzo Conte
Assumptions vs. Good Evidence. Imagine that you are a policymaker and you are faced with the challenge of designing public health intervention for HIV prevention based on lessons from other countries. You are told that failure to adequately address public health challenges in other countries is often due to policies based on beliefs and assumptions rather than on good evidence. Therefore you are determined to base your policies on good evidence and if this evidence is not sufficient then you will commission more studies. Once you take this decision, you run into the first problem: what is good evidence? In other words what are the scientific methods one should consider and one what are the bad methods one should discard because they produce bad evidence?
What is Good Evidence? You are told that in recent years, randomized control trials (RCT) have gained increasing importance and popularity for evaluating public health interventions. By comparing the outcomes between a group which receives the intervention and a group which does not (the control group) RCTs are able to isolate the impact of the intervention (e.g. male circumcision) on the intended outcome (e.g. HIV incidence). If RCTs have very strong internal validity (i.e. show a causal link between intervention and outcome) they do not prove that the findings of the trial can be replicated in another context. Furthermore, RCTs are very costly therefore many argue that it is not feasible to replicate RCTs in order to make the findings more robust across different contexts and regions. On the basis of such limitations, others have argued that good evidence is actually tantamount to good science, i.e. observational studies and clinical trial. When you look at what scientific methods produce the good evidence it turns out that each of these methods to generate evidence has its pros and cons and therefore one can hardly say what is the best evidence. You conclude that a mix of approaches is needed and that there is not just one good or best evidence. At this point you run into a second problem: different forms of good evidence can tell something different about the same intervention.
There Is Never Enough Evidence. Imagine you are evaluating the possibility of making male circumcision mandatory for men in your country. Based on the lessons learnt you are not looking for best evidence but you are looking for the different evidence produced by different methods. You find one observational study which presents some evidence that male circumcision was associated with reduced risk of HIV infection in female sexual partners; this finding is substantiated by an RCT which found a statistically significant impact between male circumcision and HIV incidence. Later on you discover that this significance is disputed by another RCT which found that circumcision of HIV infected man did not reduce HIV incidence in their female partners. Now you have a problem: the evidence gives you contrasting messages. Some of your advisors make the case that one needs to wait for more evidence. However you are pushed to take action and waiting for more evidence is however not feasible: inaction due to lack of evidence is just a no-solution. You are left therefore with making a choice based on limited evidence. You understand that basing policies exclusively on evidence is not an approach because one does not have best evidence, evidence are not definitive and therefore do not offer blueprint for action, they can be contested and sometimes they are contradictory. Since it seems that there is never enough evidence available, you then decide to revaluate the importance of basing policy decision on local beliefs (as much as on evidence) and you bump into the case of Botswana.
Beliefs. In its efforts to combat the HIV pandemic, Botswana looked towards international experts to devise the best strategy to contain HIV transmission: get people to use more condoms, Botswana was told, as there is enough good evidence which proves a link between condom use and reduction in HIV incidence. Policymakers in Botswana were therefore basing their decisions on good evidence: they ignored the beliefs. Yet this move proved fatal. The condom message failed to succeed because they didn’t take into account Tswana beliefs that conceptualize condoms as not preventing infection but as an agent in the spread and origin of the disease . Furthermore, the condom use message caters to a Western population that views sex as recreational but it was not a culturally appropriate message for communities in Botswana where sex is viewed as a procreational activity and procreation is highly valued. Beliefs, it turns out from the Botswana case, are as important as evidence.
It is problematic to think of evidence in terms of whether it is ‘good or ‘best’’: each method generating evidence has its pros and cons and none will produce the best evidence that cannot be disputed. From this perspective, the quest to base policy decisions exclusively on good evidence might turn elusive and counterproductive. In the real world policy makers often design policies on the basis of a limited amount of evidence, which are at times contradictory. Policy decisions are in sum to be based on the limited evidence available while paying attention to local beliefs and assumptions.
 This post readapts the ideas and arguments developed by Rida Bilgrami in one of the papers she wrote during her Masters at LSE.
 Potts, M et al. 2006. The parachute approach to evidence based medicine BMJ 333: 701-703
 Caldwell, John C. & Pat Caldwell. 1996. The African AIDS Epidemic. Scientific American. 174(3): 40–46.
 Gray R, Kigozi G, Serwadda D, Makumbi F, Wayta S, Nalugoda F, Kiwanuka N, Moulton L, Chaudhary M, Chen M, Sewankambo N, Wabwire-Mangen F, Bacon M, Williams C, Opendi P, Reynolds S, Laeyendecker O, Quinn T, Wawer MJ. 2007. Male circumcision for HIV prevention in men in Rakai, Uganda: a randomized trial. The Lancet, 369: 657-666.
 Wawer, Maria J, F Makumbi, G Kigozi, D Serwadda, S Watya,F Nalugoda, D Buwembo, V Ssempijja, N Kiwanuka, LH. Moulton, NK Sewankambo, SJ Reynolds, TC Quinn; R Ridzon, P Opendi,2 B Iga, and R H Gray. Trial of Male Circumcision in HIV-infected Men and its effects on HIV Transmission to female partners in Rakai, Uganda: a randomized controlled trial. The Lancet. 374: 229-37
 Heald, Suzette. 2006. Abstain or Die: the development of HIV/AIDS policy in Botswana. Journal of Biosocial Science, 38: 29-41.