advanced search

<< Back To Home

NEW THIS TUESDAY: MEASURING VACCINE CONFIDENCE

Monday, 20th of May 2013

• MEASURING VACCINE CONFIDENCE: ANALYSIS OF DATA OBTAINED BY A MEDIA SURVEILLANCE SYSTEM USED TO ANALYSE PUBLIC CONCERNS ABOUT VACCINES 

The Lancet Infectious Diseases, Early Online Publication, 13 May 2013

Full text is best viewed at http://www.thelancet.com/journals/laninf/article/PIIS1473-3099%2813%2970108-7/fulltext

Dr Heidi J Larson PhD a b   , David MD Smith DPhil a, Pauline Paterson PhD a, Melissa Cumming MS c, Elisabeth Eckersberger MPA a, Clark C Freifeld MS d e, Isaac Ghinai BSc a, Caitlin Jarrett MPH a, Louisa Paushter MPH c, John S Brownstein PhD d f, Prof Lawrence C Madoff MD c g

Summary

Background

The intensity, spread, and effects of public opinion about vaccines are growing as new modes of communication speed up information sharing, contributing to vaccine hesitancy, refusals, and disease outbreaks. We aimed to develop a new application of existing surveillance systems to detect and characterise early signs of vaccine issues. We also aimed to develop a typology of concerns and a way to assess the priority of each concern.

Methods

Following preliminary research by The Vaccine Confidence Project, media reports (eg, online articles, blogs, government reports) were obtained using the HealthMap automated data collection system, adapted to monitor online reports about vaccines, vaccination programmes, and vaccine-preventable diseases. Any reports that did not meet the inclusion criteria—any reference to a human vaccine or vaccination campaign or programme that was accessible online—were removed from analysis. Reports were manually analysed for content and categorised by concerns, vaccine, disease, location, and source of report, and overall positive or negative sentiment towards vaccines. They were then given a priority level depending on the seriousness of the reported event and time of event occurrence. We used descriptive statistics to analyse the data collected during a period of 1 year, after refinements to the search terms and processes had been made.

Findings

We analysed data from 10 380 reports (from 144 countries) obtained between May 1, 2011, and April 30, 2012. 7171 (69%) contained positive or neutral content and 3209 (31%) contained negative content. Of the negative reports, 1977 (24%) were associated with impacts on vaccine programmes and disease outbreaks; 1726 (21%) with beliefs, awareness, and perceptions; 1371 (16%) with vaccine safety; and 1336 (16%) with vaccine delivery programmes. We were able to disaggregate the data by country and vaccine type, and monitor evolution of events over time and location in specific regions where vaccine concerns were high.

Interpretation

Real-time monitoring and analysis of vaccine concerns over time and location could help immunisation programmes to tailor more effective and timely strategies to address specific public concerns.

Funding

Bill & Melinda Gates Foundation.

Introduction

Although immunisation has successfully reduced the global burden of illness and death, a range of concerns have converged to affect public confidence in vaccines. When confidence in vaccination breaks down, hesitancy can lead to delays and refusal, disrupting research and delivery programmes, and sometimes leading to disease outbreaks.1, 2

The most serious example is the 2003—04 northern Nigeria boycott of polio vaccination, which set the global polio eradication initiative back substantially, cost millions of US dollars, and led to a resurgence of the disease.3—5 The boycott, driven mainly by politics and unfounded fears of vaccine-induced sterilisation, contributed to reinfection in 20 previously polio-free countries, reaching as far as Indonesia. The fundamental breakdown in public trust still affects polio eradication efforts in Nigeria.6, 7

Another example includes fear and refusal of the measles, mumps, and rubella (MMR) vaccine, initially ignited by Andrew Wakefield and colleagues now-retracted8, 9 1998 study10 suggesting an association between the vaccine and autism. The studys findings were amplified by the media, Wakefields own public appearances advocating his research, and networks of parents who felt that Wakefield finally had an answer to the cause of their childs autism. The effect of the media and personal attention to the since-debunked research, officially retracted 4 years after it was published and deemed both unethical and fraudulent,11 resulted in a substantial decline in MMR vaccine coverage12 that has still not returned to the high of 92% before 1998.13 Meanwhile, MMR vaccine anxieties continue to circulate worldwide. In 2009—10, high non-acceptance rates of the pandemic influenza A H1N1 vaccine, including among health-care professionals, were another example of the potential effects of public distrust in vaccines.14

In such cases, the time between the prompting events and their effect on public health outcomes is important—eg, months or years can elapse, with extended periods of vaccine hesitancy or uncertainty. Various factors amplify the spread of information, and misinformation, affecting perceptions and behaviours and creating what Kasperson and colleagues15 term the “social amplification of risk”, by which they mean amplification of the spread of information via social, cultural, and institutional processes, and amplification of societys response.

We postulated that media monitoring might provide important information about perceived problems with specific vaccines or immunisation programmes that might take longer to register through official channels. We therefore aimed to track the emergence and spread, geographically and temporally, of media and social media reports on vaccines by developing a new application for rumour surveillance systems—typically designed to detect signs of disease outbreaks16—20—to detect and characterise early signs of public concern about vaccines. In doing so, we aimed to characterise and create a typology of the content and to develop an approach to assess the priority of each concern for further investigation and intervention.

In January, 2010, The Project to Monitor Public Confidence in Immunisation, now called The Vaccine Confidence Project, of which we are members, was launched to address three objectives. The first being to establish a global information surveillance system to detect emerging public concerns by monitoring media and social media and building a global key informant network. Second, to systematically code all reports to identify positive or neutral versus negative content and key areas of concern, to develop a typology of concerns. And third, to develop a diagnostic method to prioritise which reports need further investigation and intervention on the basis of patterns of clustering or persistence of reports, and when viewed against contextual and historical factors that have contributed to their amplification. This report focuses mainly on our first two objectives. Concurrent research and analysis of the third objective will be reported elsewhere.

Methods

Data collection

Early on in the project, an international advisory group21—consisting of experts in vaccine-preventable diseases, vaccine safety, risk and decision science, immunisation programme management, and public health—identified and discussed key examples of breakdowns in public confidence in vaccines or vaccine programmes that resulted in serious declines in vaccine acceptance. Detailed retrospective analyses then identified crucial factors that led to these breakdowns in acceptance, which were then used as criteria for report categorisation.

Data collection began in April, 2010. Refinements to search terms, collection, and coding were made periodically between April, 2010, and April, 2011. Most reports were collected with the HealthMap automated data collection system,22 which was adapted to monitor online reports about vaccines, vaccination programmes, and vaccine-preventable diseases. An example of the search criteria used is “intitle:vaccine OR intitle:rotavirus OR intitle:measles”. Reports included online news articles, blogs, website pages, public service announcements, government announcements, book reviews, and broadcast media. The three primary acquisition channels were Google News, Google Blog Search, and Moreover Public Health, a global news aggregator service. The Moreover content was culled from a broad range of RSS (rich site summary) news feeds by use of health-related keywords determined by Moreover. Foreign language reports were translated using Google Translate. The non-English languages that were translated were Chinese, Czech, Danish, French, German, Greek, Hindu, Indonesian, Italian, Japanese, Persian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, and Vietnamese.

Reports were passed through an exclusion filter to remove any that were not vaccine-related or were related only to animal vaccines. Any report that included a reference to a human vaccine or vaccination campaign or programme that was accessible online was included in the analysis, which included articles about vaccine-preventable diseases whenever a vaccine was mentioned. Reports not meeting these inclusion criteria were excluded from analysis but logged and archived. The articles were presented on a web-based user interface system for rapid review, while the system automatically forwarded the reports to analysts from the London School of Hygiene & Tropical Medicine and the Program for Monitoring Emerging Diseases (ProMED).

Data categorisation

Reports were automatically classified according to the HealthMap algorithms every hour. Relevant reports were classified by location and disease and entered into the project database, with title, date, report source, URL, and plain text in full automatically added, taking less than 1 min per report. The entry date and the team member entering the report were also automatically recorded. Analysts then reviewed and amended the autopopulated fields, then allocated data categories, report source, vaccine-preventable disease, vaccine, and vaccine manufacturer, and disaggregated the automatically identified locations into three fields: country of origin of report, countries referred to, and country of origin of any additional sources referenced. Additionally, a one-line summary of the report content was created by the analyst. After populating these fields, an overall impression of the report tone (either positive or neutral, or negative), and report priority level (high, medium, or low) were decided on the basis of agreed criteria.

A report was coded as negative if it contained any indication of concern about a vaccine or vaccination programme, such as information about an adverse event that occurred after immunisation, a vaccine suspension, or any other factor that has a negative effect on a vaccine programme. A report was coded as positive or neutral if it contained no indication of public concern about a vaccine or vaccination programme. A report that discussed an adverse event that occurred after immunisation could be coded as positive if, for instance, the article was about the investigation of a reported adverse event that had been confirmed to be unrelated to a vaccine. Other positive reports included topics such as a country announcing the introduction of a new vaccine or plans for a vaccination campaign.

We used the definition of adverse events as any untoward medical occurrence that follows immunisation that does not necessarily have a causal relation with the use of the vaccine, as described by the Council for International Organizations of Medical Sciences and WHO working group on vaccine pharmacovigilance.23

On average it took 5 min per report for this stage of the data curation to be completed, although time taken varied substantially depending on the length, complexity, and clarity of the article. Within 24 h of receipt, each report was read and assessed by one of the team of analysts trained with a standardised database user guide developed by the research team. Periodic quality assessment was done to ensure consistency between reviewers (figure 1).

 

Figure 1 Full-size image (79K) Download to PowerPoint

Report collection, data entry, and coding in the database

The blue HealthMap section depicts the automated computer processes, and the green LSHTM/ProMed section depicts the processes that involved data analysts from LSHTM and ProMed. LSHTM=London School of Hygiene & Tropical Medicine. ProMED=Program for Monitoring Emerging Diseases.

Data prioritisation

Report priority was classified as high if the report contained any reference to a serious adverse event that occurred after immunisation, such as death or hospitalisation, or mention of vaccine refusal, suspension, or withdrawal within the previous 6 months. Priority was classified as medium if the report did not contain reference to vaccine refusal, but did contain public questioning or concerns about vaccines or the immunisation programme occurring within the previous 6 months. Priority was classified as low whenever content was deemed positive or neutral in nature, or when the event in question occurred more than 6 months before the report date (eg, if the report was discussing the resolution of an older issue). The rationale for the high, medium, and low coding was to identify which reports needed priority investigation and their content to be shared with relevant individuals and institutions (ie, those that were coded as high priority); reports coded as medium needed to be monitored closely, and low-coded reports were archived for future reference and as background information for changes in concern level. The decision to code events reported as occurring within the previous 6 months as either high or medium priority was to distinguish recent events from articles referencing historic events that were neither new nor recurring, but that needed monitoring (medium priority). Every report classified as high priority was automatically sent to all team members and discussed at a weekly teleconference. Interested individuals, Ministries of Health, WHO, and UNICEF officials were also notified.

Duplicate reports with the same URL were automatically deleted, but those with different URLs were recorded as separate reports, recognising the fact that replicated reports show the spread of information, and are therefore important to the analyses.

Statistical analysis

Summary statistics given here comprise simple counts and percentages. All figures were generated in Matlab version 8.0.0.783 (R2012b).

Role of the funding source

The sponsor of the study had no role in research design, data collection, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

We analysed only the data that were entered into the database between May 1, 2011, and April 30, 2012 (after all refinements to the search terms and processes were made), during which 10 380 reports originating from 144 countries were collected and coded. Of these reports, 7171 (69%) contained positive or neutral content about vaccines and vaccination programmes and 3209 (31%) contained negative content (figure 2).

 

Figure 2 Full-size image (106K) Download to PowerPoint

Proportion of vaccine-related reports categorised as positive or neutral, by country

Based on analysis of all 10 380 reports. Of the 9655 reports (93%) that mentioned a country or countries, 11 535 countries were mentioned. Countries about which there were fewer than ten vaccine-related reports are shaded grey. The world proportion (69%) is shown by the arrow on scale bar. Country border data are from the Global Administrative Areas database.24

More than one data category could be allocated to one report. Of the 10 380 reports identified, 22 499 data categories were recorded (14 145 for the 7171 positive or neutral reports and 8354 for the 3209 negative reports). During the analysis of the data, related data categories were clustered into the following groups: beliefs, awareness, and perceptions (eg, religious beliefs, risk perceptions); contextual factors (eg, conflict or war); impacts (eg, vaccine suspension, vaccine refusals); vaccine delivery programme (eg, mass versus routine campaign, vaccine supply, costs); vaccine safety (eg, reports or concerns about adverse events occurring after immunisation, additives, or preservatives); vaccine development and introduction (eg, vaccine research or new products launched or approved); and recommendations about vaccines, disaggregated according to source (eg, national immunisation programme, religious leader, non-governmental health organisation; appendix).

Of the positive and neutral reports, 4632 (33%) data categories were associated with vaccine development and introduction; 4332 (31%) with vaccine delivery programmes; 2141 (15%) with vaccine recommendations; 1475 (10%) with contextual factors; 970 (7%) with beliefs, awareness, and perceptions of vaccines; 428 (3%) with vaccine safety; and 167 (1%) with impacts on vaccine programmes and disease outbreaks (figure 3). Of the negative reports, 1977 data categories (24%) were associated with impacts on vaccine programmes and disease outbreaks; 1726 (21%) with beliefs, awareness, and perceptions; 1371 (16%) with vaccine safety; 1336 (16%) with vaccine delivery programmes; 1119 (13%) with vaccine recommendations; 582 (7%) with contextual factors; and 243 (3%) with vaccine development and introduction (figure 3).

 

Figure 3 Full-size image (47K) Download to PowerPoint

Number of data categories allocated to all reports, per data category group

Data is from all 10 380 reports, obtained between May, 2011, and April, 2012, split according to whether the report was classified as positive or neutral, or negative. Reports could be classified by more than one data category.

Our grouping of data categories included the group contextual factors, because some contextual factors can amplify negative sentiments (eg, marginalised populations, negative high-profile individual) and others attenuate them (eg, high vaccine acceptance rates, positive high-profile individual), which is important for the prioritisation of reported concerns. A more detailed breakdown of the data categories allocated to the 10 380 reports is shown in the appendix.

We categorised the reported data by vaccine type—worldwide and in five selected countries (China, Finland, France, Nigeria, and Pakistan; figure 4). We chose these countries because they had known or widely reported issues with specific vaccines before or during the study period, and we wanted to show that our data collection was consistent with established concerns. We note that the dominant vaccines discussed (either negatively or positively) in the media in the five countries are consistent with genuine issues that were reported concurrently by WHO, providing some validation of the reliability of the data collection. For instance, polio vaccination dominates in Nigeria and Pakistan, two of the three remaining endemic countries, and the polio vaccination coverage in China is probably a result of the 2011 spread of the disease from Pakistan to China—the first cases of the disease in China since 1999.25 The media coverage of H1N1 influenza in Finland was due to the heightened concerns around the 2010—11 suspected links between narcolepsy and H1N1 influenza vaccination,26 and the attention to measles vaccination in France is consistent with the 2011 outbreak.27

 

Figure 4 Full-size image (236K) Download to PowerPoint

Proportionate number of times specific vaccine types were covered positively and negatively in the media, worldwide and in five selected countries

Vaccine types were reported positively (A) 9157 times worldwide, 45 times in France, 22 times in Finland, 171 times in China, 236 times in Nigeria, and 261 times in Pakistan. Vaccine types were reported negatively (B) 4900 times worldwide, 154 times in France, 29 times in Finland, 94 times in China, 193 times in Nigeria, and 311 times in Pakistan. Country border data are from the Global Administrative Areas database.24 HPV=human papillomavirus. JE=Japanese encephalitis.

One of the important dimensions of the surveillance system is the ability to monitor the evolution of concerns over time and location. We mapped reports about human papillomavirus (HPV) vaccination from India, from May, 2011, to November, 2011, to monitor the effect of the April, 2010, government suspension of the HPV vaccine demonstration projects in Gujarat and Andhra Pradesh, caused by pressure from activists (figure 5).28 Although 11 (28%) of the 39 reports about HPV in India obtained during the study period (May 1, 2011, to April 30, 2012) were positive or neutral about HPV vaccines, most, 28 (72%), were negative—with some already posted on European and US anti-HPV vaccine websites—and were still focused on the issues surrounding the suspension of the HPV vaccine demonstration projects towards the end of our data collection period (April 30, 2012), 2 years after the government suspension of the project.

 

Figure 5 Full-size image (152K) Download to PowerPoint

Cumulative number of reports from India about human papillomavirus vaccination over time, by location and priority

Data consists of 48 location mentions from 31 reports obtained between May, 2011, and November, 2011. Country border data are from the Global Administrative Areas database.24

Discussion

33% of positive reports obtained worldwide were categorised as being about vaccine development and introduction, whereas only 3% of negative reports made reference to that topic; this percentage point difference was the largest noted between positive and negative reports out of all topic categories. The category of beliefs, awareness, and perceptions shows a similar large difference, albeit in the opposite direction, with 3% of positive reports making reference to this topic compared with 21% of negative reports. These reports include those discussing religious and philosophical beliefs about vaccines, and perceptions about the motive behind vaccination (eg, government, industry). More negative reports discussed beliefs and perceptions than mentioned vaccine safety (16% of negative reports); negative reports discussing vaccine safety focused on adverse events and concerns about additives such as thiomersal.

Clearly, the proportions of positive and negative reports discussing each topic would change if examined at a more local level or by specific vaccine. Disaggregation of reports by positive or negative sentiment, country, and vaccine type could identify countries where specific vaccines might need more tailored engagement strategies than are currently in place.

The subnational analysis of reports about HPV vaccination from India (figure 5), is an example of an analysis that could help to inform delivery and engagement strategies. The HPV vaccine is available in India only through the private sector; it has not yet been included in Indias national immunisation schedule. When the HPV vaccine is considered for inclusion in the national immunisation schedule, this type of analysis, which identifies the clustering of reports and their sentiments in specific localities, might be useful to inform the design of engagement strategies, which will be particularly needed in Gujarat and Andhra Pradesh. When faced with several concerns and limited human and financial resources, this type of assessment could help to prioritise attention and allocation of resources.

Despite the strengths of our surveillance system, including the creation of a broad-spectrum and reflexive typology of concerns, it also has limitations. First, we used only English search terms to collect the initial data. Although some reports from non-English speaking countries were translated and included in the dataset, most were published in English, and included English language media from countries where the primary language is not English. However, the primary search terms are now being translated into five additional languages: Arabic, French, Mandarin, Russian, and Spanish.

Although human curation was important in the development and refinement of the system, it also restricted the volume of reports that could be processed. Efforts are focused on automating as much of the data collection and classification as possible, using the human-curated dataset as a training set. These changes will result in an increase in the geographical reach, volume, and speed at which reports can be classified and analysed. The changes will also help to enhance the typology of concerns, understand specific contextual factors, and develop the risk assessment method. Future iterations of the system will also expand the numbers of sources that are included. This expansion will be achieved by increasing the languages of our search terms and using other modes of social media (eg, Twitter).

The nature of public concerns about vaccines is complex and highly diverse. No single report can be relied upon to signal a genuine public concern. Instead, the strength and potential effects of a vaccine rumour or public concern are determined by clusters and persistent patterns of reports expressing similar concerns.

The surveillance system we present in this report allows systematic monitoring and assessment of media reports for vaccine sentiment, with the aim of detecting concerns as they emerge and evolve in real time. We created a typology of the scope of public concerns about vaccines and a way to monitor the ecology of those concerns as they develop temporally and spatially. Although this system has not been running long enough to show its long-term predictive value, it allows real-time characterisation of vaccine sentiments by topic, negative or positive content, location, time, and risk level. The third objective, to assess which reported concerns merit priority attention, needs longer-term study to validate the prioritisation methods.

Although international systems exist to monitor and investigate adverse events that occur after immunisation, and several local and vaccine-specific studies29 have investigated factors affecting vaccine acceptance over restricted periods of time (panel), no global systems are in place to routinely monitor and investigate emerging vaccine concerns that are not solely related to adverse events that occur after immunisation. Adverse event reporting systems are crucial to ensure vaccine safety, but, as shown here, they do not adequately capture the multitude of concerns that drive vaccine hesitancy and refusals.

Panel

Research in context

Systematic review

We did a systematic review with the search terms “vaccin*”, “immunis*”, “immuniz*”, AND “trust”, “confidence”, and a set of additional related terms (eg, “hesitancy”, “anxiety”), with no date restrictions, of both mainstream and regional databases including Medline, Embase, PsycINFO, International Bibliography of the Social Sciences, Cochrane, Web of Science, Cumulative Index to Nursing and Allied Health Literature, Index Medicus for the Eastern Mediterranean Region, Literatura Latino Americana em Ciências da Saúde, and Africa-Wide Information. We did the latest search on Nov 12, 2012. Within the results of the broader search, we then searched for relevant research about online monitoring, and found four relevant studies: an assessment of vaccination sentiments in social media that analysed content and correlated the findings with vaccination rates estimated by the Centers for Disease Control and Prevention;30 an analysis of discussions about the measles, mumps, and rubella vaccine in an online chat forum for parents;31 a content analysis of top search engine results for human papillomavirus vaccine information online;32 and a study that surveyed web users and assessed the popularity of influenza-vaccine-related sites during an influenza pandemic.33 None used worldwide data and none collected data about all types of vaccines.

Interpretation

This real-time information surveillance system uniquely captures both positive and negative vaccine reports and sentiments as reported in both traditional news and social media reports around the world. Because the system can categorise report content by type and monitor the evolution of concerns spatially and temporally, the system could be used to inform vaccine delivery and engagement strategies.

The importance of listening to the public throughout the design and implementation of vaccine programmes and research trials cannot be understated. Even more important is acting on what is learned and acting early. The retrospective analyses that informed this systems development showed that early signs of concern have often been available well before their most serious effects occurred, but were not acted on, largely because the potential results were not expected. Although we show that vaccine concerns can vary geographically, evidence of global dissemination of vaccine concerns was apparent, enhanced by internet-based communication. Thus a global approach might help to detect confidence issues before they become established in other geographical areas.

We now have a growing body of evidence of the potential risks of the spread of unchecked rumours, and of failing to address legitimate questions and concerns. Even if concerns driven by misinformation or reported adverse events are investigated and confirmed as not being caused by vaccines, concerns and reputations in the public mind need to be addressed. Although 69% of the global reports were positive about vaccines, 31% were not. And, of these 31%, a large proportion of concerns are related to belief systems. The public health community should not underestimate the implications of the global burden of belief.

Contributors

All authors contributed to study design, data analysis and interpretation, and writing and approval of the manuscript. HJL was the principal investigator. DMDS did the statistical analysis and produced all figures.

Conflicts of interest

One representative from the Bill & Melinda Gates Foundation, which provided the funding, is a member of the projects international advisory group, which provided feedback over the course of the research, but the research team had sole responsibility for all decisions about the conduct of the research and analysis of the findings. Since submitting this Article, Epidemico has been formed by Boston Childrens Hospital to handle the licensing of HealthMap data for commercial companies. JSB holds an equity stake in Epidemico. All other authors declare that they have no conflicts of interest.

Acknowledgments

We thank the international advisory group of The Vaccine Confidence Project for their valuable feedback throughout this research.

For Google News see https://news.google.co.uk/

For Google Blog Search see http://www.google.com/blogsearch

For more on Moreover Technologies see http://www.moreover.com/

For Google Translate see http://translate.google.co.uk/

Supplementary Material

Supplementary appendix

PDF (330K)

References

1 Larson HJ, Cooper LZ, Eskola J, Katz SL, Ratzan S. Addressing the vaccine confidence gap. Lancet 2011; 378: 526-535. Summary | Full Text | PDF(199KB) | CrossRef | PubMed

2 Black S, Rappuoli R. A crisis of public confidence in vaccines. Sci Transl Med 2010; 2: 61mr1. PubMed

3 Yahya M. Polio vaccines—“no thank you!” barriers to polio eradication in Northern Nigeria. Afr Aff (Lond) 2007; 106: 185-204. PubMed

4 Kauffman JR, Feldbaum H. Diplomacy and the polio immunisation boycott in Northern Nigeria. Health Affairs 2009; 28: 1091-1101. PubMed

5 Heymann DL, Sutter RW, Aylward RB. Polio eradication: interrupting transmission, towards a polio free-world. Future Virology 2006; 1: 181-188. PubMed

6 Larson HJ, Ghinai I. Lessons from polio eradication. Nature 2011; 473: 446-447. CrossRef | PubMed

7 Global Polio Eradication Initiative. Report by the WHO Secretariat to the 60th World Health Assembly. Poliomyelitis: mechanism for management of potential risks to eradication (May 24, 2008). http://www.polioeradication.org/content/general/WHA61_Resolution_English.pdf. (accessed Jan 31, 2013).

8 Institute of Medicine. Immunisation safety review: vaccines and autism. Washington, DC: The National Academies Press, 2004.

9 Honda H, Shimizu Y, Rutter M. No effect of MMR withdrawal on the incidence of autism. J Child Psychol Psychiatry 2005; 46: 572-579. CrossRef | PubMed

10 Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet 1998; 351: 637-641. Summary | Full Text | PDF(942KB) | CrossRef | PubMed

11 Godlee F, Smith J, Marcovitch H. Wakefields article linking MMR vaccine and autism was fraudulent. BMJ 2011; 342: c7452. CrossRef | PubMed

12 Health Protection Agency. Summary of trends in vaccination coverage in the UK. Annual COVER report: 2005/06. http://www.hpa.org.uk/webc/HPAwebFile/HPAweb_C/1194947367316. (accessed Jan 31, 2013).

13 NHS. NHS immunisation statistics, England 2011—12. https://catalogue.ic.nhs.uk/publications/public-health/immunisation/nhs-immu-stat-eng-2011-2012/nhs-immu-stat-eng-2011-12-rep.pdf. (accessed Jan 31, 2013).

14 Larson HJ, Heymann DL. Public health response to influenza A (H1N1) as an opportunity to build public trust. JAMA 2010; 303: 271-272. CrossRef | PubMed

15 Kasperson RE, Renn O, Slovic P, et al. The social amplification of risk: a conceptual framework. Risk Anal 1988; 8: 177-187. CrossRef | PubMed

16 Madoff LC. ProMED-mail: an early warning system for emerging diseases. Clin Infect Dis 2004; 39: 227-232. CrossRef | PubMed

17 Public Health Agency of Canada. The Global Public Health Intelligence Network (GPHIN). http://www.phac-aspc.gc.ca/gphin/. (accessed Jan 31, 2013).

18 WHO. Global Outbreak Alert & Response Network. http://www.who.int/csr/outbreaknetwork/en/. (accessed Jan 31, 2013).

19 Surveillance Sans Frontières: Internet-based emerging infectious disease intelligence and the HealthMap project. PLoS Med 2008; 5: e151. CrossRef | PubMed

20 Guy S, Ratzki-Leewing A, Bahati R, Gwadry-Sridhar F. Social media: a systematic review to understand the evidence and application in infodemiology. Lect Notes Inst Comput Sci Soc Inform Telecomm Eng 2012; 91: 1-8. PubMed

21 The Vaccine Confidence Project. Advisory group. www.vaccineconfidence.org/advisorygroup.html. (accessed Jan 31, 2013).

22 Freifeld CC, Mandl KD, Reis BY, Brownstein JS. HealthMap: global infectious disease monitoring through automated classification and visualization of Internet media reports. J Am Med Inform Assoc 2008; 15: 150-157. CrossRef | PubMed

23 Council for International Organizations of Medical Sciences. Definition and application of terms for vaccine pharmacovigilance: report of CIOMS/WHO Working Group on Vaccine Pharmacovigilance. Geneva: Council for International Organizations of Medical Sciences, 2012.

24 Global Administrative Areas (GADM). GADM database of global administrative areas. http://www.gadm.org. (accessed Feb 19, 2013).

25 Donaldson L, Sayed NE, Koplan J, et alfor the Independent Monitoring Board of the Global Polio Eradication Initiative. Report. http://www.polioeradication.org/Portals/0/Document/Aboutus/Governance/IMB/4IMBMeeting/IMBReportOctober2011.pdf. (accessed Jan 31, 2013).

26 Dauvilliers Y, Montplaisir J, Cochen V, et al. Post H1N1 narcolepsy-cataplexy. Sleep 2010; 33: 1428-1430. PubMed

27 WHO, Regional Office for Europe. Measles outbreaks across Europe show no sign of slowing. www.euro.who.int/en/what-we-do/health-topics/disease-prevention/vaccines-and-immunization/news/news/2011/04/measles-outbreaks-across-europe-show-no-sign-of-slowing. (accessed Jan 31, 2013).

28 Larson HJ, Brocard P, Garnett G. The India HPV-vaccine suspension. Lancet 2010; 376: 572-573. Full Text | PDF(109KB) | CrossRef | PubMed

29 Amarasinghe A, Black S, Bonhoeffer J. Effective vaccine systems in all countries: a challenge for more equitable access to immunization. Vaccine 2013; 31 (suppl): B108-B114. CrossRef | PubMed

30 Salathé M, Khandelwal S. Assessing vaccination sentiments with online social media: implications for infectious disease dynamics and control. PLoS Comput Biol 2011; 7: e1002199. CrossRef | PubMed

31 Skea ZC, Entwistle VA, Watt I, Russell E. Avoiding harm to others considerations in relation to parental measles, mumps and rubella (MMR) vaccination discussions—an analysis of an online chat forum. Soc Sci Med 2008; 67: 1382-1390. CrossRef | PubMed

32 Madden K, Nan X, Briones R, Waks L. Sorting through search results: a content analysis of HPV vaccine information online. Vaccine 2012; 30: 3741-3746. CrossRef | PubMed

33 Seeman N, Ing A, Rizo C. Assessing and responding in real time to online anti-vaccine sentiment during a flu pandemic. Healthc Q 2010; 13: 8-15. PubMed

a London School of Hygiene & Tropical Medicine, London, UK

b Chatham House Centre for Global Health Security, Royal Institute of International Affairs, London, UK

c ProMED-mail, International Society for Infectious Diseases, Brookline, MA, USA

d HealthMap, Boston Childrens Hospital Informatics Program, and Division of Emergency Medicine, Boston Childrens Hospital, Boston, MA, USA

e Department of Biomedical Engineering, Boston University, Boston, MA, USA

f Department of Paediatrics, Harvard Medical School, Boston, MA, USA

g University of Massachusetts Medical School, Worcester, MA, USA

Correspondence to: Dr Heidi J Larson, London School of Hygiene & Tropical Medicine, London WC1E 7HT, UK