Our research group at the Programme for Ocular Inflammation and Infection Translational Research (PROTON) is led by Associate Professor Rupesh Agrawal, from the National Healthcare Group Eye Institute (NHGEI), Singapore.
PROTON is committed to advancing research and innovation in ocular inflammatory and infectious diseases. As part of our efforts, we have performed more than 10 systematic reviews and meta-analyses, providing valuable insights into these complex conditions and contributing to the development of more effective treatments and interventions.
Our group recently read up on an article by Nanji et al., which discussed the statistical fragility of meta-analyses in the ophthalmology field.1 His paper found that unfortunately, the statistical significance of various papers often hinged on the outcome of a handful of patients. Associated with this, Nanji’s group recommended the usage of the Fragility Index (FI) parameter – an easily interpretable measure of statistical robustness of any published systematic review and meta-analysis.2
So, what are the implications for the everyday eyecare professional?
First, we need to define meta-analyses. These are a form of systematic review that synthesise data from multiple previous studies addressing the same research question, thereby increasing the overall sample size and improving the statistical power and generalisability of the findings.3 They are a valuable tool in evidence-based practice, often used by clinicians to inform decision-making and develop review articles. The process of conducting a meta-analysis involves several rigorous steps, including the systematic selection of relevant studies, development of strict inclusion and exclusion criteria, data extraction, and advanced statistical analyses to combine the results across studies, among other methodological considerations.
Clinical practice is inherently shaped by academic literature, the best available evidence, and the strength of its recommendations.4 The summative nature of existing evidence portrayed by meta-analyses make it the strongest candidate for informing clinical practice worldwide.5 To put this into perspective, the meta-analysis published in 2014 by Tham et al, titled ‘Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040: A Systematic Review and Meta-Analysis’ is among the most highly cited papers in the field of ophthalmology ever published.6 According to Google Scholar at the time of writing, this meta-analysis has been cited a whopping 7,083 times, placing it within the top 1% of most highly cited ophthalmology articles of all time.7 It has gone on to influence public policy, including Optometry Australia’s Clinical Practice Guide on Glaucoma.8
However, the meta-analyses which are most influenced by the usage of the Fragility Index (FI) statistic are those which evaluate differences between the clinical outcomes of two interventions. This is because the downstream impact is the favoured practice of one approach over the other, and misleading statistical outcomes could starkly affect patient care across the field. Some examples of such meta-analyses include these by Tai et al and Quiroz-Reyes et al.9,10
In response to Nanji’s group, we added on by outlining a series of recommendations for the ophthalmology community with regards to future meta-analyses publications.11 These include:
• Incorporation of Fragility Index (FI) in the reporting and publication of all meta-analyses, with guidelines governing a standard minimum FI before publication can proceed
• Greater community focus on the quality of meta-analyses over the quantity
• Provision of deeper contextual interpretations of findings to draw relevance between statistical significances and clinical practice
• Enhancement of peer review processes on meta-analyses, including the usage of appraisal tools
• More strategic funding and resource allocation towards training junior researchers on advanced statistical methods and meta-analyses proficiency
• Exploration of innovative approaches to meta-analyses, including large language models (LLMs) and other artificial intelligence (AI) technologies
• Enhancement of search strategies and increased availability of Cochrane Courses to reduce publication bias
• Improvement in the declaration of statistical methods and more thorough guidelines on their usage to maintain statistical rigour within the field
Translationally, clinicians likely won’t be able to observe any improvements in the reliability of meta-analyses. However, as the pinnacle of statistical rigour in the academic world, meta-analyses have great influence in the scientific advancement of the field as a whole.12 While our patients may look the same as yesterday, inaccurate clinical practice guidelines may take decades, or even the next century, to manifest. It is paramount that we work towards a goal of reliable scientific literature as a community to ensure we tread along the right path, shoulder-to-shoulder.
ABOUT THE AUTHOR
Name: Mattias Wei Ren Kon
Qualifications: Currently an undergraduate reading my Bachelor’s
Affiliations: Yong Loo Lin School of Medicine, National University of Singapore
Location: Singapore
Years in industry: 1
NOTE: Dr William Rojas-Carabali and A/Prof Rupesh Agrawal co-authored this article.
More reading
What space travel teaches us about dry eye
Australian optometrist Dr Daisy Shu on her lessons from Harvard
Meeting real-world needs with the bionic eye – A/Prof Penelope Allen
References
1. Nanji K, Xie J, Hatamnejad A, Pur DR, Phillips M, Zeraatkar D, et al. Exploring the fragility of meta-analyses in ophthalmology: a systematic review. Eye. 2024 Jul 20; https://doi.org/10.1038/s41433-024-03255-2
2. Borkar NB, Nair A. Significance of fragility index in meta-analysis. Saudi Journal of Anaesthesia. 2024 Mar 14;18(2):310–1. https://doi.org/10.4103/sja.sja_749_23
3. Field AP, Gillett R. How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology. 2010 May 25;63(3):665–94. https://doi.org/10.1348/000711010×502733
4. De Leo A, Bloxsome D, Bayes S. Approaches to clinical guideline development in healthcare: a scoping review and document analysis. BMC Health Services Research. 2023 Jan 16;23(1). https://doi.org/10.1186/s12913-022-08975-3
5. Wang X, Zhang X, Li Z, Zhong W, Yang P, Mao C. A brief introduction of meta‐analyses in clinical practice and research. The Journal of Gene Medicine. 2021 Jan 16;23(5). https://doi.org/10.1002/jgm.3312
6. Tham YC, Li X, Wong TY, Quigley HA, Aung T, Cheng CY. Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040. Ophthalmology. 2014 Jun 26;121(11):2081–90. https://doi.org/10.1016/j.ophtha.2014.05.013
7. Koh BMQR, Banu R, Sabanayagam C. The 100 Most Cited Articles in Ophthalmology in Asia. Asia-Pacific Journal of Ophthalmology. 2020 Sep 1;9(5):379–97. https://doi.org/10.1097/apo.0000000000000325
8. Haines C, Hart K, Optometry Australia, Phu J, Ashby B, Armitage J, et al. Clinical Practice Guide for the Diagnosis and Management of Open Angle Glaucoma. Optometry Australia Glaucoma Clinical Practice Guide. 2020.
9. Tai F, Nanji K, Garg A, Zeraatkar D, Phillips M, Steel DH, et al. Subthreshold Compared with Threshold Macular Photocoagulation for Diabetic Macular Edema. Ophthalmology Retina. 2023 Oct 6;8(3):223–33. https://doi.org/10.1016/j.oret.2023.09.022
10. Quiroz-Reyes M, Quiroz-Gonzalez E, Quiroz-Gonzalez M, Lima-Gómez V. Early versus Delayed Vitrectomy for Open Globe Injuries: A Systematic Review and Meta-Analysis. Clinical Ophthalmology. 2024 Jun 1;Volume 18:1889–900. https://doi.org/10.2147/opth.s466144
11. Kon MWR, Rojas-Carabali W, Cifuentes-Gonzalez C, Agrawal R. Meta-mistake: are fragile meta-analyses in ophthalmology worth the high cost? Eye. 2024 Sep 9; https://doi.org/10.1038/s41433-024-03331-7
12. Berlin JA, Golub RM. Meta-analysis as Evidence. JAMA. 2014 Aug 12;312(6):603. https://doi.org/10.1001/jama.2014.8167
