Mbs reviews vitamin b12 testing report february 2014 table of contents section Page

APPENDIX 5 – Tools for assessing the evidence in the systematic review

Table A5.1: NHMRC Dimensions of Evidence⁽¹⁰²⁾

Type of evidence	Definition
Strength of the evidence Level Quality Statistical precision	The study design used, as an indicator of the degree to which bias has been eliminated by design. The methods used by investigators to minimise bias within a study design. The p-value or, alternatively, the precision of the estimate of the effect (as indicated by the confidence interval). It reflects the degree of certainty about the existence of a true effect.
Size of effect	The distance of the study estimate from the “null” value and the inclusion of only clinically important effects in the confidence interval.
Relevance of evidence	The usefulness of the evidence in clinical practice, particularly the appropriateness of the outcome measures used.

Table A5.2: NHMRC designations of levels of evidence for an intervention (NHMRC)⁽¹⁰²⁾

Level	Intervention
I	A systematic review of level II studies
II	A randomised controlled trial
III-1	A pseudo randomised controlled trial (i.e. alternate allocation or some other method)
III-2	A comparative study with concurrent controls: Non-randomised, experimental trial Cohort study Case-control study Interrupted time series with a control group
III-3	A comparative study without concurrent controls: Historical control study Two or more single arm study Interrupted time series without a parallel control group
IV	Case series with either post-test or pre-test/post-test outcomes

Source: Hierarchies adapted and modified from: NHMRC 1999; Bandolier 1999; Lijmer et al. 1999; Phillips et al. 2001
Table A5.3: NHMRC quality criteria for RCTs, cohort studies, case-control studies and systemic reviews⁽¹⁰²⁾

Study type	Quality criteria
Randomised controlled trials^a	Was the study double blinded? Was allocation to treatment groups concealed from those responsible for recruiting the subjects? Were all randomised participants included in the analysis?
Cohort studies^b	How were subjects selected for the ‘new intervention’? How were subjects selected for the comparison or control group? Does the study adequately control for demographic characteristics, clinical features and other potential confounding variables in the design or analysis? Was the measurement of outcomes unbiased (i.e. blinded to treatment group and comparable across groups)? Was follow-up long enough for outcomes to occur? Was follow-up complete and were there exclusions from the analysis?
Case-control studies^b	How were cases defined and selected? How were controls defined and selected? Does the study adequately control for demographic characteristics and important potential confounders in the design or analysis? Was measurement of exposure to the factor of interest (e.g. the new intervention) adequate and kept blinded to case/control status? Were all selected subjects included in the analysis?
Systematic reviews^c	Was an adequate search strategy used? Were the inclusion criteria appropriate and applied in an unbiased way? Was a quality assessment of included studies undertaken? Were the characteristics and results of the individual studies appropriately summarised? Were the methods for pooling the data appropriate? Were sources of heterogeneity explored?

Source: National Health and Medical Research Council (NHMRC), 2000. How to review the evidence: systematic identification and review of the scientific literature, NHMRC, Commonwealth of Australia, Canberra.

^aBased on work of Schulz et al (1995) and Jadad et al (1996).

^bBased on quality assessment instruments developed and being tested in Australia and Canada.

^cBased on articles by Greenhalgh (1997) and Hunt and McKibbon (1997).

1^ Parietal cells are cells in the stomach that produce gastrichydrochloric acid and also produce the intrinsic factor, which rebinds vitamin B12 after it is released from haptocrrin by digestion.

2^ The search included: Guidelines International Network (G-I-N) at http://www.g-i-n.net/library/international-guidelines-library/; National Guidelines Clearinghouse at www.guidelines.gov; National Health and Medical Research Council (NHMRC) at http://www.nhmrc.gov.au/guidelines-publications

3^ The following HTA websites were searched: Agency for Healthcare Research and Quality (AHRQ) at www.ahrq.gov; Canadian Agency for Drugs and Technologies in Health (CADTH) at http://www.cadth.ca/en; National Institute for Health and Care Excellence (NICE) at www.nice.org.uk; Australasian College of Surgeons (ASERNIP-S) at http://www.surgeons.org/for-health-professionals/audits-and-surgical-research/asernip-s/

4^ Based on data processed to 31 May 2013; 2012/13 is therefore incomplete.

5^ The Clinical Practice Guideline for anaemia in chronic kidney disease77. (KDIGO) KDIGO. KDIGO Clinical Practice Guideline for Anemia in Chronic Kidney Disease2012. is based upon systematic literature searches last conducted in October 2010, supplemented with additional evidence through March 2012.

6^ The University of Bristol. 2013. QUADAS-2. Retrieved from http://www.bris.ac.uk/quadas/quadas-2/

7^ MMA and Hcy are the two most commonly reported reference standards in the literature

8^ The likelihood ratio (LR) of any clinical finding is the probability of that finding in patients with disease divided by the probability of the same finding in patients without disease. Positive likelihood ratio is equivalent to the probability that a person with the disease tested positive for the disease (true positive) divided by the probability that a person without the disease tested positive for the disease (false positive).

9^ Negative likelihood ratio (NLR) is equivalent to the probability that a person with the disease tested negative for the disease (false negative) divided by the probability that a person without the disease tested negative for the disease (true positive).