AGREE II Assessment of RACGP Recommendation of Incremental Physical Activity for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome - Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS)

Overview

Introduction

The AGREE II Instrument

Assessment of the RACGP Guideline

Assessment of the NICE Guideline

Assessment of IOM Guideline

ME/CFS Guideline Comparison

Conclusion

References

Appendices

Appendix A: RACGP guideline scoresheet

Appendix B: NICE guideline scoresheet

Appendix C: IOM guideline scoresheet

Overview

An assessment of the Royal Australian College of General Practitioners (RACGP) Treatment Guideline For Incremental Physical Activity In Chronic Fatigue Syndrome/Myalgic Encephalomyelitis ¹ (ME/CFS) rated the guideline as not fit for purpose, with a score of 28%.

The assessment evaluated the RACGP guideline under the AGREE II instrument ². The AGREE II instrument is a rigorous, internationally used tool for assessing guideline quality ².

The RACGP guideline received the minimum score for 11 of the 23 AGREE II items; 2% for rigour of development; and 11% for stakeholder involvement. The main deficits are:

The guideline provides no evidence of a systematic review of the research, or a critical assessment of the methodology and limitations of the cited papers. Consequently, the guideline failed to meet the requirements of items 7 to 10 of AGREE II ².
The listed benefits of exercise therapy are for other fatigue-related conditions; not for ME/CFS. On this point, the guideline fails to meet item 12 of AGREE II ², which requires an explicit link between the recommendations and the evidence.
Item 5 of AGREE II²requires stakeholder consultation and consideration of stakeholder views. Although the HANDI project team included practitioners and researchers from a wide variety of disciplines, they did not appear to be involved in the care of people with ME/CFS. No attempt appears to have been made to consult with the patient population and no patients were included in the committee. Therefore, the RACGP guideline fails to have actively consulted with stakeholders and to have considered their views.
The guideline acknowledges reports of harm from ME/CFS patients undergoing exercise programs but dismisses the reports without providing evidence for the dismissal. This contravenes items 5, 11 and 12 of AGREE II ². These items require consultation with the target population; an explicit link between the recommendations and the supporting evidence; and equal consideration of benefits and harms.
The guideline concedes that the recommendations may not apply to patients diagnosed with ME/CFS under stricter criteria, with more serious postexertional malaise, but does not indicate how to differentiate between the patients who will benefit and those who will experience harm. Therefore, the guideline fails to fulfil items 3 and 15 of AGREE II², which require that the target population be well defined; and that the recommendations be specific and unambiguous.

Overall, the guideline development group failed to rigorously review the evidence; link recommendations to the evidence; involve stakeholders; or give equal consideration to benefits and harms. Consequently, the assessment found that the RACGP guideline is low quality and not fit for purpose.

This report also documents an assessment of the IOM guideline for ME/CFS diagnosis ³ and the NICE guideline for diagnosis and management of myalgic encephalomyelitis/chronic fatigue syndrome ⁴ under the AGREE II instrument ². In contrast to the RACGP guideline, the AGREE II assessment rated the IOM guideline ³ and the NICE guideline ⁴ as high quality, scoring 83% and 94% respectively. Pages 18-19 contain a table comparing the RACGP, NICE and IOM guideline scores. Both the IOM and NICE guidelines scored well for scope of purpose, rigour of development, stakeholder involvement and clarity. Both guidelines scored higher than the RACGP guideline for editorial independence. Given that the high quality NICE guideline covers both diagnosis and treatment, it is recommended that the RACGP guideline be replaced as soon as possible with the NICE guideline, until the NHRMC has completed its own guideline review.

Introduction

In April 2024, the RACGP published a treatment guideline for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) recommending incremental physical activity and graded exercise therapy ¹. Patient and advocacy groups assert that people with ME/CFS are harmed by graded exercise therapy. These groups state that the guideline is biased; relies on poor quality evidence; and fails to consider consistent reports of harm. To assess the claims of patient groups, this report evaluates the RACGP guideline using the AGREE II instrument.

The AGREE II Instrument

The Appraisal of Guidelines for REsearch & Evaluation (AGREE) Instrument² was developed to address the variable quality of guidelines. AGREE II is a tool for assessing the methodological rigour and transparency with which a guideline was developed (p. ii) ².

The AGREE II instrument contains 23 items grouped under six domains. The items are scored from 1 to 7, where 1 is strongly disagree and 7 is strongly agree. High domain scores indicate a high quality guideline.

Scope and purpose. Does the guideline outline the purpose, expected benefits, target population, gender, age, comorbidities, disease severity, excluded populations, precautions, adverse effects, interventions, comparisons, outcomes and health care settings?
Stakeholder involvement. Does the guideline development group contain people from all relevant professional groups? Are names, discipline/expertise, institution, committee role and geographical location published? Were the views and preferences of the target population (patients, public, etc.) sought? How was the information gathered and used to inform the guideline development process and formation of the recommendations? Who is the intended audience and how will the guideline be used?
Rigour of development. Did the committee conduct a systematic search for evidence? How was relevant evidence selected? Are the strengths and limitations of the evidence described? Does the guideline describe how the recommendations were formulated? Were health benefits, side effects and risks considered? Is there an explicit link between the recommendations and the supporting evidence? Has the guideline been externally reviewed by experts prior to publication? Does the guideline include a procedure for updates?
Clarity of presentation. Are the recommendations specific and unambiguous? Are different options for management of the condition presented? Are key recommendations easily identifiable?
Applicability. Does the guideline describe barriers and facilitators to implementation? Does the guideline provide tools and advice on how the recommendations can be put into practice? Have the potential resource implications been considered? Does the guideline include monitoring and/or auditing criteria?
Editorial independence. Have the views of the funding body influenced the recommendations? Have conflicts of interest been recorded and addressed?

Assessment of the RACGP Guideline

In 2024, the RACGP published a guideline Incremental Physical Activity for Chronic Fatigue Syndrome/Myalgic Encephalomyelitis¹. The following sections contain comments on the RACGP guideline for each AGREE II domain, and a domain percentage score. The comments address the assessment criteria and considerations described by the AGREE II manual ². The summary section contains a table of the RACGP guideline domain scores. The RACGP guideline scoresheet is in Appendix A.

Domain: Scope and purpose

The document fully describes the purpose of the guideline and expected benefits. However, the RACGP guideline defines the target population poorly. The guideline states that graded exercise therapy may be helpful for people diagnosed with chronic fatigue syndrome under broader, more inclusive diagnostic criteria; and that the recommendations may not apply to people diagnosed with ME/CFS under more recent, stricter diagnostic criteria. However, the guideline provides no advice on how to differentiate between the two groups.

The guideline does not address age, gender or comorbidities.

The guideline mentions disease severity and precautions. The document acknowledges that ‘subgroups within the spectrum of CFS/ME who are more vulnerable to more severe PEM’ may experience harm in response to graded exercise therapy. However, the guideline does not indicate how to identify these patients and does not explicitly exclude them.

The listed benefits of exercise therapy are for other fatigue-related conditions; not for ME/CFS.

Score: 44%

Domain: Stakeholder involvement

Committee members were the HANDI project team ¹⁶. The committee membership was not referenced by the guideline and was difficult to find. Although the HANDI project team included practitioners and researchers from a wide variety of disciplines, none appeared to be involved in the care of people with ME/CFS. Anecdotally, one team member has a child with ME/CFS. However, the committee cannot be said to represent practitioner stakeholders. No attempt appears to have been made to consult with the patient population and no patients were included in the committee. Therefore, the RACGP guideline fails to have actively consulted with stakeholders or to have considered their views. Name, expertise and position were provided. Geographical location and member roles in the guideline development group were not provided.

The development group did not actively seek the views and preferences of the target population. The guideline mentions the harm reported in response to graded exercise therapy by some ME/CFS patients. It dismisses these opinions without providing evidence to support the dismissal. Patient reports of harm were not adequately considered.

The guideline does not describe the intended guideline audience or how the guideline may be used.

Score: 11%

Domain: Rigour of development

The RACGP guideline comprises a single web page, representing about 3 pages of printed text. The guideline provides no description of a systematic search for evidence. The web page references three research papers ^1,5,6,7. The guideline contains no explanation of how these three papers were selected or why other research papers were not considered.

The guideline did not critically assess the cited research papers. There is no evaluation of study design, methodological limitations, relevance of outcome measures, consistency of results, bias or applicability to clinical practice. In considering benefits versus potential harm, reports of harm are noted but dismissed without providing evidence to support the dismissal.

The guideline cites two research papers supporting benefits, one of which is a systematic review.

The guideline mentions reports of harm. However, no sources or supporting data for harms or risks are cited. Reports of harm are dismissed without providing supporting research. Consequently, the discussion of the balance between harms and benefits is inadequate. With reference to item 11 of AGREE II ², the guideline development group has not considered the benefits and harms equally.

There is no description of the recommendation development process, outcomes of the development process, or the way in which the development process affected the recommendations.

The guideline does not appear to have been externally reviewed prior to publication.

No procedure is provided for guideline updates.

Score: 2%

Domain: Clarity of presentation

The therapy and intent of the therapy are described. The intended population is ‘people with chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME)’. The guideline does not consider age, gender or comorbidities. Disease severity is discussed. However, the guideline does not give instructions in the case of severe disease or adverse reactions. The guideline suggests that the recommendations may not apply to some patients but does not explain how to identify those patients.

The guideline does not present alternatives for management of the condition.

Key recommendations are easily identifiable.

Score: 50%

Domain: Applicability

Financial and systemic facilitators and barriers are not addressed. The guideline mentions attitudinal barriers. The methods by which this information was obtained are not described. There is no description of how the information influenced the guideline development process.

The guideline does not provide a summary document; links to checklists; links to how-to manuals; tools to capitalize on guideline facilitators; outcome of pilot test; or directions on how users can access tools and resources. The guideline does include some advice on addressing attitudinal barriers.

The potential resource implications of applying the recommendations were not addressed.

The guideline does not provide:

criteria to assess guideline implementation or adherence to recommendations;
criteria for assessing the impact of implementing the recommendations;
advice on frequency and interval of measurement;
descriptions or operational definitions of how the criteria should be measured.

Score: 8%

Domain: Editorial independence

The guideline does not state the funding body. The guideline contains no statement that the views of the funding body did not influence the content of the guideline.

One committee member declared competing interests, which had no relevance to ME/CFS. All other committee members declared that they had no competing interests.

Score: 50%

Summary

Domain scores for the RACGP guideline are as follows:

Domain	Score
Scope and purpose	44%
Stakeholder involvement	11%
Rigor of development	2%
Clarity of presentation	50%
Applicability	8%
Editorial independence	50%
Average score	28%

The RACGP guideline received the minimum score on 11 of the 23 AGREE II items. The guideline development group failed to systematically review the body of evidence and did not identify the strengths and limitations of the cited research studies. The target population was not consulted. Consequently, the RACGP guideline displays very serious limitations and cannot be considered to be fit for purpose.

Assessment of the NICE Guideline

In 2021, the UK’s National Institute for Health and Care Excellence published NICE guideline 206 – Myalgic encephalomyelitis (or encephalopathy)/chronic fatigue syndrome: diagnosis and management ⁴. A supporting document – NICE guideline document G, Evidence Reviews for the Non Pharmacological Management of ME/CFS – reviews 10 exercise studies and their outcome measures ⁸. Document G comprises 411 pages. The NICE guideline’s AGREE II scoresheet is in Appendix B.

Domain: Scope and purpose

The document fully describes the purpose of the guidelines, expected benefits, target population, gender, age, comorbidities, disease severity, excluded populations, precautions, adverse effects, interventions, comparisons, outcomes and health care settings.

Score: 100%

Domain: Stakeholder involvement

The guideline development group contains people from all relevant professional groups. Names, discipline/expertise, institution and committee role are published. Geographical location is not published. Location can be inferred from the institution of professional members, but not for lay members.

The guideline describes the strategies used to capture the views and preferences of patients and the public. It describes the information gathered; and explains how the information gathered was used to inform the guideline development process and formation of the recommendations.

The guideline provides a clear description of the intended guideline audience and a description of how the guideline may be used by its target audience.

Score: 94%

Domain: Rigour of development

The NICE committee used the GRADE process to evaluate the strengths and limitations of the body of evidence with regards to study design; study methodology limitations; and relevance of primary and secondary outcomes ^9,15. Consistency and direction of results were considered. Risk of harm was considered. The committee considered the suitability of the guideline for clinical use.

The NICE guideline describes the guideline development process; the outcomes of the development process; and how the process influenced the guidelines.

NICE guideline document G includes supporting data and reports of benefits and harms ⁸. The guideline discusses the balance between benefits and harms. Recommendations reflect consideration of both benefits and harms.

The guideline describes how the guideline development group used the evidence to inform recommendations. Each recommendation is linked to a key evidence section and reference list. Recommendations are linked to evidence summaries and evidence tables.

The draft guideline was open to review by experts and the general public via an online comments form. Commenters, comments and responses were published online. Some comments initiated changes to the guideline.

NICE publishes a methodology for updating the guideline. Updates are conducted in response to comments and requests, or to changes in the medical environment, rather than at a fixed interval.

Score: 100%

Domain: Clarity of presentation

The recommendations; intent of the recommendations; relevant population; and exclusions are clearly listed.

The guideline describes treatment options and the population or clinical situation most appropriate to each option.

Key recommendations are easily identifiable.

Score: 100%

Domain: Applicability

NICE document C identifies types of barriers and facilitators; describes the barriers and facilitators; describes how this information was collected; explains how the information influenced the guideline development process; and identifies strategies to address facilitators and barriers ¹⁰.

The NICE guideline provides a summary, a baseline assessment tool and a how-to manual. Solutions are linked to a barrier analysis. The guideline does not provide tools to capitalize on guideline facilitators. There appears to have been no pilot testing of recommendations, questionnaires and tools. Information for health professionals is embedded in hundreds of pages of guideline and supporting documents. Apart from the baseline assessment tool, the guideline appears to lack user-friendly, concise, accessible information for GPs.

The resource impact summary predicts little change in costs, because the recommendations do not need any additional resources to implement. Responsibility was delegated to local authorities. It was expected that proper management of ME/CFS may have a marginal effect on initial costs, which are counterbalanced by the protective effects of more effective disease management. The committee was unable to calculate actual costs due to “a lack of robust data on current practice and the variation across organisations and services” ¹¹. Hence, costs would need to be calculated at a local level.

The Surveillance Assessment Process section of the Developing NICE Guidelines manual describes the data collected to assist in monitoring and auditing of guideline uptake ¹². There appears to be no advice on the frequency and interval of measurement. The auditing guidelines appear to be general to NICE, rather than specific to the ME/CFS guidelines. It is unclear whether auditing is in progress.

Score: 71%

Domain: Editorial independence

The guideline was funded by the National Institute for Health and Care Excellence⁹. The guideline lacks a definitive statement regarding the level of influence of the funding body. However, much detail is provided on the literature search and evaluation of the evidence. The guideline contains a statement that the content was based on the body of evidence, and this is supported by documentation of the development group’s processes and the evaluations of the evidence.

NICE has a strict policy on declaring and managing interests¹³.

Score: 100%

Summary

When assessed under AGREE II, the NICE guideline received the following domain scores:

Domain	Score
Scope and purpose	100%
Stakeholder involvement	94%
Rigor of development	100%
Clarity of presentation	100%
Applicability	71%
Editorial independence	100%
Average score	94%

NICE has produced a high quality guideline that received the maximum score in four out of the six domains. The guideline displays excellent scope, rigour of development, editorial independence, clarity and stakeholder involvement. The most apparent flaw is that the NICE guideline fails to provide easily accessible tools to support health practitioners in implementing the guideline. The high AGREE II rating suggests that the NICE guideline is a reliable source for the diagnosis and management of ME/CFS.

Assessment of IOM Guideline

In 2015, the US Institute of Medicine (IOM) published a 282-page book ³ and research paper entitled Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness ¹⁴. The book and paper recommended updated diagnostic criteria for ME/CFS and advised that the disease be renamed ‘systemic exertion intolerance disease’. The IOM guideline’s AGREE II scoresheet is in Appendix C.

Domain: Scope and purpose

Score: 100%

Domain: Stakeholder involvement

The guideline committee includes individuals from a broad array of professional backgrounds. The document details the following information about committee members: name; discipline/content expertise; institution. Geographical location is not stated but can be inferred from the committee member’s institution if required. There is no description of the committee members’ roles in the guideline development.

The guideline describes the strategies used to capture the views and preferences of patients and the public. The guideline explains how the information gathered was used to inform the guideline development process and formation of the recommendations.

The guideline provides a clear description of the intended guideline audience and a description of how the guideline may be used by its target audience.

Score: 94%

Domain: Rigour of development

The guideline lists the search terms and databases searched; date of search; and search strategy used in the literature review.

Target population and outcomes are described. Study designs and comparisons are not described.

The IOM committee used the GRADE process to evaluate the strengths and limitations of the body of evidence with regards to study design; study methodology limitations; and relevance of primary and secondary outcomes. The committee used discussion of the evidence to come to consensus. Risk of harm was considered in the context of over/underdiagnosis. The guideline’s recommendations are suitable for clinical use.

The document describes the guideline development process; the outcomes of the development process; and how the process influenced the guidelines.

The document considers the health benefits of new diagnostic criteria and the risk over over/underdiagnosis. The document discusses research evidence suggesting that patients struggle to obtain a diagnosis.

The guideline describes how research evidence was used to inform the recommendations. The document cites research evidence for each ME/CFS symptom. The recommendations follow from the symptom descriptions. Each section contains a reference list.

The document’s Review section describes an independent review; the purpose of the review; the review method; the names and institutions of the reviewers; and the way the information obtained from the review was used to inform the final document.

The guideline states that the diagnostic criteria should be revised within 5 years; suggests a future funding body; and assigns the responsibility for the update to the US Department of Health and Human Services. The specified evaluation/auditing procedure is vague.

Score: 90%

Domain: Clarity of presentation

The recommendations; intent of the recommendations; relevant population; and exclusions are clearly listed.

The purpose of the guidelines was to create diagnostic criteria; not to define treatment options. Therefore, management options are not considered by the guideline. The guideline has not been scored on this item.

Key recommendations are easily identifiable.

Score: 100%

Domain: Applicability

The guideline identifies types of barriers and facilitators; describes the barriers and facilitators; describes how this information was collected; explains how the information influenced the guideline development process; and identifies strategies to address facilitators and barriers.

A guideline summary is available in the form of a research paper ¹⁴. Links to questionnaires and tools are provided. Links to how-to manuals are provided. Solutions linked to barrier analysis are provided. No tools are provided to capitalize on guideline facilitators. There appears to have been no pilot testing of recommendations, questionnaires and tools. The tools and manuals are hard to locate. The guideline delegates responsibility for implementation to the US Department of Health and Human Services.

Costings were not considered. Responsibility was placed on government agencies and professional bodies.

The guideline contains a short section suggesting that uptake of the diagnostic criteria be monitored via website hits and conference attendee statistics. The section also suggests a research study evaluating attitudinal change in health care providers. The document does not suggest auditing for diagnostic accuracy.

Score: 50%

Domain: Editorial independence

The report lists sponsors in the preface. The preface and the recommendations for future updates infer that the funding bodies did not influence the development of the guidelines. However, this was not explicitly stated. The document describes an evidence-based guideline development process that is informed by the views of patients and advocates, as well as health professionals and researchers.

Professional experience and involvement in ME/CFS-related activities are mentioned. However, competing interests are not specifically addressed.

Score: 67%

Summary

When assessed under AGREE II, the IOM guideline received the following domain scores:

Domain	Score
Scope and purpose	100%
Stakeholder involvement	94%
Rigor of development	90%
Clarity of presentation	100%
Applicability	50%
Editorial independence	67%
Average score	83%

The IOM guideline received high scores for scope and purpose, stakeholder involvement, rigour of development and clarity of presentation. The IOM guideline fails to provide easily accessible tools to support health practitioners in implementing the diagnostic guideline.

The AGREE II rating suggests that the IOM guideline is high quality. The IOM guideline provides well supported advice on the diagnosis of ME/CFS.

ME/CFS Guideline Comparison

The following table compares the NICE, IOM and RACGP guideline scores under the AGREE II framework:

AGREE II Domain	NICE (score)	IOM (score)	RACGP (score)
Scope and purpose	Met (100%)	Met (100%)	Partially met (44%)
Stakeholder involvement	Met (94%)	Met (94%)	Not met (11%)
Rigour of development	Met (100%)	Met (90%)	Not met (2%)
Clarity of presentation	Met (100%)	Met (100%)	Partially met (50%)
Applicability	Partially Met (71%)	Partially Met (50%)	Not met (8%)
Editorial independence	Met (100%)	Partially Met (67%)	Partially Met (50%)
Overall rating	High (94%)	High (83%)	Very serious limitations (28%)

The AGREE II instrument suggests that the RACGP guideline has very serious limitations and is not fit for purpose.

The IOM guideline received a high AGREE II score. The guideline provides well supported ME/CFS diagnostic criteria that may be used in clinical practice. It provides no advice on the management of ME/CFS.

The NICE guideline received the highest score of the three guidelines assessed under AGREE II. The guideline conducted a rigorous literature search; used the GRADE process to assess the evidence; and consulted all stakeholders, including practitioners, carers and patients. The guideline considers both diagnosis and management of ME/CFS.

Conclusion

Of the three guidelines considered, the NICE guideline is both the most comprehensive and the highest quality option for guidance of diagnosis and management of ME/CFS in Australia. This report recommends that the low quality RACGP guideline be replaced with the NICE guideline until the NHMRC has completed its own evidence-based review of ME/CFS diagnosis and treatment in Australia.

References

Royal Australian College of General Practitioners (2023). Incremental Physical Activity for Chronic Fatigue Syndrome/Myalgic Encephalomyelitis. Downloaded from https://www.racgp.org.au/clinical-resources/clinical-guidelines/handi/handi-interventions/exercise/incremental-physical-activity-for-cfs-me. Accessed on 9 May, 2024.
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., Littlejohns, P., Makarski, J., Zitzelsberger, L., & AGREE Next Steps Consortium (2010). AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne, 182(18), E839–E842, downloaded from http://www.agreetrust.org, accessed 6 May, 2024. https://doi.org/10.1503/cmaj.090449
IOM (Institute of Medicine). (2015). Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press, downloaded from https://nap.nationalacademies.org/download/19012, accessed 9 May, 2024
National Institute for Health and Care Excellence (2021). Myalgic encephalomyelitis (or encephalopathy)/chronic fatigue syndrome: Diagnosis and management. Royal College of Physicians Guidelines, downloaded from https://www.nice.org.uk/guidance/ng206/chapter/Recommendations#assessment-and-care-and-support-planning-by-an-mecfs-specialist-team, accessed 22 October, 2022.
Larun, L., Brurberg, K. G., Odgaard-Jensen, J., Price, J. R. (2019). Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews, 10, Article Number CD003200. doi: 10.1002/14651858.CD003200.pub8
White, P. D., Goldsmith, K. A., Johnson, A. L., Potts, L., Walwyn, R., & DeCesare, J. C. (2011). Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): A randomised trial. The Lancet, 377(9768), 823-836. https://doi.org/10.1016/S0140-6736(11)60096-2
Fawzy, N. A., Abou Shaar B., Taha R. M., et al. (2023). A systematic review of trials currently investigating therapeutic modalities for post-acute COVID-19 syndrome and registered on WHO International Clinical Trials Platform. Clin Microbiol Infect. 29(5), 570-577.
National Institute for Health and Care Excellence (2021). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: Diagnosis and management [G] Evidence reviews for the non pharmacological management of ME/CFS, NICE guideline NG206, downloaded from https://www.nice.org.uk/guidance/ng206/evidence/g-nonpharmacological-management-of-mecfs-pdf-9265183028, accessed 22 October, 2022.
National Institute for Health and Care Excellence (2020). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: Diagnosis and management, NICE guideline: Methods, NICE guideline NG206, downloaded from https://www.nice.org.uk/guidance/ng206/evidence/methods-pdf-333546588761, accessed 7 May, 2024.
National Institute for Health and Care Excellence (2020). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: Diagnosis and management [C] Accessing health and social care services, NICE guideline NG206, downloaded from https://www.nice.org.uk/guidance/ng206/documents/evidence-review-3, accessed 29 April, 2024.
National Institute for Health and Care Excellence (2020). Myalgic encephalomyelitis (or encephalopathy) / chronic fatigue syndrome: Resource impact summary, NICE guideline NG206, downloaded from https://www.nice.org.uk/guidance/ng206/resources/resource-impact-summary-report-9266867821, accessed 29 April, 2024.
National Institute for Health and Care Excellence (2024). Developing NICE guidelines: The manual. Downloaded from https://www.nice.org.uk/process/pmg20, accessed 22 September, 2024.
National Institute for Health and Care Excellence (2024). Declaring and managing interests: Advisory committees. Downloaded from https://www.nice.org.uk/Media/Default/About/Who-we-are/Policies-and-procedures/Declaring-managing-interests-for-advisory-committees.docx, accessed 23 December, 2024.
Institute of Medicine. (2015). Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Military Medicine, 180(7), 721–723. https://doi.org/10.7205/MILMED-D-15-0008
Schünemann, H., Brożek, J., Guyatt, G., & Oxman, A. (Eds.). (2013). GRADE handbook for grading quality of evidence and strength of recommendations. The GRADE Working Group. Downloaded from https://gdt.gradepro.org/app/handbook/handbook.html, accessed 13 December, 2024.
Royal Australian College of General Practitioners (2023). Meet the HANDI Project Team. Downloaded from https://www.racgp.org.au/clinical-resources/clinical-guidelines/handi/about-handi/about-the-authors. Accessed on 17 January, 2025.

Appendices

Appendix A: RACGP guideline scoresheet

The following scoresheet assesses the RACGP guideline Incremental Physical Activity for Chronic Fatigue Syndrome/Myalgic Encephalomyelitis ¹using the AGREE II instrument for evaluating guidelines ²:

AGREE II Score Sheet – RACGP
Please rate each of the 23 items from 1 to 7. where 1 is strongly disagree and 7 is strongly agree.
For guidance on scoring, see Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., Littlejohns, P., Makarski, J., Zitzelsberger, L., & AGREE Next Steps Consortium (2010). AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne, 182(18), E839–E842, downloaded from http://www.agreetrust.org. https://doi.org/10.1503/cmaj.090449

Domain	Item	AGREE II Rating	Domain Score	Domain %	Comments
Scope and purpose	1. The overall objective(s) of the guideline is (are) specifically described.	6			The document fully describes the purpose of the guidelines and expected benefits. The document describes the target population.
	2. The health question(s) covered by the guideline is (are) specifically described.	3			The document describes the target population as people with chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME). However, it does not provide diagnostic criteria for this group. The guideline does not differentiate people who will benefit from incremental physical activity from those who are harmed by it. The intervention is described in some detail. The outcomes of the intervention are not compared with the outcomes of other management techniques for ME/CFS. The listed benefits are for other fatigue-related conditions; not for ME/CFS. The text describes the health care settings in which the guideline should be used.
	3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.	2			The document broadly describes the target population as people with chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME). It does not address gender, age or comorbidities. It mentions disease severity and precautions. The guideline acknowledges that ‘subgroups within the spectrum of CFS/ME who are more vulnerable to more severe PEM’ may experience harm. However, the guideline does not indicate how to identify these patients and does not explicitly exclude them.
			11	44%
Stakeholder involvement	4. The guideline development group includes individuals from all the relevant professional groups.	4			Committee members were the HANDI project team ¹⁶. The committee membership was not referenced by the guideline and was difficult to find. None of the bios mention ME/CFS. Name, expertise and position were provided. Geographical location and member roles in the guideline development group were not provided.
	5. The views and preferences of the target population (patients, public, etc.) have been sought.	1			The views and preferences of the target population were not sought. The document dismisses the harm reported in response to graded exercise therapy by some patients with ME/CFS without providing evidence to support the dismissal. Patient reports were not adequately considered.
	6. The target users of the guideline are clearly defined.	1			The guideline does not include a description of the intended guideline audience or a description of how the guideline may be used.
			6	17%
Rigor of development	7. Systematic methods were used to search for evidence.	1			The guideline development process does not appear to have included a systematic search for evidence.
	8. The criteria for selecting the evidence are clearly described.	1			There is no description of the criteria for selecting the evidence.
	9. The strengths and limitations of the body of evidence are clearly described.	1			The evidence has not been evaluated for bias. There is no evaluation of study design, methodological limitations, relevance of outcome measures, consistency of results, considerations of benefits versus harm, or applicability to clinical practice.
	10. The methods for formulating the recommendations are clearly described.	1			There is no description of the recommendation development process, outcomes of the development process, or the way in which the development process affected the recommendations.
	11. The health benefits, side effects and risks have been considered in formulating the recommendations.	2			The guideline cites two research papers supporting benefits, one of which is a systematic review. The guideline mentions reports of harm. No supporting data for harms or risks are cited. Reports of harm are dismissed without providing supporting research. The discussion balance between harms and benefits is inadequate. The guideline development group has not considered the benefits and harms equally.
	12. There is an explicit link between the recommendations and the supporting evidence.	1			The guideline does not describe how the guideline development group used the evidence to inform recommendations. Recommendations are not linked to a key evidence description/paragraph and/or reference list. Recommendations are not linked to evidence summaries.
	13. The guideline has been externally reviewed by experts prior to its publication.	1			The guideline does not appear to have been externally reviewed before publication.
	14. A procedure for updating the guideline is provided.	1			No procedure for updating the guideline is provided.
			9	2%
Clarity of presentation	15. The recommendations are specific and unambiguous.	4			Recommended actions are described. The intent is described. The intended population is ‘people with chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME)’. The guideline does not take age, gender or comorbidities into account. Disease severity is discussed but the guideline does not give instructions in the case of severe disease or adverse reactions. The guideline suggests that the recommendations may not apply to some patients but does not explain how to identify those patients.
	16. The different options for management of the condition or health issue are clearly presented.	1			The guideline does not consider other options for management of the condition.
	17. Key recommendations are easily identifiable.	7			Key recommendations are easily identifiable.
			12	50%
Applicability	18. The guideline describes facilitators and barriers to its application.	2			Financial and systemic facilitators and barriers are not addressed. The guideline mentions attitudinal barriers. The methods by which this information was obtained are not described. There is no description of how the information influenced the guideline development process.
	19. The guideline provides advice and/or tools on how the recommendations can be put into practice.	2			The guideline does not provide a summary document, links to checklists, links to how-to manuals, tools to capitalize on guideline facilitators, outcome of pilot test or directions on how users can access tools and resources. The guideline does include some advice on addressing attitudinal barriers.
	20. The potential resource implications of applying the recommendations have been considered.	1			Cost was not addressed.
	21. The guideline presents monitoring and/ or auditing criteria.	1			The guideline does not provide criteria to assess guideline implementation or adherence to recommendations; criteria for assessing the impact of implementing the recommendations; advice on frequency and interval of measurement; or descriptions or operational definitions of how the criteria should be measured.
			6	8%
Editorial independence	22. The views of the funding body have not influenced the content of the guideline.	1			The guideline does not explicitly state the funding body or source of funding. There is no statement that the views of the funding body did not influence the content of the guideline.
	23. Competing interests of guideline development group members have been recorded and addressed.	7			Only one committee member declared competing interests. The competing interests had no relevance to ME/CFS.
			8	50.00%
Overall Guideline Assessment	Rate the overall quality of this guideline.			29.00%	Poor quality
Overall Guideline Assessment	I would recommend this guideline for use.		No	No	The guideline development group failed to systematically review the body of evidence. The guideline did not identify the strengths and limitations of the cited research studies. The guideline development process may not have been be impartial. The target population was not consulted.

Appendix B: NICE guideline scoresheet

The following scoresheet assesses the NICE guideline for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) ⁴using the AGREE II instrument for evaluating guidelines ²:

AGREE II Score Sheet – NICE
Please rate each of the 23 items from 1 to 7. where 1 is strongly disagree and 7 is strongly agree.
For guidance on scoring, see Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., Littlejohns, P., Makarski, J., Zitzelsberger, L., & AGREE Next Steps Consortium (2010). AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne, 182(18), E839–E842, downloaded from http://www.agreetrust.org. https://doi.org/10.1503/cmaj.090449

Domain	Item	AGREE II Rating	Domain Score	Domain %	Comments
Scope and purpose	1. The overall objective(s) of the guideline is (are) specifically described.	7			The document fully describes the purpose of the guidelines, expected benefits and target population.
	2. The health question(s) covered by the guideline is (are) specifically described.	7			The document describes the target population, interventions, comparisons, outcomes and health care settings.
	3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.	7			The document describes the target population, gender, age, comorbidities, disease severity, excluded populations, precautions and adverse effects.
			21	100%
Stakeholder involvement	4. The guideline development group includes individuals from all the relevant professional groups.	6			The guideline development group contains people from all relevant professional groups. Names, discipline/expertise, institution and committee role are published. Geographical location is not published. Location can be inferred from the institution of professional members, but not for lay members.
	5. The views and preferences of the target population (patients, public, etc.) have been sought.	7			The guideline describes the strategies used to capture the views and preferences of patients and the public. The guideline describes the information gathered and explains how the how the information gathered was used to inform the guideline development process and formation of the recommendations.
	6. The target users of the guideline are clearly defined.	7			The guideline provides a clear description of the intended guideline audience and a description of how the guideline may be used by its target audience.
			20	94%
Rigor of development	7. Systematic methods were used to search for evidence.	7			The guideline lists the search terms and databases searched; date of search; and search strategy.
	8. The criteria for selecting the evidence are clearly described.	7			Target populations, study designs, comparisons and outcomes are described.
	9. The strengths and limitations of the body of evidence are clearly described.	7			Committee used the GRADE process to evaluate the strengths and limitations of the body of evidence with regards to study design; study methodology limitations; and relevance of primary and secondary outcomes. Consistency and direction of results were considered. Risk of harm and applicability to practice were considered.
	10. The methods for formulating the recommendations are clearly described.	7			The document describes the guideline development process; the outcomes of the development process; and how the process influenced the guidelines.
	11. The health benefits, side effects and risks have been considered in formulating the recommendations.	7			The document includes supporting data and reports of benefits and harms. The document discusses the balance between benefits and harms. Recommendations reflect consideration of both benefits and harms.
	12. There is an explicit link between the recommendations and the supporting evidence.	7			The guideline describes how the guideline development group linked and used the evidence to inform recommendations. Each recommendation is linked to a key evidence section and reference list. Recommendations are linked to evidence summaries and evidence tables.
	13. The guideline has been externally reviewed by experts prior to its publication.	7			The draft guideline was open to review by experts and the general public via an online comments form. Commenters, comments and responses were published online. Some comments initiated changes to the guideline.
	14. A procedure for updating the guideline is provided.	7			NICE publishes a methodology for updating the guideline. Updates are conducted in response to comments and requests, or to changes in the medical environment, rather than at a fixed interval.
			56	100%
Clarity of presentation	15. The recommendations are specific and unambiguous.	7			The recommendations; intent of the recommendations; relevant population; and exclusions are clearly listed.
	16. The different options for management of the condition or health issue are clearly presented.	7			The guideline describes treatment options and the population or clinical situation most appropriate to each option.
	17. Key recommendations are easily identifiable.	7			Key recommendations are easily identifiable.
			21	100%
Applicability	18. The guideline describes facilitators and barriers to its application.	7			NICE document C identifies types of barriers and facilitators; describes the barriers and facilitators; describes how this information was collected; explains how the information influenced the guideline development process; and identifies strategies to address facilitators and barriers.
	19. The guideline provides advice and/or tools on how the recommendations can be put into practice.	5			Guideline summary available; baseline assessment tool provided; how-to manual provided; solutions linked to barrier analysis provided; no tools provided to capitalize on guideline facilitators; no pilot testing of recommendations, questionnaires and tools in ME/CFS. Information for health professionals is embedded in hundreds of pages of guideline and supporting documents. Apart from the baseline assessment tool, the guideline appears to lack user-friendly, concise, accessible information for GPs.
	20. The potential resource implications of applying the recommendations have been considered.	5			The resource impact summary predicts little change in costs, because the recommendations do not need any additional resources to implement. Responsibility was delegated to local authorities. It was expected that proper management of ME/CFS may have a marginal effect on initial costs, which are counterbalanced by the protective effects of more effective disease management. The committee was unable to calculate actual costs due to “a lack of robust data on current practice and the variation across organisations and services”. Hence, costs would need to be calculated at a local level.
	21. The guideline presents monitoring and/ or auditing criteria.	4			The Surveillance Assessment Process section of the Developing NICE Guidelines manual ¹² describes the data collected to assist in monitoring and auditing of guideline uptake. There appears to be no advice on the frequency and interval of measurement. The auditing guidelines appear to be general to NICE, rather than specific to the ME/CFS guidelines.
			21	71%
Editorial independence	22. The views of the funding body have not influenced the content of the guideline.	7			The guideline was funded by the National Institute for Health and Care Excellence. The guideline lacks a definitive statement regarding the level of influence of the funding body. However, much detail is provided on the literature search and evaluation of the evidence. The guideline contains a statement that the content was based on the body of evidence, and this is supported by documentation of the development group’s processes and the evaluations of the evidence.
	23. Competing interests of guideline development group members have been recorded and addressed.	7			NICE has a strict policy on declaring and managing interests.
			14	100%
Overall Guideline Assessment	Rate the overall quality of this guideline.			94%	The guideline is high quality, scoring well across most domains. The most apparent flaw is the paucity of tools for implementation by health practitioners.
Overall Guideline Assessment	I would recommend this guideline for use.		Yes	Yes

Appendix C: IOM guideline scoresheet

The following scoresheet assesses the IOM guideline for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) ³using the AGREE II instrument for evaluating guidelines ²:

AGREE II Score Sheet – IOM
Please rate each of the 23 items from 1 to 7. where 1 is strongly disagree and 7 is strongly agree.
For guidance on scoring, see Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., Littlejohns, P., Makarski, J., Zitzelsberger, L., & AGREE Next Steps Consortium (2010). AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne, 182(18), E839–E842, downloaded from http://www.agreetrust.org. https://doi.org/10.1503/cmaj.090449

Domain	Item	AGREE II Rating	Domain Score	Domain %	Comments
Scope and purpose	1. The overall objective(s) of the guideline is (are) specifically described.	7			The document fully describes the purpose of the guidelines, expected benefits and target population.
	2. The health question(s) covered by the guideline is (are) specifically described.	7			The document describes the target population, intervention, comparisons with other diagnostic criteria, outcomes and health care settings in which the guideline should be used.
	3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.	7			The document describes the target population, gender and age, clinical condition, severity/stage of disease, comorbidities and excluded populations.
			21	100%
Stakeholder involvement	4. The guideline development group includes individuals from all the relevant professional groups.	6			The guideline committee includes individuals from a broad array of professional backgrounds. The document details the following information about committee members: name; discipline/content expertise; institution. Geographical location is not stated but can be inferred from the committee member’s institution if required. There is no description of the committee members’ roles in the guideline development.
	5. The views and preferences of the target population (patients, public, etc.) have been sought.	7			The guideline describes the strategies used to capture the views and preferences of patients and the public. The guideline explains how the information gathered was used to inform the guideline development process and formation of the recommendations.
	6. The target users of the guideline are clearly defined.	7			The guideline provides a clear description of the intended guideline audience and a description of how the guideline may be used by its target audience
			20	94%
Rigor of development	7. Systematic methods were used to search for evidence.	7			Document lists the search terms and databases searched; date of search; and search strategy.
	8. The criteria for selecting the evidence are clearly described.	4			Target population and outcomes are described. Study designs and comparisons are not described.
	9. The strengths and limitations of the body of evidence are clearly described.	7			Committee used the GRADE process to evaluate the strengths and limitations of the body of evidence with regards to study design; study methodology limitations; and relevance of primary and secondary outcomes. The committee used discussion of the evidence to come to consensus. Risk of harm was considered in the context of over/underdiagnosis. Recommendations were considered to be applicable to practice context
	10. The methods for formulating the recommendations are clearly described.	7			The document describes the guideline development process; the outcomes of the development process; and how the process influenced the guidelines.
	11. The health benefits, side effects and risks have been considered in formulating the recommendations.	7			The document considers the health benefits of new diagnostic criteria and the risk over over/underdiagnosis. The document discusses the research evidence showing that patients struggle to obtain a diagnosis.
	12. There is an explicit link between the recommendations and the supporting evidence.	7			The guideline describes how research evidence was used to inform recommendations. The document cites research evidence for each ME/CFS symptom, describing the attributes of the symptom. The recommendations follow from the symptom descriptions. Each section contains a reference list.
	13. The guideline has been externally reviewed by experts prior to its publication.	7			The document’s Review section describes an independent review; the purpose of the review; the review method; the names and institutions of the reviewers; and the way the information obtained from the review was used to inform the final document.
	14. A procedure for updating the guideline is provided.	5			Guideline states that the diagnostic criteria should be revised within 5 years, suggests a funding body, and assigns the responsibility for the update to the HHS. The specified evaluation procedure is vague.
			51	90%
Clarity of presentation	15. The recommendations are specific and unambiguous.	7			The recommendations; intent of the recommendations; relevant population; and exclusions are clearly listed.
	16. The different options for management of the condition or health issue are clearly presented.	NA			The purpose of the guidelines was to create diagnostic criteria; not to define treatment options.
	17. Key recommendations are easily identifiable.	7			Key recommendations are easily identifiable.
			14	100%
Applicability	18. The guideline describes facilitators and barriers to its application.	7			The guideline identifies types of barriers and facilitators; describes the barriers and facilitators; describes how this information was collected; explained how the information influenced the guideline development process; and identified strategies to address facilitators and barriers.
	19. The guideline provides advice and/or tools on how the recommendations can be put into practice.	5			Guideline summary available; links to questionnaires and tools provided; links to how-to manuals provided; solutions linked to barrier analysis provided; no tools provided to capitalize on guideline facilitators; no pilot testing of recommendations, questionnaires and tools in ME/CFS. Tools and manuals hard to locate. The guideline delegates responsibility for implementation to the US Department of Health and Human Services.
	20. The potential resource implications of applying the recommendations have been considered.	1			Costings not considered. Responsibility placed on government agencies and professional bodies.
	21. The guideline presents monitoring and/ or auditing criteria.	3			The guideline contains a short section suggesting that uptake of the diagnostic criteria be monitored via website hits and conference attendee statistics. The section also suggests a research study evaluating attitudinal change in health care providers. The document does not suggest auditing of diagnostic accuracy.
			16	50%
Editorial independence	22. The views of the funding body have not influenced the content of the guideline.	7			The report lists sponsors in the preface. The preface and the recommendations for future updates suggest that the funding bodies did not influence the development of the guidelines. The document describes an evidence-based guideline development process that is also informed by the views of patients and advocates.
	23. Competing interests of guideline development group members have been recorded and addressed.	3			Professional experience and Involvement in ME/CFS-related activities are mentioned. However, competing interests are not specifically addressed.
			10	67%
Overall Guideline Assessment	Rate the overall quality of this guideline.			83%	The guideline received high scores for scope and purpose, stakeholder involvement, rigour of development and clarity of presentation. The IOM guideline fails to provide easily accessible tools to support health practitioners in implementing the diagnostic guideline. Competing interests not addressed. However, the guideline’s recommendations are well supported. The guideline is of high quality.
Overall Guideline Assessment	I would recommend this guideline for use.		Yes	Yes

AGREE II Assessment of RACGP Guideline for Incremental Physical Activity in ME/CFS (PDF)Download

Table of Contents

Overview

Introduction

The AGREE II Instrument

Assessment of the RACGP Guideline

Domain: Scope and purpose

Domain: Stakeholder involvement

Domain: Rigour of development

Domain: Clarity of presentation

Domain: Applicability

Domain: Editorial independence

Summary

Assessment of the NICE Guideline

Domain: Scope and purpose

Domain: Stakeholder involvement

Domain: Rigour of development

Domain: Clarity of presentation

Domain: Applicability

Domain: Editorial independence

Summary

Assessment of IOM Guideline

Domain: Scope and purpose

Domain: Stakeholder involvement

Domain: Rigour of development

Domain: Clarity of presentation

Domain: Applicability

Domain: Editorial independence

Summary

ME/CFS Guideline Comparison

Conclusion

References

Appendices

Appendix A: RACGP guideline scoresheet

Appendix B: NICE guideline scoresheet

Appendix C: IOM guideline scoresheet