Robinson, K.A., Saldanha, I.J., & Mckoy, N.A. (2011). Frameworks for determining research gaps during systematic reviews. Methods Future Research Needs Report No. 2. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. HHSA 290-2007-10061-I). AHRQ Publication No. 11-EHC043-EF. Rockville, MD: Agency for Healthcare Research and Quality.
Relevance for Public Health
This framework for identifying research gaps from systematic reviews can direct research agendas to influence future public health policy and practice. Although primarily developed for clinical settings, this framework could be used to identify research gaps from systematic reviews on public health interventions and develop research agendas to address these gaps.
This technical report, developed by the Agency for Healthcare Research and Quality (AHRQ), outlines a framework for identifying research gaps from systematic reviews. The framework identifies research gaps by examining Evidence-based Practice Centers and organizations that conduct systematic reviews. The framework specifies where and why the current evidence is lacking.
Although there are rigorous methods for conducting systematic reviews, there has not been a systematic process for identifying research gaps when developing the future research sections. As part of a process designed to develop guidance for Evidence-based Practice Centers (EPCs) to use to generate technical reports and evidence reports for the AHRQ, the AHRQ asked EPCs to respond to seven questions about meeting research needs.
The developers of this technical report sought to answer the question:
- What are the various frameworks, concepts and principles used to determine research gaps within a systematic review?
A research gap is defined as a topic or area for which missing or insufficient information limits the ability to reach a conclusion for a question. A research need is defined as a gap that limits the ability of decision-makers (policy-makers, patients, practitioners) from making decisions.
This technical report found that there is no specific process for identifying research gaps during systematic reviews. Organizations most commonly used variations of the PICO (population, intervention, comparison, outcomes) framework, which is proposed as a framework for identifying research gaps from systematic reviews.
Accessing the Method/Tool
Implementing the Method/Tool
Time for Participation/Completion
Information not available
Additional Resources and/or Skills Needed for Implementation
Steps for Using Method/Tool
The proposed framework includes two major components:
- identification and classification of the reasons why the research gap exists
- characterization of the research gap using the PICOS (population, intervention, comparison, outcomes, setting) elements
A) Identifying reasons for the existence of the research gap:
- Choose the most important reason(s) for the existence of the research gap
- Select the reason(s) that prevent conclusions about the evidence from being made
- Classify the reasons for research gaps, including:
- insufficient or imprecise information
- biased information
- inconsistency or unknown consistency
- not the right information
B) Characterizing research gaps:
- Use the PICOS framework to characterize research gaps related to interventions, screening tests, etc. The framework organizes research gaps as follows:
1. Population (P): information regarding the population that is not adequately represented in the evidence base (gender, race/ethnicity, age, etc.)
2. Intervention (I): information regarding the specific intervention that is inadequately included in the evidence base, the duration of the intervention, etc.
3. Comparison (C): lack of information regarding the comparison intervention or standard intervention
4. Outcomes (O): information regarding outcomes of interest, organized by type of outcome or timing of outcomes, to delineate where information is lacking
5. Setting (S): information regarding the relevant settings for research gaps
A worksheet is provided on p. 20 to identify and organize research gaps from systematic reviews.
Who is involved
Individuals and groups involved in conducting evidence reviews and systematic reviews, or those using the results of systematic reviews, would benefit from the proposed framework.
Conditions for Use
Evaluation and Measurement Characteristics
Information not available
Karen A. Robinson
Ian J. Saldanha
Naomi A. McKoy
Method of Development
The developers contacted 12 Evidence-based Practice Centers (EPCs) associated with the Agency for Healthcare Research and Quality (AHRQ) in the U.S. and Canada, and 64 other organizations internationally that conduct systematic reviews, cost-effectiveness analyses or technology assessments. Based on feedback from four EPCs and three other organizations, the authors developed and refined the proposed framework. In general, there is no specific process for identifying research gaps during systematic reviews. Organizations most commonly used variations of the PICO (population, intervention, comparison, outcomes) framework.
The developers used this six-step process to develop a framework to identify research gaps:
- Step 1: Focused literature review
- Step 2: Review of current practices of evidence-based practices (EPCs), Agency for Healthcare Research and Quality (AHRQ)
- Step 3: Review of current practices of organizations involved with evidence synthesis
- Step 4: Development of framework
- Step 5: Pilot testing framework
- Step 6: Refining the framework
The Agency for Healthcare Research and Quality (AHRQ)
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.