Organizational readiness to change assessment (ORCA) tool

A summary of

Helfrich, C. D., Li, Y.-F., Sharp, N. D. & Sales, A. E. (2009). Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science, 4: 38. doi: 10.1186/1748-5908-4-38.

How to cite this NCCMT summary:

National Collaborating Centre for Methods and Tools (2013). Organizational readiness to change assessment (ORCA) tool. Hamilton, ON: McMaster University. (Updated 29 June, 2017) Retrieved from http://www.nccmt.ca/resources/search/187.

Categories: Tool, Implement, Organizational capacity and management, Organizational change

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Relevance for Public Health

This tool can be used to identify and monitor organizational strengths and weaknesses to support implementation of evidence-based practices. Although developed for clinical settings, the ORCA tool can be applied to support innovation in public health settings.

Description

This tool measures organizational readiness to implement evidence-based practices in clinical settings. The Organizational Readiness to Change Assessment (ORCA) instrument consists of three major scales that measure:

  • strength of the evidence for the proposed change/innovation;
  • quality of the organizational context to support the practice change; and
  • organizational capacity to facilitate the change.

The ORCA tool was developed from the Promoting Action on Research Implementation in Health Services (PARIHS) framework, a theoretical model to guide implementation of evidence-based interventions. The ORCA instrument operationalizes the constructs defined in the PARIHS framework.

The tool consists of 77 items, with subscales, grouped according to the main areas of the PARIHS framework:

  • Evidence: the nature and strength of the evidence and its potential for implementation (4 subscales)
  • Context: the environment or setting in which the proposed change is to be implemented (6 subscales)
  • Facilitation: capacity or types of support needed to help people change their attitudes, behaviours, skills and ways of thinking and working (9 subscales)
Implementing the Method/Tool
Steps for Using Method/Tool

The Organizational Readiness to Change Assessment (ORCA) tool consists of 77 items in the following scales:

1. Evidence Assessment:

  • Amount of discord within the practice team about evidence (the extent to which a respondent sees colleagues concluding a weaker or stronger evidence base than the respondent)
  • Research evidence
  • Clinical experience
  • Patient preferences

2. Context Assessment:

  • Senior leadership culture
  • Staff culture
  • Leadership behaviour
  • Measurement (leadership feedback)
  • Opinion leaders
  • General resources

3. Facilitation Assessment (to assess the organization's capacity for internal facilitation to support change):

  • Senior leaders practices
  • Champion characteristics
  • Leadership implementation roles
  • Implementation team roles
  • Implementation plan
  • Project communication
  • Project progress tracking
  • Project resources and context
  • Project evaluation
Who is involved

Any individuals who are involved in implementing an evidence-based practice, and using the ORCA tool, would be involved in administering and interpreting the tool.

Conditions for Use

Copyright © 2009 Helfrich et al; licensee BioMed Central Ltd. The ORCA tool is open to use without licensing permissions. Users should acknowledge the Quality Enhancement Research Initiative (QUERI) Program at the US Department of Veterans Affairs and alert the program when using the tool.

Evaluation and Measurement Characteristics
Evaluation
Has been evaluated.

The authors conducted two sets of psychometric analyses on data from three quality improvement projects conducted in the Veterans Health (VA) Administration between 2002 and 2006. In each project, the ORCA instrument was administered to staff. Two sets of psychometric analyses were conducted:

1) item analysis to determine if items within scales correlate as predicted (reliability)

2) exploratory factor analyses of aggregated subscales to determine how many underlying factors may be present, and their relationships to each other (validity)

Also, Hagedorn and Heideman (2010) tested the utility of the ORCA tool in a clinical setting, where the ORCA was used at baseline and after implementation of hepatitis prevention services in substance use disorders (SUD) clinics. This study provides preliminary support of ORCA as a measure of organizational readiness to change.

Validity
Validity properties meet accepted standards.

Exploratory factor analysis applied to the aggregated subscale scores support three underlying factors, with the majority of subscale scores clustered according to the core elements of the PARIHS framework. However, subscales measuring champion characteristics and availability of resources failed to load significantly on any factor, and the leadership practices subscale loaded on context rather than facilitation.

Reliability
Reliability properties meet accepted standards.

Reliability tests indicate that most subscales of the ORCA tool meet standard requirements of 0.80. Cronbach's alpha for reliability for the scales were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. However, reliability was poor for three evidence subscales.

Methodological Rating
Moderate
Method/Tool Development
Developer(s)

Christian D. Helfrich
Yu-Fang Li
Nancy D. Sharp
Anne E. Sales

Method of Development

The Veterans Administration's (VA) Ischemic Heart Disease Quality Enhancement Research Initiative developed the ORCA instrument to assess organizational readiness to implement evidence-based health care interventions.

Release Date
2009
Contact Person/Source

Christian D. Helfrich
Northwest HSR&D Center of Excellence
VA Puget Sound Healthcare System
Seattle WA

Department of Health Services
University of Washington School of Public Health
Seattle WA

email: christian.helfrich@va.gov

Resources

Title of Primary Resource
Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework
File Attachment
None
Web-link
Reference
Helfrich, C. D., Li, Y.-F., Sharp, N. D. & Sales, A. E. (2009). Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science, 4: 38. doi: 10.1186/1748-5908-4-38.
Type of Material
Journal article
Format
Periodical
Cost to Access
None.
Language
English
Conditions for Use
Copyright © 2009 Helfrich et al; licensee BioMed Central Ltd. The ORCA tool is open to use without licensing permissions. Users should acknowledge the Quality Enhancement Research Initiative (QUERI) Program at the US Department of Veterans Affairs and ale

Title of Supplementary Resource
The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: a case study
File Attachment
None
Web-link
Reference
Hagedorn, H. J. & Heideman, P. W. (2010). The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: a case study. Implementation Science, 5: 46. doi: 10.1186/1748-5908-5-46.
Type of Material
Journal article
Format
Periodical
Cost to Access
None.
Language
English
Conditions for Use
Copyright © 2010 Hagedorn and Heideman; licensee BioMed Central Ltd.

Have you used this resource? Share your story!