In my early years as a healthcare epidemiologist, there was a fair amount of conversation initially about the Keystone project by Peter Pronovost demonstrating an impressive reduction in central line-associated bloodstream infections in 108 intensive care units in Michigan hospitals. Soon, that was followed by a similar amount of conversation about the controversy generated by Office of Human Research Protections looking into whether Johns Hopkins Institutional Review Board was correct in judging the project to be not research. Following the hue and cry that ensued from quality improvement professionals and hospital administrators, OHRP concluded that the project was organizational research and quality improvement and not human subjects research, and acquitted the Johns Hopkins IRB of any wrongdoing. In response to this event, two articles in the New England Journal of Medicine took contrasting perspectives. According to Miller and Emanuel, “Study not exempt as it was prospective”, “Should have been reviewed by IRB via full or expedited review”. Per Mary Ann Baily, “Sophisticated IRB had difficulty with interpretation of regulations is a bad sign in itself. Project was a combination of QI and research on organizations.”
We have a lot more clarity on this topic in recent years and I thought it’s a good time to review this and help readers looking to learn more. Joanne Lynn et al. define QI as a “structured, data-guided form of the innovation and adaptation that has always been part of normal health care operations, and it has proven to be effective in improving health care”. In a discussion on the ethics of quality improvement in healthcare, they argue that “because QI activities are, and should be, part of normal health care operations, patients also have a responsibility to participate in quality improvement, which often requires direct cooperation”. On the other hand, the Code of Federal Regulations 45 CFR 46.102 (d) defines research as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy, whether or not they are conducted or supported under a program which is considered research for other purposes.” The Belmont principles for proper conduct of research are Respect for human participants in research through voluntary participation and a right to informed consent, Beneficence (evaluation of risk vs. benefit) and Justice (fair distribution of risks and benefits. Today, we have access to OHRP Decision trees to help us determine if a project is research or non-research, exempt research, expedited research, or full board review research. The OHRP webpage also has FAQs on QI activities – see links below.
A project may require waiver or alteration of written informed consent, and/ or a waiver or alteration of HIPAA authorization. For this, the principal investigator needs to provide justification for why the study poses no more than minimal risk to the human subjects, why the study does not have an adverse effect on their rights and welfare, and why it is not possible to do the study without a waiver or alteration. For example, a retrospective study based on chart review may fit these criteria if proper protections are followed while handling the data. Verbal consent may still be needed even if written informed consent is waived. Subjects may be given additional information after participation even if waiver is obtained. Studies that are not covered by IRB are hospital or community outbreaks and other public health investigations (covered by Health and Safety code), organizational research and quality improvement work (in contrast to human subjects research), and any study that does not involve human subjects (e.g., basic science research in a laboratory).
The following table outlines the differences between quality improvement and research.
|Hypotheses||Change Hypotheses||Null Hypotheses to be tested if true|
|Intent||To Improve Care||To Create Generalizable Knowledge|
|Measurement||Rigorous, but real world issues;||Rigorous with multiple inclusion and exclusion criteria both for accuracy and ‘enrichment of patient population’; Tests for confounding factors and interaction terms|
|Study Design||Usually simple Pre and Post- Intervention||Experimental; Simple or Complicated;|
|Steps||Aim statement, decide metric, Plan-Do-Study-Act||Hypotheses, Specific Aims, Methods, Statistical Analyses, Results and Conclusions|
|Basis||Based on organizational need||Based on prior studies or observations|
|Obligation||Part of clinical care||Not usually part of routine clinical care. Exceptions – Pragmatic Clinical Trials; Clinical Effectiveness Studies|
|Reproducibility of methods and results||Not emphasized||Generally emphasized|
For example, when Ignaz Semmelweiss implemented use of chlorine handwashing prior to patient care in the postpartum wards in the 1840s, his intent was to improve patient care (to prevent mortality), but he was not seeking to create generalizable knowledge and he did not have an experimental study design and he may or may not have thought that the findings or methods might be reproducible, to the extent we know. That being said, he used an intervention that was experimental at that time. If similar work were to be carried out in today’s healthcare settings and there are questions about whether the work constitutes quality improvement or research, and whether IRB approval needs to be obtained, most academic medical centers have policies and procedures to address this question. While some centers have a QI research committee, others handle the issues through a checklist of questions to differentiate QI from research, and if the criteria for research are not met, the research office or the IRB may have a process in place to issue a letter to the principal investigator stating that the study does not constitute research. When there is doubt, the better part of valor may be to request an IRB review. Publication alone does not make a study research as the purpose of any publication is to communicate to the community at large. In summary, it is possible to distinguish QI from research in many cases although it’s a complex issue.
- Miller FG, Emanuel EJ. Quality Improvement Research and Informed Consent. N Engl J Med 358;8 Feb 21, 2008
- Baily MA. Harming Through Protection? N Engl J Med 2008;358:768-769.
- Lynn J et al. The Ethics of Using Quality Improvement Methods in Health Care. Ann Intern Med. 2007;146:666-673.
- OHRP Decision trees: http://hhs.gov/ohrp/regulations-and-policy/decision-charts/index.html Accessed 10/21/2016
- OHRP FAQs on QI Activities: http://hhs.gov/ohrp/regulations-and-policy/guidance/faq/quality-improvement-activities/index.html Accessed 10/21/2016
A special thank you to Robert Haley, MD for his comments and to Susan Partridge, BSN, MBA for comments and some of the references.