NURS 4113 NURSING RESEARCH
Checklists
We have identified a practice problem, formulated a PICO question, and found articles, referred to as "evidence". In this part of EBP, we decide which articles we consider "good enough" to include in our next phase of evaluating and synthesizing the evidence.
Why must we "appraise" the evidence? Unfortunately, publication of a study or clinical investigation in a scientific journal does not truly guarantee that the results are true. A striking example is the article published in the prestigious journal THE LANCET. The article is published here:
(https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31180-6/fulltext), by Mandeep Mehra et al (2020). Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.
This article ensured that hydroxychloroquine reduced the death rate of Covid-19 patients. However, the study was highly questioned by the scientific community due to a lack of validity of the study, which made doubtful the results shown. Thus, after an independent audit, and after finding the presence of error or important biases in the methodology utilized to conduct the study, the article was retracted from the journal.
Questions applicable to any article, are:
- Validity (How was the study done?)
- Importance (What were the results?)
- Applicability (Will it help me to care for my patient?)
Before appraising the evidence, you need to learn to read the article. While the title and abstract may tell you whether it is suitable research, applicable to your population, you often must read the article. The table below gives an example how to organize initial reading of the articles.
In the table, you will find the link to the article. The article is annotated (highlights and comments). The article notes information in the comments, which may help me select articles for appraisals during the next stage.
| Link to Article | Topic | Comments |
|---|---|---|
| Randomized Controlled Trial | Turning patients in a long-term care facility with a high and moderate Braden pressure ulcer risk 2, 3, or 4 hourly makes a difference in developing pressure ulcers | Identify independent variable; dependent variables Population and Sampling; Findings - which are significant, and those which are not significant; null hypothesis: which rejected; which not rejected. Note this is a 2014 article - before using this article as evidence, we will search for more updated research studies. |
| Quantitative Research | How do patients experience selective chronic care illnesses? | Abstract states "semi-structured interviews". Patients requested to complete a self-administered questionnaire of closed questions. After each question, patients were asked to give their opinions/comments after each question. Descriptive study: due to response rate 43%, the sample size did not meet the requirements to do statistical analysis and test hypotheses. |
| Systematic Review | How does nurse skill mix impact on nurse-sensitive patient outcome indicators? |
Authors did a quantitative review of articles published over a 18 year period. The authors used Joanna Briggs institute criteria to select the articles. They could not do a meta-analysis, as most of the studies gave narrative comments, and many of the outcomes (such as pain) were self-reported, either by the patient or the nurse. Outcomes such as falls and pain were not significantly correlating with nurse skill mix, contrary to a common belief. It did correlate with outcomes such as length of stay, sepsis outcomes. The authors concluded research is required to explore what happens during periods of low nursing skill mix; who provides care relating to nurse-sensitive patient outcomes during this time, what care is provided, and the quality of care documented. |
| What are nursing students' knowledge and attitudes towards end-of-life care and how can nursing care be optimized? | The authors identified observational research studies and pooled the results (statistical analysis of findings in individual studies). Followed PRISM criteria to identify studies to include. This meta-analysis describes how the quality of the individual studies was evaluated to include studies with high quality. This was a large meta-analysis, with 9749 participants across 13 countries. This meta-analysis indicated while student nurses have positive attitudes towards giving care to patients during end-of-life, they lack the knowledge. Recommendations include providing simulations of end-of-life care to student nurses |
|
| Meta-synthesis | What is nursing autonomy? | The authors synthesized qualitative studies on the concept of nursing autonomy. 13 studies were reviewed, and the concept of autonomy was described as represented in professional competence, decision-making, and interactions. It is unclear whether they selected the original studies from the Databases, or whether they used the reference list of a doctoral dissertation. Some information on how the search was conducted is repetitive, while some information is unclear. |
Watch this video on how to read a research paper https://youtu.be/eSEP2T-xz8g?si=n67NkkkY6mslSLcu
Rapid Critical Appraisal
After finished reading the articles to have a broad overview, continue with the appraisal and evaluation of the evidence. Decide if an article is worth to be kept in the body of evidence. Check the article against your PICO question and see what the research question was (what did the researcher want to achieve?). What were the findings and recommendations? Are those applicable to your PICO question and area of interest? Are there anything contradicting the use of this evidence, e.g., organizational policies, financial feasibility, and patient preferences. Continue with asking:
- What was the purpose of the study? (What was the research question(s) and hypotheses? Does the study answer the PICO question? Example: the PICO is about the use of pressure relief devices to prevent pressure injuries. An article about determining the patient's nutritional status, as nutrition is a variable in the development of pressure ulcers is not appropriate to analyze further)
- What is the Level of Evidence? (The type of research design determines the Level of Evidence. The more rigorous a design, the higher the Level of Evidence. Sometimes a project group/institution will set a minimum level of evidence to include in the report (the next section describes the Levels of Evidence).
- Quality of the evidence: Does the design adheres to the requirement? Are the sample participants similar to the population defined in the PICO question? Is the sample size adequate for the study design? (It is sometimes difficult to determine whether the sample size is adequate. Some designs require a bigger sample size than others. Do the authors explain how the sample size was determined?)
Example: In a qualitative study, a researcher interviewed 9 participants. Is this a valid sample size? In qualitative studies it is common to see small sample size. In a cohort study aiming to include 209 participants, the researchers included 154 participants - no reason is given why the required sample size was not included. This may be considered as an inadequate sample size. - Data-analysis: Is the statistical analysis explained? Is the statistical analysis appropriate for the study?
Example: A researcher describes a non-experimental, descriptive study of patients' experiences at an Urgent Care Clinic. The researcher continues describe rejecting a null hypothesis stating older patients were not treated within one hour after arriving at the clinic, versus younger patients, being attended to within one hour after arriving at the clinic. This may be considered as inappropriate statistical analysis.
Based on these questions, it is likely some of the articles will be excluded from the body of evidence. Further review of evidence is based on a structured checklist. In this course, we use the Critical Appraisal Checklists. There is a different checklist for every type of research. Bas). Accessible at https://casp-uk.net/ ©CASP this work is licensed under the Creative Commons Attribution – Non-Commercial- Share A like. To view a copy of this licence, visit https://creativecommons.org/licenses/by-nc-sa/4.0/
Watch this video explaining how to use a CASP checklist. In the video they use a CASP checklist for a systematic review https://youtu.be/ZO_DEPL3dgE?si=EH9z8nxhYW6IClR3
Level of Evidence
The Level of Evidence is based on the research design. Research designs with more control over variables (things that can affect the outcome) have a higher level of evidence. Generally, we want to include articles with the highest level of evidence. The Level of Evidence is usually presented in a pyramid hierarchy with the research having the strongest rigor and credibility at top, and the evidence being less rigorous and credible towards the bottom. Some hierarchies have 5 levels, others may have more levels. It is important when analyzing evidence to cite the hierarchy used.
We use an evidence hierarchy (pyramid) based on 8 levels. The highest Level is Level I, and the lowest Level is Level VIII.
Polit, D. & Beck, C. (2021). Lippincott CoursePoint Enhanced for Polit’s Essentials of Nursing Research (10th ed.). Wolters Kluwer Health in https://uen.pressbooks.pub/ebpresearchmethods/back-matter/appendix-f2-level-of-evidence-pyramid/
In Nursing, similar to other human and social sciences, there are fewer quantitative studies due to the nature of the discipline. The evidence in nursing is inevitably graded at a lower level. The "best available" evidence may be a Level V, VI, or VII level of evidence. Note Level VIII evidence. This is referred to as nonresearched sources of evidence, and thus unlikely to be used as evidence. While Level VIII evidence includes expert opinion (and EBP includes expert opinion), it will not be included as evidence if there is no scientific evidence supporting the clinical specialist's opinion.
Other types of evidence
- Cohort and case-control studies (also referred to as correlational studies) are considered Level V studies on the 8-Level of Evidence Hierarchies.
- While textbooks can provide information, that information is not considered research evidence (although it may refer to research studies). The textbook will not be included as part of the evidence.
- Many peer-reviewed, scholarly articles that do not report on a research study are published. Most of these are described as literature reviews or narrative reviews. These articles may refer to other research articles. These are not research articles (and will be viewed as Level VIII nonresearch sources).
- Clinical guidelines, position papers, and protocols: While these are based on research, they do not count as research. Thus, on the Level of Evidence, they will count as Level VIII evidence. However, if there is no restriction within a team to limit the type of evidence used, a solid, well-developed clinical guideline, position paper or protocol can be an excellent evidence resource and has feasible practice recommendations. These documents may be more valuable than a research study with flaws in the study design! Many of these guidelines and position papers are developed by nursing professional organizations (such as American Association for Critical Care Nurses), safety organizations (such as Joint Commission International, Agency for Healthcare Research and Quality, Center for Disease Control, World Health Organization).
Evaluation of the Evidence
At this time, you must start comparing the evidence, and determine the similarities and differences in the evidence. This is usually done by presenting an evidence table.
| Source | Question | Study design, population and sampling, expected outcome measures |
Level of Evidence Reference and number of levels |
Study Findings and Recommendations |
|---|---|---|---|---|
| Reference according to APA 7th Ed. (Authors and date) |
Note the research question/ purpose of the research. A PICO(T) or PICOS stated |
Type of study: report important aspects of design as documented in evidence. Include population and sampling (method and size), interventions, independent and dependent variables, hypotheses, data gathering tools, and data analysis. Identify any deviations from the expected findings for the type of research design. Specify if specific criteria are used to evaluate any aspect of the study, e.g., selection of articles in a systematic review, meta-analysis, or meta-synthesis. Identify any limitations (shared by authors and identified by a reviewer that could compromise the quality of the study. |
Specify the Level. Allocate the level of evidence based on the research design. |
Report as applicable to the study design. *Hypotheses - rejected/not rejected. *Level of significance *Themes/subthemes Report statistics answering the clinical question. Report recommendations that could apply to the PICO question |
Synthesis of Evidence
During this phase, we pull out information from the evaluation of evidence to provide a "snapshot" of the evidence.
Some authors recommend making a summary table which is helpful for the synthesis of the evidence. This table summarizes the evidence of 8 articles regarding the outcome of music therapy. From this summary, it is easy to see that 6/8 studies reported an improved oxygen saturation, 4/4 studies reported a decrease in respiratory rate - both positive outcomes. Improved parental engagement was noted in 2/5 studies, in 3 studies parental engagement was not evaluated and in 3 studies, there was no difference.
Fineout-Overholt, Ellen PhD, RN, FNAP, FAAN. A guide to critical appraisal of evidence. Nursing Critical Care 14(3):p 24-30, May 2019. | DOI: 10.1097/01.CCN.0000554830.12833.2f
Recommendations
At the end of the critical appraisal, the reviewer will identify recommendations to be implemented as EBP, based on the synthesized evidence.
Recommendations are either to change practice or, if the evidence is not convincing, to continue current practice until more evidence becomes available.
Knowledge Check
Answer the questions in the self-paced questionnaire. These questions assess if you have mastered the content. You can repeat the questionnaire as many times as you like. Just click on the "reset button"