THE SYSTEMATIC LITERATURE REVIEW

The systematic literature review is a method/process/protocol in which a body of literature is

aggregated, reviewed and assessed while utilizing pre-specified and standardized techniques. In other
words, to reduce bias, the rationale, the hypothesis, and the methods of data collection are prepared
before the review and are used as a guide for performing the process. Just like it is for the traditional
literature reviews, the goal is to identify, critically appraise, and summarize the existing evidence
concerning a clearly defined problem.

Systematic literature reviews allow us to examine conflicting and/or coincident findings, as well as to
identify themes that require further investigation. Furthermore, they include the possibility of
evaluating consistency and generalization of the evidence regarding specific scientific questions and are,
therefore, also of great practical value within the psychological field. The method is particularly useful to
integrate the information of a group of studies investigating the same phenomena and it typically
focuses on a very specific empirical question, such as ‘Does the Rational Emotive Therapy intervention
benefit the well-being of the patients diagnosed with depression?’.

Systematic literature reviews include all (or most) of the following characteristics:

Objectives clearly defined a priori;

Explicit pre-defined criteria for inclusion/exclusion of the literature;

Predetermined search strategy in the collection of the information and systematic following of the
process;

Predefined characteristic criteria applied to all the sources utilized and clearly presented in the
review;

Systematic evaluation of the quality of the studies included in the review;

Identification of the excluded sources of literature and justification for excluding them;

Analysis/synthesis of the information (i.e., comparison of the results, qualitative synthesis of the
results, meta-analysis);

References to the incoherences and the errors found in the selected material.

The process of performing a systematic literature review consists of several stages and can be reported
in a form of an original research article with the same name (i.e., systematic literature review):
1: Start by clearly defining the objective of the review or form a structured research question.

Place in the research article: Title, Abstract, Introduction.

Example of the objective: The objective of this literature revision is to systematically review and analyse
the current research on the effects of music on the anxiety levels of children in hospital settings.

Example of a structured research question: What are the most important factors associated with the
development of PTSD in soldiers?

Tip: In the title, identify that the report is a systematic literature review.

2: Clearly specify the methodology of the review and define eligibility criteria (i.e., study selection
criteria that the published material must meet in order to be included or excluded from the study). The
search should be extensive.

Place in the research article: Methods.

Examples of inclusion criteria: Publication was an academic and peer-reviewed study. Publication was a
study that examined the effects of regular physical exercise intervention on depression and included a
control group.

Examples of exclusion criteria: Publication was involving male adults. Studies that also examined non-
physical activities as interventions. Studies that were only published in a language other than English.

Tips: The eligibility criteria sometimes fit to be presented in tables.

3: Retrieve eligible literature and thoroughly report your search strategy throughout the process.
(Ideally, the selection process is performed by at least two independent investigators.)

Place in the research article: Methods.

Example: The EBSCOhost and PsychInfo electronic databases from 2010 to 2017 were searched. These
were chosen because of the psychological focus that encompasses psychosocial effects of emotional
abuse in childhood. Search terms were ‘emotional abuse’, ‘childhood’, ‘psychosocial effects’, and
‘psychosocial consequences’. The EBSCOhost produced 200 results from the search criteria, while
PsychInfo produced 467, for a total of 667 articles. […] Articles were rejected if it was determined from
the title and the abstract that the study failed to meet the inclusion criteria. Any ambiguities regarding
the application of the selection criteria were resolved through discussions between all the researchers
involved.

Tip: Sometimes it is nice to represent the selection process in a graphical representation; in the form of
a decision tree or a flow diagram (check PRISMA).

4: Assess the methodological quality of the selected literature whenever possible and exclude the
articles with low methodological quality. Keep in mind that the quality of the systematic review depends
on the validity and the quality of the studies included in the review.

Place in the research article: Methods.

Examples of the instruments available for evaluating the quality of the studies: PEDro, Jadad scale, the
lists of Delphi, OTseeker, Maastricht criteria.

Tip: Present the excluded articles as a part of the selection process mentioned in step 3.

5: Proceed with the so-called characterization of the studies. Decide which data to look for in all the
selected studies and present it in a summarized way. If the information is missing in some specific paper,
always register it in your reports. (Ideally, the characterization of the studies is performed by at least
two independent investigators.)

Place in the research article: Results.

Examples of the information that should and/or could be collected for characterization of the literature:
authors, year, sample size, study design, aims and objectives, findings/results, limitations.

Tip: Sometimes results can be presented nicely in a form of a table depicting the main characteristics.
6: Write a synthesis of the results – integrate the results of different studies and interpret them in a
narrative form.

Place in the research article: Interpretation, Conclusions.

Patterns discovered as results should be summarized in a qualitative, narrative form. Modulate one (or
more) general arguments for organizing the review. Some trick to help you do this is to choose two or
three main information sources (e.g., articles, books, other literature reviews) to explain the results of
other studies through a similar way of organization. Connect the information reported by different
sources and do not just summarize the results. Find patterns in the results of different studies, identify
them, address the theoretical and/or methodological conflicts and try to interpret them. Summarize the
principal conclusions and evaluate the current state on the subject by pointing out possible further
directions.

CONCLUSIONS

The results emerging from the data that were included in such retrospective studies can lead to a certain
level of credibility regarding their conclusions. Actually, systematic literature reviews are thought to be
one of our best methods to summarize and synthesize evidence about some specific research question
and are often used as the main ‘practice making guidelines’ in many health care disciplines. Therefore, it
is no wonder why systematic reviews are gaining popularity among researchers and why journals are
moving in this direction as well. This also shows in the development of more and more specific
guidelines and checklists for writing systematic literature reviews (see for example PRISMA or Cochrane
Handbook for Systematic Reviews of Interventions). To find examples of systematic literature review
articles you can check Cochrane Database of Systematic Reviews, BioMed Central’s Systematic Reviews
Journal, and PROSPERO. If you are aware of the concept of ‘registered reports’, it is worth mentioning
that submitting with PROSPERO provides you with the option of publishing the latter as well. I suggest
that you go through the list of useful resources provided below and hopefully, you can get enough
information about anything related that remained unanswered. Now, I encourage you to try to be a little
more to be systematic whenever researching some topic, to try to write a systematic literature review
yourself and to maybe even consider submitting it to JEPS.

A systematic review is a highly rigorous review of existing literature that addresses a clearly formulated
question. The review systematically searches, identifies, selects, appraises, and synthesizes research
evidence relevant to the question using methodology that is explicit, reproducible, and leads to
minimum bias. Systematic reviews are regarded as the best source of research evidence. Systematic
reviews are absolutely crucial in the field of evidence-based medicine, but are also highly valued in other
fields.
A systematic review is more exhaustive than a literature review as it includes both published and
unpublished literature, often called grey literature. Grey literature is a significant part of a systematic
review and adds value to the review. This is because grey literature is often more current than published
literature and is likely to have less publication bias. Grey literature includes unpublished studies, reports,
dissertations, conference papers and abstracts, governmental research, and ongoing clinical trials.

Conducting a systematic review is a complex process. This article aims to guide you on the different
kinds of systematic review, the standard procedures to be followed, and the best approach to
conducting and writing a systematic review.

Course

How to conduct a literature search and review

This course provides quick and useful tips to become…

5k views

Types of systematic reviews

Qualitative: In this type of systematic review, the results of relevant studies are summarized but not
statistically combined.

Quantitative: This type of systematic review uses statistical methods to combine the results of two or
more studies.

Meta-analysis: A meta-analysis uses statistical methods to integrate estimates of effect from relevant
studies that are independent but similar and summarize them.

Writing a protocol

Any good systematic review begins with a protocol. According to the National Institutes of Health (NIH),
a protocol serves as a road-map for your review and specifies the objectives, methods, and outcomes of
primary interest of the systematic review. The purpose of having a protocol is to promote transparency
of methods.
A protocol defines the search terms, inclusion and exclusion criteria, data that will be analyzed, etc. The
protocol needs to be submitted to the journal along with your manuscript. Most journals expect authors
of systematic reviews to use the PRISMA statement or similar other guidelines to write their protocol.

The PRISMA Statement:

Anybody writing a systematic literature review should be familiar with the PRISMA statement. The
PRISMA Statement is a document that consists of a 27-item checklist and a flow diagram and aims to
guide authors on how to develop a systematic review protocol and what to include when writing the
review.

A protocol ideally includes the following:

Databases to be searched and additional sources (particularly for grey literature)

Keywords to be used in the search strategy

Limits applied to the search.

Screening process

Data to be extracted

Summary of data to be reported

Registering systematic review protocols:

Once you have written your protocol, it is advisable to register it. Registering your protocol is a good
way to announce that you are working on a review, so that others do not start working on it.

What is the best approach to conducting a systematic review?

The essence of a systematic review lies in being systematic. A systematic review involves detailed
scrutiny and analysis of a huge mass of literature. To ensure that your work is efficient and effective, you
should follow a clear process:
1. Develop a research question

2. Define inclusion and exclusion criteria

3. Locate studies

4. Select studies

5. Assess study quality

6. Extract data

7. Analyze and present results

8. Interpret results

9. Update the review as needed

It is helpful to follow this process and make notes at each stage. This will make it easier for you to write
the review article.

How is a systematic review article structured?

A systematic review article follows the same structure as that of an original research article. It typically
includes a title, abstract, introduction, methods, results, discussion, and references.

Title: The title should accurately reflect the topic under review. Typically, the words “a systematic
review” are a part of the title to make the nature of the study clear.
Abstract: A systematic review usually has a structured Abstract, with a short paragraph devoted to each
of the following: background, methods, results, and conclusion.

Introduction: The Introduction summarizes the topic and explains why the systematic review was
conducted. There might have been gaps in the existing knowledge or a disagreement in the literature
that necessitated a review. The introduction should also state the purpose and aims of the review.

Methods: The Methods section is the most crucial part of a systematic review article. The methodology
followed should be explained clearly and logically. The following components should be discussed in
detail:

Inclusion and exclusion criteria

Identification of studies

Study selection

Data extraction

Quality assessment

Data analysis

Results: The Results section should also be explained logically. You can begin by describing the search
results, and then move on to the study range and characteristics, study quality, and finally discuss the
effect of the intervention on the outcome.

Discussion: The Discussion should summarize the main findings from the review and then move on to
discuss the limitations of the study and the reliability of the results. Finally, the strengths and
weaknesses of the review should be discussed, and implications for current practice suggested.

References: The References section of a systematic review article usually contains an extensive number
of references. You have to be very careful and ensure that you do not miss out on a single one. You can
consider using reference management software to help you tackle the references effectively.

Personalized Medicine in Systemic Lupus Erytemathosus

Genetic profiling and Precision Medicine in Systemic Lupus Erythematosus