GRASP

Systematic reviews in the Social Sciences - what I learnt from the Campbell Collaborations online course

March 18th, 2026, by Kitty Delaney Tag(s): Research methods, Literature reviews

In this post, I share my key takeaways from the Campbell Collaboration’s Systematic Review and Meta-Analysis online course, plus an overview of what’s covered.

Systematic reviews are increasingly being used in social science research, and I have noticed more social science HDR students asking about them. I wanted to increase my own understanding of the method and find some training that wasn’t written solely for medical and health researchers.

I was pleased to discover that the Campbell Collaboration Training Group has developed a free online course hosted on Carnegie Mellon University’s Open Learning Initiative. The course exists to address a gap in systematic review training materials for disciplines outside medicine and health. This course is especially useful for HDR students in the social sciences who are considering evidence synthesis as part of their research.

Course at a glance

  • Provider: Campbell Collaboration Training Group
  • Platform: Open Learning Initiative (Carnegie Mellon University)
  • Date: 2023
  • Cost: Free
  • Access: https://oli.cmu.edu/jcourse/webui/guest/join.do?section=srma
  • Format: Self-paced online modules with “learning by doing” activities
  • Assumes: Prior graduate training in research methodology and statistics recommended
  • Best for: HDR students and researchers who want a step-by-step overview of systematic review methods in the social sciences

My takeaways

I found the course genuinely useful. It provided me with an understanding of what a protocol is (think of it as your plan or roadmap) and why it matters. It also reinforced the team-based nature of systematic reviews—especially the value of having more than one person involved in developing the protocol, drafting and testing the screening guide, and screening material for inclusion or exclusion.

One thing that surprised me was just how iterative systematic reviews can be. As you learn more about the literature, you may need to refine aspects of your methods—particularly while you’re testing the protocol and screening questions. The crucial part is to document what has changed, why it changed, and when. Transparency and documentation are key.

I came away with a clearer understanding that systematic reviews can be conducted across qualitative, quantitative, or mixed methods studies. While I only have a very basic understanding of meta-analysis and why it’s used to compare results across quantitative studies, it was interesting to gain some knowledge around the topic. My colleague Claire Hulcup has promised to write a future Ideas Hub post on meta-analysis.

Much of the advice in the course is useful beyond a systematic review. The approach to planning your review, the documentation of changes and decisions you make regarding your approach as well as your criteria for including or excluding research material is equally important if you are undertaking traditional literature reviews.

One helpful reminder from doing the course is that your research can lead to multiple research outputs, not just a journal article, for example:

  • A technical report, which fully explains how the review was conducted and presents all the results, can be published as a Campbell systematic review or a Cochrane review, or can be made publicly available on a website, perhaps in connection with another publication.
  • A summary report, often the length of a journal article, explains the main steps and findings of a systematic review and meta-analysis. If published as a journal article, supplementary files may be hosted on the journal website to support expectations regarding transparency and full reporting.
  • A plain language summary (PLS), written for people who are not researchers, avoids statistical terms and technical jargon. This can be useful when sharing your research with industry or academics in other disciplines.
  • Conference presentations and working meetings with stakeholders offer other excellent opportunities for discussion of the processes and results of systematic reviews and meta-analyses.

I didn’t expect to enjoy it as much as I did. Although it is assumed that you have some prior training in research methodology, I found it easy to follow and to understand. It was a significant time commitment, occasionally challenging and it left me with a deeper respect for researchers who commit to doing a systematic review.

The course structure

Because the course is self-paced, it’s possible to dip in and out at different times. I found it easier to stay on track when I scheduled a regular time each day to complete a module. Content-wise, it walks through the steps involved in conducting a systematic review of the results from multiple quantitative studies.

There are 8 units with a total of 40 individual modules to be completed:

  1. Introduction
  2. Problem Formulation
  3. Searching the Literature
  4. Screening Potentially Eligible Studies
  5. Data Extraction and Coding
  6. Introduction to Effect Sizes
  7. Introduction to Meta-Analysis
  8. Course Wrap-up

Each module includes a “learning by doing” section. These are usually multiple-choice questions and/or a question requiring a brief description that test your understanding of the module. This reinforces key concepts and check understanding as you go.

After dithering for a while, I committed to spending an hour a day until I’d finished the modules—and that worked well. If you leave too long between modules, it’s easy to lose momentum (and forget some of the terminology), so shorter, regular sessions helped.

The content

The course provides practical skills for conducting systematic reviews, focusing on methods used in social sciences and is intended for both independent study and integration into formal instruction. The systematic review process is described as a series of successive steps where each step should always inform the next actions taken.

Diagram showing progression through the successive steps of a systematic review process

Image Source: Campbell Collaboration

There is an assumption that anyone doing the online course will have prior graduate training in research methodology and statistics. This made the latter half of the course dealing with statistics and meta-analysis a bit more challenging for me, but overall, it was very informative and I am pleased that I completed it.

Course introduction

There is a lot of valuable information in the introduction including a good description of a systematic review, the types and the strengths and limitations of systematic reviews.

Problem formulation

The “Problem Formulation” module is very insightful and has given me an appreciation for constructing answerable questions. This is very relatable and important to the whole process of being able to find information and resources. Having a well-defined research question is crucial. Determining the scope of the review and writing a protocol (that is a plan of action or roadmap of the review) is crucial to the success of conducting a systematic review. This can be an equally useful approach when conducting a database search or any other type of literature review.

Searching the literature

The section on “Searching the Literature” is extremely well done and gives an excellent overview and introduction to search processes. In the section on publication bias the authors noted that “An exhaustive literature search is the foundation of any systematic review, and if this stage is not well done then what follows is not very trustworthy”.

The authors include a section on translating the search across different databases and note that you may need to contact the authors or researchers of individual studies to get additional information to inform your analysis.

I found the section on grey literature informative. As we progressed through the course a section on publication bias outlined why it is important that grey literature is included. In short, publication bias or “small study bias” can occur when the small studies that happen to observe large enough effects will tend to get published, while small studies that do not observe large enough effects will tend to go unpublished. A robust search of the grey literature may identify studies that have not been published but can still be relevant to your topic.

The unit also includes a module on “Supplementary searching” and “Running, documenting and reporting searches”. Both of which are very useful to non-systematic reviews.

Screening potentially eligible studies

The unit on “Screening Potentially Eligible Studies” is fascinating, and the process is well explained. This process can be very useful to apply when you are evaluating the results of a database search. The use of more than one person to screen for eligible studies

Data extraction, effect sizes, and meta-analysis

I skimmed the “Introduction to Effect Sizes” and the “Introduction to Meta-analysis” modules. I will leave that to those of you who have more statistical knowledge than me and more use for an in-depth overview.

Course wrap-up

I found the Course Wrap-up one of the most useful modules in the course. It brought together the components of a systematic review and how researchers can disseminate their research topic via different “products” or outputs. It highlighted the fact that as an author your research has the potential to provide much more than a journal article.

This last section explains why it is important when undertaking a systematic review, that researchers identify and locate all the various products or outputs resulting from the one systematic review. The different products can provide extra information that may not otherwise be found by concentrating on just the one output such as a journal article.

In this section the authors note that research has shown that many published systematic reviews are either poor quality or nonsystematic. It emphasised that in the Social Sciences it is important that we understand exactly what a systematic review is, and whether it is the most appropriate methodology to use.

The section describing other methods of evidence synthesis is useful and indicates that there are many other forms of reviews that HDR students can consider such as:

  • Systematic Maps and Evidence & Gap Maps
  • Scoping Reviews
  • Narrative synthesis
  • SWiM (Synthesis without meta-analysis)
  • Realist reviews – using qualitative studies rather than quantitative studies

References and resources

The Curtin Library Research Toolkit – Systematic & Scoping Reviews – in depth information on Systematic reviews. Although focusing on health, there are plenty of resources to help anyone undertaking a scoping review including a section on Reviews in non-health disciplines.

The Campbell Collaboration has several recorded workshops, seminars and other training presentations available on their website. There are videos on question formulation, searching, coding and quality, Meta-analysis methods, advanced methods, policy and much more. https://www.youtube.com/user/collaborationtube/playlists

The Campbell Collaboration has also made some tools and resources available such as an effect size calculator and tools for screening, data extraction and analysis. https://www.campbellcollaboration.org/get-involved/tools/

Covidence – Curtin University has provided staff and HDR students with access to Covidence. You can find details on how to use the Curtin institutional license via the Research Toolkit – Systematic review tools page.

Please make any anonymous comments/ feedback, or suggestions for further posts at this link. If you would like to get in touch, or write a post for the Ideas Hub blog, please email karen.miller@curtin.edu.au.

Photo by Tim Mossholder on Unsplash