Tips and Tricks for using Rayyan for Systematic Review – chloe forte.

What is Rayyan?

Rayyan is a software specifically designed to conduct and coordinate systematic literature reviews.

What is the benefit of Rayyan?

1.       It’s free.

2.       It’s web-based (so you can access it anywhere) and does not have a desktop version (as far as I’m aware).

3.       You can create and collaborate on as many reviews as you need.

4.       You can oversee the work of your review team.

5.       You can tailor your review folder to facilitate screening.

When to use Rayyan?

I would recommend completing your systematic search as you normally would and import your search results to Endnote first. Then use Rayyan for the screening phases. I used Endnote folders to divide my papers randomly between my review team so that I knew which reviewer has which papers.

My step-by-step guide for using Rayyan for screening papers with multiple reviewers

Phase 1 – Title and abstract screening

Setting Up

1.       Create an account.

2.       Ensure each member of your review team creates an account.

3.       Create a separate review for each review team member. Remember to number these so you can keep track of which reviewer has which review. Separate reviews for each reviewer make it easier to see their own workload and for you to see overall progress.

4.       Export your search results for each reviewer from Endnote/excel/Scopus etc. to a folder on your desktop, then upload to Rayyan. If you export from Endnote the title and abstract will be available to view in Rayyan, if you export from Excel or Scopus, you will need to find the paper manually to view the abstract.

5.       Invite each reviewer to their separate review. Once they have confirmed their email for the Rayyan account, reviewers can view their review under the ‘Collaborations Review Tab’ on the header of the main page.

6.       On the main review page, click ‘Blind Off’ for all reviews. This will allow you to view the decisions and labels of all reviewers. Only you can see all the reviews you created. The reviewers can only see what they have been invited to. If you keep one review for your entire team, keep ‘Blind on’, so all decisions are blinded.

Screening

7.       To access the review, click ‘Show’.

8.       You can see all your articles in the viewing window. You can sort by author name, date etc.

9.       On the left side of the Rayyan page you can see several orange tabs with useful functions for the review. At the top you can see the ‘Inclusion Decisions tab’ just click to see all included articles.

10.   The ‘keywords for include and exclude’ tabs appear along the left side of the screen when you open your review. Delete Rayyan’s auto-generated keywords and customise your own. When you screen your articles the include/exclude words will be highlighted in green and red. For example, if you add ‘meta-analyses’ to your exclude keywords, you can click on them and all articles that have those words in the title or abstract will come up. A quick way of deciding to exclude quickly.

11.   When you open an article, you will see a banner on the middle of the screen with options to include, exclude, label, reason or add note. For phase 1, reviewers will mainly deal with the include and exclude tabs. Once you assign a decision to an article it gets highlighted in red or green and will be in the assigned folder on the top left.

12.   Reviewers can also assign ‘maybe’ as a decision and include a note on their decision.

13.   Chat function: on the bottom right corner of the review screen, you can see a tab on the right side saying ‘Review Chat’ this is an in-software chat function that can be used if multiple reviewers are working simultaneously.

14.   Before you click ‘show articles’ in the main Rayyan home screen, you can see overall review progress, number of sessions and total time spent in the screening summary. Progress is shown in a pie-chart form with include, exclude and maybe decisions shown for total number of articles.

Phase 2 – Full Text screening

Setting Up

1.       When Phase 1 screening is completed export all included papers from all separate reviews back to Endnote/Excel.

2.       Then organise in Endnote subfolders so each reviewer has their randomly allocated papers for Phase 2 in a separate folder on Endnote. Then follow same process as in Phase 1 for exporting to Rayyan.

3.       For phase 2 I found it easier to create entire new separate reviews for each reviewer. This kept things streamlined and less cluttered.

4.       Follow same process above to invite reviewers to collaborate on reviews.

5.       For Phase 2 we need the full text of the article. To do this find the PDF of the article, and in your review page open the associated paper, on the middle banner click ‘Upload PDF full-texts’. Then upload your paper. It will appear as a yellow icon beside the article title. To access the PDF scroll down below the abstract to find the full-text link.

Screening

1.       For full-text screening, your key words will not highlight in your PDFs. So do not waste time customising your key words for Phase 2.

2.       Use the label, reason, and note functions. This way other reviewers can understand your thinking behind a decision. It will also help for data quality and extraction later. For example, use the label function to label specific study designs, population types, measures and instruments etc. Then when you export, these will already be highlighted for you. NB: when you export the papers to Excel you can select to include labels and reasons but not notes! Highlight this to your reviewers in advance.

3.       Once phase 2 has been completed export your included papers to Endnote and now you have your folder of papers to be included for review.

Thanks for reading my tips and tricks for using Rayyan for systematic review, I hope you found it helpful.