Demonstrating impact

Selecting the best tool

What am I trying to demonstrate?

• Citations are traditionally used to demonstrate research impact, whereas there are alternative metrics which can be used to demonstrate broader impact and engagement
• Are you being asked to demonstrate international or industrial impact and engagement? Possibly you could look at collaborations and where citations are coming from.
• If applying for a grant or a job look at the wording; exactly what are you being asked to demonstrate?

Different databases will retrieve different results

• Different databases index different sources i.e. journals covering different subjects as well as covering different date ranges. You will need to select a database or databases that provide adequate coverage of your work.
• Some include poor quality data (duplicate records, misspelt citations etc).

What types of documents should be included in a report?

• Generally citation analyses would include articles, notes and reviews as they are document types that traditionally receive citations
• Book reviews, editorials and meeting abstracts are normally excluded as they are not generally cited, they decrease the average cites per paper.
• If demonstrating impact or engagement as opposed to pure research impact a wider variety of research output could be included outside of research articles.

Citation data should be used to compare like with like

• Different disciplines, and even subdisciplines, have different citation behaviour and patterns and patterns so it is more meaningful to compare publications from within the same field
• Document age influences the number of citations it has, or is likely to receive.
• When making comparisons, ensure the data has been normalised or adjusted to take into account differences between the disciplines.

Why create a report?

A citation report is a collation of bibliographic details for a set of publications, usually the output of an individual researcher or a group, along with the numbers of times those documents have been cited. This could include more detailed analysis of the citations such as average number of citations for a researcher or the normalised average compared to others publishing in the same area. It might also contain information about others the author(s) have collaborated with or the impact of the journals the works have been published in.

The intended purpose of the report will influence what is included and how the information is presented e.g. grant or job applications might have specific metrics that are required.

Finding author publications

Databases such as Scopus, Web of Science or InCites will allow you to create a citation report on any group of papers that are indexed in the database.

If you are trying to generate a citation report for an individual researcher you will need to identify all the publications for that individual in the database. If a researcher’s publications are all linked with an identifier this can make the process of collating all of the publications easier and more accurate.

Scopus

Scopus   

Benefits:

• Indexes more than 22,000 titles, predominantly sciences, with a higher proportion of social sciences and humanities journals than Web of Science with citation data from primarily from 1996 onwards.
• Indexes trade publications, book series, conference papers, patents and the deep web.
• Automatically assigns a Scopus Author-ID to all authors.

Demonstrating impact - Scopus reports guide.
Scopus can produce citation reports for individuals or groups.

Web of Science

Web of Science   

Benefits:

• Indexes over 12,000 high impact, predominantly science and engineering and health science journals with citation data from 1970 onwards
• Also contains over 150,000 conference proceedings.
• Captures important emerging and regional publications through the Emerging Sources Citation Index (ESCI).
• Researchers can create a Researcher ID to use an identifier.

Demonstrating impact - Web of Science reports guide.
Web of Science can produce citation reports for researchers and groups.

InCites

InCites (Clarivate Analytics)   

Benefits:

• Can create highly customisable citation reports drawing on citation data from the Web of Science database
• Uses either Researcher ID or ORCID
• Can produce citation reports for researchers or groups and can be used for benchmarking

Note:

• You will need to register a personal log in to access and use InCites.

Demonstrating impact - InCites reports guide.

Altmetric Explorer

Altmetric Explorer for Institutions

Benefits:

• Tracks mention of research publications online, in social media, media and other sources such as patents and government policy documents.
• Can produce reports for researchers or departments.

Altmetric Explorer - Sign up for an Explorer account (Curtin)
Curtin users need to sign-up to access the institution specific version of Altmetric Explorer using a Curtin email address.

Publish or perish

Publish or Perish   

Benefits:

• Analyses citations from Google Scholar. Useful for subject areas not well covered in Web of Science or Scopus such as business and the humanities and is more comprehensive overall than either, including a lot of types of content, outside of books and articles, not found in the scholarly databases
• Searches for theses, books, abstracts, court proceedings etc. retrieving material from academic publishers, professional societies, online repositories and other sources
• Data from Google Scholar needs to be verified for accuracy as it often misidentifies or duplicates citations leading to artificial inflation of citation figures
• The search functionality is quite basic but there are recommended ways of improving searches including techniques for author disambiguation.