TOBI: an app to track open scholarly metadata is now available
Are open data sources reliable enough to perform bibliometric analyses? TOBI aims to answer this question by assessing the quality of open datasets. Among its first deliverables, an app is now available.
Are open data sources reliable enough to perform bibliometric analyses? TOBI (Towards Open Bibliometric Indicators) is a project that aims to answer this question by identifying the most important open bibliometric data sources and assessing their quality. Data sources are evaluated with a focus on the research outputs of Swiss higher education institutions (HEIs). The project is led by the Knowledge Management team from the ETH Library and is co-funded by external pageswissuniversities.
Why is it important to ensure that bibliometric data are of high quality? Scholarly metadata are the basis for quantitative research assessment. If some key metadata are missing from a list of publications authored at a particular institution (e.g. the affiliation or a significant number of relevant citations), the bibliometric analysis will not yield results reflecting the effective output of the institution. Hence, the quality of the metadata needs to be investigated in order to identify the strengths and weaknesses of the data sources and make an informed decision on how to use the indicators for research assessment.
This is particularly critical for institutions and actors seeking to comply with the Declaration on Research Assessment (DORA) and the CoARA Agreement on Reforming Research Assessment, both of which highlight the need for open and fair evaluation and advocate for responsible use of bibliometric indicators.
TOBI started in March 2023. One of its first deliverables is a external pageweb application to track scholarly metadata. The user can input a custom list of DOIs or choose a random sample of DOIs belonging to a given Swiss HEI. The app looks up the metadata associated with the DOIs in open data sources and returns associated counts. The user can therefore inspect differences in citation, reference and author counts for a small set of DOIs and check for potential biases or errors.
The second deliverable consists of a list of open bibliometric data sources and a report about their features. It is available on the external pageTOBI website.