NISO s IOTA Working Group Creating an Index for Measuring the Quality of OpenURL Links Charleston Conference - Nov. 5, 2010 Rafal Kasprowski, Rice U. Susan Marcin, Columbia U.
Agenda Background: Full-text linking and Advent of OpenURL Problems with OpenURL linking IOTA: Created in response to OpenURL linking problems Overview of IOTA's work IOTA project site Community-derived data reports Publicizing IOTA: wiki, blog, Twitter Get involved and Take advantage of current effort
Before OpenURL: Proprietary Linking A&I database providers offered option for full-text linking (e.g., CSA, PubMed, etc.). Libraries manually activated linking to full-text providers they had subscriptions with. A&I --> Full Text
Proprietary Linking: Cons and Pro Linking had to be activated manually by libraries for each full-text provider. A&I providers offering this option were few. Selection of full-text providers was limited. But... Once set up, the static links to full texts were accurate.
Advent of OpenURL Objective: Deliver full texts unrestrained by proprietary silos. Open standard generating link at time of request. Library's holdings indicate provider of "appropriate copy". A-Z list (e.g., e-journal, e-books): Knowledge base (KB) with library's holdings. Intermediary in linking. A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")
OpenURL Syntax and Resolver Source Citation A, Bernand, et al. "A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices." Nanotechnology 21, no. 44 (November 5, 2010): 445201. Academic Search Complete, EBSCOhost (accessed October 24, 2010). Target OpenURL (Source OpenURL structured similarly) http://ps4ps6lm2r.search.serialssolutions.com/?issn=09574484&volume=21&issue=44&date=20101105&spage=445201&title=nanotechnology &atitle=a+ versatile+nanotechnology+to+connect+individual+nanoobjects+for+the+ fabrication+of+hybrid+single-electron+devices. &aulast=a++bernand
Pros & Cons of OpenURL Pros: KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers. Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better Cons: Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures OpenURL linking not improved significantly last 10 years. No systematic method exists to benchmark OpenURLs.
Problem Statement & Methodology "72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products)." Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p. 33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf. Recently, researchers have indicated the need for metadata quality metrics, including: completeness; accuracy; conformance to expectations; logical consistency and coherence. Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice. Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256.
Année philologique OpenURL Study 2008 Cornell study led by Adam Chandler* Problem: Too often links sent from Aph did not successfully resolve to requested resource. Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking. Aph Study investigated: Faulty citation metadata from source database. Method to evaluate the OpenURLs. *Chandler, Adam. 2009. Results of L Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.
Scoring System & Aph Study Outcomes Concept of scoring in Aph study (based on B. Hughes study)* establish a baseline for comparison; results to be shared with data providers; develop a best practice. Problem analysis in Aph study limited to: source link presence/absence of citation metadata elements Results: OpenURL quality model: compares elements in Aph OpenURLs to those of other providers. No scores, but model is first step towards scoring system. *Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
Creation of IOTA NISO accepts proposal to take Aph Study to wider community Improving OpenURLs Through Analytics (IOTA): Formed in January 2010. Basic Assumptions: Results are achieved through an analytical investigation of how OpenURL links work. Practical: Not OpenURL standard is addressed, but links (OpenURLs) generated by standard. Selective changes to OpenURLs will lead to significant improvement in linking success rate. Motto: "small changes. big improvements"
Desired Outcomes Develop community-recognized index for measuring the quality of OpenURL links generated by content providers. Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers. Method: fair; transparent; scalable across all OpenURLs and their providers.
Initial Focus 1. Core elements: Contained in IOTA's OpenURL reporting system; 9 million OpenURLs from libraries and providers. 2. Scoring system based on assumption: Correlation exists between # of core elements ("OpenURL completeness") & ability of OpenURLs to link to specific content. 3. Weighting assigned to core elements: Based on relative importance spage vs atitle issn vs jtitle doi/pmid vs date, etc.
Work in Progress Element weighting still in progress: E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage). Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by: knowledge base, resolver, full-text provider (target). High "completeness" score of OpenURLs not always indicative of "success" in linking to full texts Combination of multiple indexes along linking nodes may provide more complete picture.
Issues with openurl linking Why might "article" links not appear or work? target vendor does not support openurl linking problems with information was passed from source into openurl string. ex:) not enough info, incorrect info
IOTA Project Site
OpenURL log file reports Over 9 million openurl strings in system
Report Types
Report Types
Source Report - vendor or database
Source Report : Elements (vendor)
Source Report : Patterns (vendor)
Element & Pattern Frequency Report
Element & Pattern Frequency Report : Choose Metric
Element & Pattern Frequency Report : Metric = Issue
Click from Element & Pattern Frequency Report to Source Report
Documentation Wiki http://openurlquality.niso.org/
Glossary Provides working definitions of terms such as: Elements Link-to / Target Patterns SID Vendor
IOTA Blog & Twitter http://openurlquality.blogspot.com/ Find us on Twitter Hash Tag #nisoiota
How can I get involved in IOTA? If you are a vendor -Look at IOTA data... or... Point your tech folks to our data This data is meant to help make improvements in your openurl linking. If you are a librarian -Contribute data to IOTA Help spread the word to vendors More information on the IOTA project at: http://openurlquality.niso.org/
Questions? Group email: openurlquality@list.niso.org More info... http://openurlquality.niso.org/