Share
Embed
The HathiTrust Research Center (HTRC): An Overview and Demo
The session will provide an overview of the HathiTrust Research Center including its mission and current status. It will also include a demonstration of current HTRC phase one technology and services. Additionally, the speakers will address the HTRC's role in supporting humanities research at scale.
Robert H. McDonald
June 07, 2013
Transcript
17 slides
-
The HathiTrust Research Center (HTRC): An Overview and Demo
IU Librarians’ Day | IUPUI Libraries | 06.07.13 Robert H. McDonald -‐ @mcdonald -‐ IU Libraries Yiming Sun – IU Data to Insight Center Miao Chen – IU Data to Insight Center Tweet US -‐ @HathiTrust #HTRC -
Speaker Deck Slides 06.07.13 IU Librarians’ Day
#HTRC @HathiTrust hHp://bit.ly/13gWD7C -
HTRC Mission • Public research arm of the HathiTrust
• Help researchers world-‐wide to accomplish tera-‐scale text data-‐mining and analysis – Develop cuOng-‐edge soPware tools for processing, analyzing text – Develop cyberinfrastructure to enable HPC access to the HathiTrust Digital Library • Established: July, 2011 • CollaboraWve center: Indiana University & University of Illinois 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
HTRC Governance • Reports to the HathiTrust Board of
Governors • HTRC ExecuWve Commi[ee – J. Stephen Downie (Co-‐director), Professor and Associate Dean for Research, University of Illinois GSLIS – Beth Plale (Co-‐director and Chair), Director Data To Insight Center and professor in the School of InformaWcs and CompuWng at Indiana University – Robert H. McDonald, Associate Dean of Libraries/Deputy Director Data to Insight Center at Indiana University – Beth Sandore Namachchivaya, Associate University Librarian for InformaWon Technology Planning & Policy at the University of Illinois – John Unsworth, Vice Provost for Library & Technology Services and Chief InformaWon Officer at Brandeis University • HTRC Advisory Board (See members next slide) • Google Public Domain agreement – in place for IU and UIUC -
HTRC Advisory Board • Cathy Blake, University of Illinois,
Urbana-‐Champaign • Beth Cate, Indiana University • Greg Crane, TuPs University • Laine Farley, California Digital Library • Brian Geiger, University of California at Riverside • David Greenbaum, University of California at Berkeley • FoWs Jannidis, University of Wurzberg, Germany • Ma[hew Jockers, Stanford University • Jim Neal, Columbia University • Bill Newman, Indiana University • Bethany Nowviskie, University of Virginia • Andrey Rzhetsky, University of Chicago • Pat Steele, University of Maryland • Craig Stewart, Indiana University • David Theo Goldberg, University of California at Irvine • John Towns, NaWonal Center for SupercompuWng ApplicaWons • Madelyn Wessel, University of Virginia -
HTRC Timeline • Phase I: 18-‐month development cycle
– Began 01 July 2011 – Demo of capability September 2012 (14 mo mark) at HTRC UnCamp I • Phase II: broad availability of resource, begins 31 March 2013 – New HTRC Asst. Director for EducaWon and Outreach (Miao Chen) – New listserv to drive user input: htrc-‐usergroup-‐l @ list.indiana.edu 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
HTRC Next Steps • Phase 2 availability of resource
31 March 2013 • Thanks to: Photos from HTRC UnCamp 9.10.12 at Indiana University -
HTRC Phase 2: Current Thrusts • Grow HTRC User-‐base
– Outreach and Engagement • Input from HTRC Advisory Board • Input from HT BOG – Town Hall Groups at DH, JCDL, JADH – Online Town Hall Groups • Develop New SpecificaWons from User-‐Based Agile Development Methodology • Develop and Integrate Sloan Cloud Components into the HTRC Infrastructure 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
06.07.13 IU Librarians’ Day #HTRC
@HathiTrust HTRC Architecture Overview Data API access interface Portal Security (OAuth2 WSO2 IS) Algorithms and Worksets Registry (WSO2 GR) ApplicaWon submission Audit Cassandra cluster volume store Solr index EnWty ExtracWon Topic Modeling Sentence Tokenizer Word posiWon Latent semanWc analysis High level apps Compute resources Storage resources Blacklight -
06.07.13 IU Librarians’ Day #HTRC
@HathiTrust VM Request VM Image Manager VM Image Store VM Image Builder VM Manager VM instance Sloan Cloud SSH Non-‐consumpWve Output Storage user HTRC Non-‐ ConsumpUve Research Access (Sloan Cloud) -
HTRC DemonstraWon • Yiming Sun – Lead Technical Architect
HTRC 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
Metadata Enhancement • Current metadata fields are MARC-‐based
– E.g. publicaWon date, authors, Wtle, subject • MARC fields are fundamental • Needed more fields of users’ interest for granular analyWcs (Metadata Enhancement) • Solicit user requirements and prioriWze for implementaWon – Mainly digital humaniWes uses now 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
Top Metadata Enhancement Items • 1st round user requirement
collecWon, top 3 items were metadata related: – Word frequency count and document length for a volume – Metadata de-‐duplicaWon – Author Gender Analysis • These top 3 items are in process for funcWonality within the current producWon system and will be available in the next quarter. 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
Other Metadata Enhancement Items • Stats analysis: m-‐idf
• Readability score • Language • Topic modeling (e.g. LDA probability) • Genre • Era of compilaWon • Book length (e.g. short or long) • Concordance index (indexing with context) 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
HTRC Upcoming Events • DH 2013 – July 16-‐29,
2013 • JCDL 2013 – July 22-‐26, 2013 • HathiTrust Research Center UnCamp – Sept 8-‐9, 2013 – University of Illinois • Catapult Symposium – IUB – Sept 2013 • JADH 2013 – September 19-‐21, 2013 • Ohio State University – Library Symposium – October 2013 • Educause 2013 – October 15-‐18, 2013 -
Thank You • This presentaWon was made possible with
content provided by many HTRC colleagues John Unsworth, J. Stephen Downie, Robert H. McDonald, Beth Sandore, Yiming Sun, Guangchen Ruan, Lore[a Auvil, Kirk Hess, and many others… • The HTRC Non-‐ConsumpWve Research Grant is graciously funded by the Alfred P. Sloan FoundaWon • IU D2I-‐PTI is graciously funded by The Lilly Endowment, Inc. • HTRC -‐ h[p://www.hathitrust.org/htrc • IU D2I Center -‐ h[p://d2i.indiana.edu/ • UIUC GSLIS -‐ h[p://www.lis.illinois.edu/ 06.07.13 IU Librarians’ Day #HTRC @HathiTrust -
Contact InformaWon • General – Robert H. McDonald, HTRC
ExecuWve Commi[ee – [email protected] • Technical – Yiming Sun, Chief Architect, [email protected] • Requests for capability, interest – Miao Chen, HTRC Asst. Director of EducaWon and Outreach, [email protected] 06.07.13 IU Librarians’ Day #HTRC @HathiTrust