EDDI 2012 - European DDI Users Meeting, Bergen, Norway

Data Management, DDI-based Documentation and Visualization of Business and Organizational Research Data at the DSZ-BO

Transcript

1. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Data Management, DDI-based Documentation and Visualization of Business and Organizational Research Data at the DSZ-BO Johanna Vompras University Library Bielefeld Dec 4th, 2012 EDDI2012 – Bergen, Norway Session B1: Infrastructure for Data Collection, Research, and Archiving Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld Universität Bielefeld
2. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Contents 1 DSZ-BO 2 Data Infrastructure 3 Technical Solutions and Tools 4 Summary Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
3. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary You have never heard about ’Bielefeld’??? Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
4. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary University of Bielefeld Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
5. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Task and Scope Services in DSZ-BO Contents 1 DSZ-BO 2 Data Infrastructure 3 Technical Solutions and Tools 4 Summary Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
6. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Task and Scope Services in DSZ-BO Task and Scope of the DSZ-BO Collect, archive, distribute and maintaining a catalogue of Business and Organizational Data from the Social Sciences, like ... Surveys with multiple organizations, e.g. interviews with human resource managers of different firms, Qualitative case studies and mixed methods, Process generated numbers, e.g. average time of patients in different hospitals, business catalogues, Observations, e.g. informal processes in one local office, Linked employer employee data (LEE). Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
7. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Task and Scope Services in DSZ-BO DSZ-BO Services 'DWD VHUYLFH FHQWHU IRU %XVLQHVV DQG 2UJDQL]DWLRQDO '$7$ '6=%2 'DWD &DWDORJXH 'DWD 3URGXFHUV 6XSSRUW 7UDLQLQJ 'DWD 8VHUV 'DWD 5HJLVWUDWLRQ $UFKLYLQJ 'DWD 6KDULQJ 'DWD 0DQDJHPHQW 'RFXPHQWDWLRQ 'DWD 3URWHFWLRQ $QRQ\PL]DWLRQ 'DWD $FFHVV 6HFRQGDU\ 8VH *HQHUDO ,QIRUPDWLRQ 'HWDLOHG 0HWDGDWD 'LVFRYHU\ 6HUYLFHV Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
8. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Specific Requirements DSZ-BO as a part of Institutional Infrastructures Contents 1 DSZ-BO 2 Data Infrastructure 3 Technical Solutions and Tools 4 Summary Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
9. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Specific Requirements DSZ-BO as a part of Institutional Infrastructures Specific Requirements Standardized Documentation: Mapping of ”complex” study structure into DDI: BEATA Emloyer inquiry Quant. Survey Qual. Interview Document analysis Employee survey Partner survey Employee units Employer units Couples Study Structure Analysis levels Data Catalogue: Data with certain methods, Data which contains variables that operationalize certain research questions, Datasets with certain levels of analysis, Examples for good practice in a certain field. Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
10. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Specific Requirements DSZ-BO as a part of Institutional Infrastructures Co-Operation DSZ-BO and Library Services meta- und microdata relation to business or organizational unit (e.g. linked employer-employee data) data Faculty of Sociology University Library cooperation study: data collection questions, variablen data sets in SPSS/ STATA, etc. internal representation metadata-format DDI 3.1 xml schemes xml database(s) data re-use, study comparison, support of the data lifecycle multiple-language support DSZ-BO data access distribution data sharing long-term archiving authenti- cation indexing retrieval textual information, publications, surveys study coding into DDI 3.1, semi-automatically services functionalities privacy policies data cleaning, user support/help desk (editing, access, training, etc.), content maintance data privacy policies development of the technical infrastructure Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
11. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Specific Requirements DSZ-BO as a part of Institutional Infrastructures Co-Operation DSZ-BO and Library Services 12345 67891ABCD8434 C2E43AD793D9F6A729DC9DC47A43AD74E967A39 29EA728921ED2C521ED2298434 8434 4B6E39D9 DBADED 7A2CA3 AFC4C BDD2C43AD7 368984349BDEE2B3AD7 623AD7!94CA4FE27 84349239A79"# $%$%!923B A732C74E9C2C227343AD7 123484345DC143 1123456 &1E9B'212 &1E98434F42 84349C2562!93689BD14CAD7! 6DC39D93'2 78983ABCDEFEAD 16E3AE25E4764296DC3 1  843494BB2 8A3CAF63AD7 84349'4CA7 ED7532C19 4CB'AA7 463'273A5 B43AD7 A782&A7 C23CA24E 1231456789ABCD518B9E7F4 68 518B9 E7 4C 2 3689BD8A79A73D9(()9*+!9 21A5463D143AB4EE 2CAB2 67B3AD74EA3A2 CA4B9 DEABA2 84349BE247A7! 62C96DC3#'2E982 28A3A7!94BB2!93C4A7A7!923B! BD73273914A7347B2 84349CA4B9DEABA2 822ED12739D93'2932B'7AB4E9 A7C43C6B36C29 Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
12. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Contents 1 DSZ-BO 2 Data Infrastructure 3 Technical Solutions and Tools 4 Summary Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
13. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Documentation or putting ”research” into a DDI ... 6WXG\ ,QYHVWLJDWLRQ  ([SHUW .QRZOHGJH  6\VWHPDWLF /LWHUDWXUH 6HDUFK DQG 6HOHFWLRQ  $FTXLVLWLRQ RI 'DWD IURP 'DWD 3URGXFHUV 6WXG\ &ODVVLILFDWLRQ 6WDQGDUGV $ % &  $ 2ZQ 5HVHDUFK  'HWDLOHG 'RFXPHQWDWLRQ  % 1RUPDO 'RFXPHQWDWLRQ  & 0LQLPDO 'RFXPHQWDWLRQ IRU &DWDORJXLQJ DQG 5HJLVWUDWLRQ SXUSRVHV 6WXG\ 'RFXPHQWDWLRQ XVLQJ '', Y 3XEOLFDWLRQV &RGHERRNV 'RFXPHQWDWLRQV PDFKLQH DQG QRQ PDFKLQH UHDGDEOH 4XHVWLRQQDLUHV 5HIHUHQFHV 6WDWLVWLFDO 'DWD )LOHV $GGLWLRQDO SDUDPHWHUV IRU FODVVLILFDWLRQ .LQG RI 'DWD 'DWD DQG 'RFXPHQWDWLRQ 4XDOLW\ 7UDFHDELOLW\ RI 5HVHDUFK :RUN ([SHUW .QRZOHGJH 3URFHVVLQJ (IIRUW 6SHFLILF 'DWD 3URFHVVLQJ DQG 'RFXPHQWDWLRQ :RUNIORZV ([DPSOHV  &RQYHUVLRQ RI 6366 WR '', IRU YDULDEOH OHYHO GRFXPHQWDWLRQ  '',;0/7HPSODWHV IRU $ 6WDQGDUG GRFXPHQWDWLRQ  ;0/ )RUPV IRU &6WDQGDUG GRFXPHQWDWLRQ Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
14. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Tool 1: Editor for C-Standard Studies Form template (.xsn) provides a view on XML data → fill it out! Form is compressed, XML Schemas, default XML data, XSLT files for view in the form, Script files and form definition files. Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
15. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Tool 2: Content Administration Backend Inspired by researchers workflows for Content Creation and Publishing Concentration of all processing steps needed to publish the ’DDI-Instance’ Upload and Archival of the DDI XML file, Assignment of internal ID, Storage of Metadata and XML Database Generation, Preview and Publishing Options. Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
16. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Technical Infrastructure: 3 Layers Requirements: Easy to maintain (contents), Easy to extend (DDI model and queries) General approach for mapping into a visualization Data Encapsulation ,QIRUPDWLRQ 3RUWDO  6HDUFK  4XHU\ 3URFHVVLQJ  9LVXDOL]DWLRQ 6WRUDJH /D\HU  ;0/64/ 'DWDEDVHV  2WKHU 5HVVRXUFHV 7DEOHV 6WDWLVWLFDO )LOHV HWF $GPLQLVWUDWLRQ %DFNHQG 7HVWLQJ3URGXFWLYH  &DWDORJ 0DLQWHQDQFH  (GLWRULDO :RUN  &RQWHQW 0DQDJHPHQW Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
17. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Data Storage Layer Data storage: DDI File: XML Database (BaseX) Other Metadata: Relational Database Data Queries/Modification: XQUERY/MySQL Data Visualization: JavaScript Framework PHP Script with XQUERY ⇒ Results ⇒ JSON ⇒ Input for Components (e.g. GridPanel or DataView) Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
18. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Data Retrieval and Visualization Each ’Study’ is stored in a single XML database Queried by XQuery language, e.g. ’select all publications related to the study ALLBUS’ Results are returned as lists of items and transformed into JSON format XQuery: find all publications related to ALLBUS study FOR $node IN doc("allbus")//s:StudyUnit/r:OtherMaterial[@type=’text’] RETURN ( CONCAT( data($node/r:Citation/r:Creator)," (",data($node/r:Citation/r:PublicationDate),"),", data($node/r:Citation/r:Title)), data($node/r:Citation/r:PublicationDate), data($node/r:ExternalURLReference) ) Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
19. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Visualization of structured JSON data Components: Windows, (Tree)Panels, Tabs, (Grouping)Grid Functions e.g. within GroupingGrid: Sorting and Grouping Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
20. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Documentation Information Portal and its Technical Background Functionalities of the Search Search, Browse, Visualization Selection of studies, display of general study information Listings of data collections, questions, concepts, etc. Linking of data with questionnaires and publications, and other materials Search by keywords, or (thesaurus) concepts Filtering (e.g. by year, country, standard) Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
21. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Appendix: Dynamic Query Generation Contents 1 DSZ-BO 2 Data Infrastructure 3 Technical Solutions and Tools 4 Summary Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
22. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Appendix: Dynamic Query Generation DSZ-BO: Experiences from technical point of view To find a collection of light-weighted documentation tools tailored for specific scope of studies and data, which are easy to learn and easy to operate, Special requirements for afterwards documentation → decision about classification Adjustment to researchers’ working workflows: tools, editors, data processing, and content authoring system. WYSIWYG-like information portal, Easy-to-use content management system, not only for technical staff. Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
23. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Appendix: Dynamic Query Generation Thank you! Questions? Contact Bielefeld University Library Research Data Management Services and Infrastructure Projects Johanna Vompras johanna.vompras@uni-bielefeld.de Data Service Center for Business and Organizational Data dsz-bo@uni-bielefeld.de Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld
24. DSZ-BO Data Infrastructure Technical Solutions and Tools Summary Appendix: Dynamic Query Generation Appendix: XQUERY Query Generation Table ddi path: id attribute label path root ... 1 title Titel data($node/r:Citation/r:Title) s:StudyUnit 2 creator Erstellt von $node/r:Citation/r:Creator s:StudyUnit 3 funding Gef¨ ordert durch for $knoten in doc($database) ... ... 18 studyresults Ergebnisse NULL ... 144 realization.sampling notes Anmerkungen $node/r:Note/r:Content d:DataCollection Example ’Funding’: FOR $knoten in doc($database)//a:Archive/a:OrganizationScheme/a:Organization WHERE $knoten/@id = $node/r:FundingInformation/r:AgencyOrganizationReference/r:ID RETURN concat(data($knoten/a:OrganizationName),utilities:ifexistsPar(data($knoten/a:Nickname))) corresponds to the relational statement: π(a:OrganisationName) (a:Archive/a:OrganizationScheme/a:Organization @id==r:ID r:FundingInformation/r:AgencyOrganizationReference ) Johanna Vompras EDDI2012, December 3–4 2012, Bergen, Norway Universität Bielefeld