DRI Viewer: The gLibrary/DRI platform. New features added to the User Interface and to the business layer

Transcript

1. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Viewer The gLibrary/DRI platform. New features added to the User infterface and to the business layer Autors: R.Barbera, A.Calanducci, J.Engelmo, JM Franco, JM González, R.Ramos, M.Rubio Evento: EELA-2 Conference Lugar / Fecha: Bogotá, 25/02/2009
2. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview DRI Generic Viewer DRI Test tool Conclusions 1 3 4 5 DRI Viewer INDEX DRI Repository generator tool 2
3. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Introduction  gLibrary/DRI is a framework for developing, hosting and managing digital repositories on Grid eInfrastructures.  It has effectively reduced the cost of hosting new repositories and thus enlarging their number  The extensions presented on this work are focused to allow efficient mgmnt of large numbers of repositories:  The process of generation of new repositories has been strongly simplified  A generic viewer has been developed. It allows users to browse and view any piece of data of any repository. It can be used also as the basis for extending it to a more specific viewer.  Newly generated repository can be quickly and automatically tested to assure its correct working without manually testing all use cases. 3
4. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions gLibrary/DRI User interface
5. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions gLibrary/DRI global architecture This example: 1.- Suposses two repositories, Mammograms one and Ancient Manuscripts one. 2.- Both of them use their own viewer application. Mammograms rep. Uses the default storage funcionality (which is based on AMGA). 3.- Ancient Manuscripts rep. Uses their own persistance layer. 4.- Both of them use Storage Element for storing the digital content files.
6. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Data model example This figure shows a patient node definition following DRI specification: <TableName name="patient" primaryIdAttr="MGPlusPatientId" AMGAPrefix="patient" ClassName="MGPlusPatient" > <attr name="MGPlusPatientId"> <dbAttrName>MGPlusPatientId</dbAttrName> <type>string</type> <dbAttrType>int</dbAttrType> </attr> <attr name="patientName"> <dbAttrName>PatientName</dbAttrName> <type>string</type> <dbAttrType>varchar(75)</dbAttrType> </attr> <attr name="gender"> <dbAttrName>Gender</dbAttrName> <type>string</type> <dbAttrType>varchar(1)</dbAttrType> </attr> <attr name="ageAtMenopause"> <dbAttrName>AgeAtMenopause</dbAttrName> <type>string</type> <dbAttrType>int</dbAttrType> </attr> </TableName>
7. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Repository Generator Tool  New repository creation in gLibrary/DRI platform is a non-straightforward process  The main objective of the DRI Repository Generator is to create all the necessary basic elements to generate a new repository  All repositories have a common structure:  The base Java classes (for the data model and the different modules)  The name and structure of the directories as well as the name of the Java classes  The needed AMGA structure for storing both navigation and metadata information 7
8. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Repository Generator Tool  Basics components to generate  Java Classes  Data Model  Storage Module  User Interface Module  Directory structure  Needed structure,configuration files and datamodel description XML files  AMGA collections  All datamodel collections  Configuration collections 8
9. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Repository Generator Tool  New XML files for describing repositories  RepositoryExample.xml  Describes the repository main components  Label “Configuration” refers to the XML files that define the navigation trees and search filters that will appear in the gLibrary/DRI user interface  Label "DataModel" defines all the XML files that describe the data model and are necessary to build Java classes and AMGA collections <Repository Name="Repo" AMGAHierarchy ="" > <Configuration> <TreeXML>./NavigationTreeByPatientName.xml</TreeXML> <TreeXML>./NavigationTreeByPatologyName.xml</TreeXML> <FilterXML>./FilterExample.xml</FilterXML> </Configuration> <DataModel > <DataModelXML AMGApath="client/">RepoStorageClient.xml</DataModelXML> <DataModelXML AMGApath="client/image/">RepoStorageImage.xml</DataModelXML> </DataModel> </Repository> 9
10. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Repository Generator Tool  New XML files  Tree.xml  This file describes the navigation trees located at the UI  Filter.xml  This file describes the search filters located at the UI  Any number of filters can be added  gLibrary/DRI shows activates and places them dinamically  There are three types of filters: » Type 1: Two combos are deployed, one in which we can choose an attribute and other one with a set of values associated with that attribute. » Type 2: It consists on a combo with a pre-defined query. » Type 3: It is a customizable filter. It consists of a set of three components. First of them is a combo containing attributes, another one contains different kinds of operators and the last one contains a text box where a concrete value. This way, a real-time query can be launched. 10
11. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Repository Generator Tool  Repository generator is a program that:  Generates AMGA collections and Java classes from the DataModel.xml repository specification.  Generates a build.xml file with basic functionality to compile the previously generated sources and to create a jar file.  Generates a skeleton application to be able to access programatically the repository for quickly adding/accessing data. 11
12. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Generic Viewer  DRI Generic Viewer is compatible with any repository hosted on gLibrary/DRI. Its main features are:  Default functional repository viewer  No need to program a viewer application for each repository.  A generic tool for data manipulation and representation  Can edit, delete, add new repository items (including pictures, metadata, etc.)  A 100% Java Desktop application  Java SE 6 features  Swing GUI Libraries  Java Web Start  gLibrary/DRI APIs  A basis to build ad-hoc applications for implementing functionalities for a concrete repository 12
13. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Data representation • gLibrary/DRI web portal parametrizes the data the user will load into DRI Viewer. • Metadata is displayed into a form • Digital content is displayed by associated components User credentials gLibrary/DRI API GridFTP 13
14. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Operations over repositories  DRI Viewer allows create/edit/read/delete operations over repository data (both metadata and digital content)  Interaction with repositories through gLibrary/DRI APIs  Form-based metadata edition  Digital content edition through metadata attachments  Data persistence on GRID Storage or local file system  Recursive data addition and removal Metadata edition form Digital content as an attachment 14
15. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Future enhancements  DRI Viewer future work  Include Job Launching  Drivers for local and gLite environments developed  BOINC driver planned  Increase the number of viewer components  To support the most commonly used file types (pdf, txt, rtf, etc.)  Field label formatting  Metadata edition forms designed by repository providers in the repository definition 15
16. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Test tool overview 16  The automatic testing tool checks infrastructure coherence and repository working. Needs for a test environment:  Growth of the gLibrary/DRI platform components  Can be quite tedious and inefficient process to verify the functionality of a new repository or application deployed having to flow through all its options.  Checks the components working status...  Focuses specially on those components more probable to give a system global failure.  ...and reports about the results given  May be in two ways, a short one, or a longer one.  Main objetive:  To check automatically the basic platform components giving quick response of the correct working of them.
17. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Test architecture  Layers  Structure of the test environment is similar to gLibrary/DRI platform  The internal processes the test performs are quite similar to the steps a normal user would perform in a complete use case in which he access to the repository by uploading and then saving a complete-structure node, including those data marked as LFN which will be stored on the SE.  Two main parts:  The entry point, which includes testing the login applet, the application server testing and the PHP-Java Bridge.  The other part is aimed to test the whole gLibrary/DRI and repositories deployed at the platform and focusing on the functionality of the core operations, such us insert, load and remove a certain number of nodes. The test check also APIs gLibrary/DRI platform (LFC, Servlet, etc)  The test is aimed to repository providers for checking the integrity of their repositories 17
18. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Test – Working example 18 GENERATION INSERT LOADING DELETING CONFIGURING TEST
19. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions DRI Test – Java Desktop 19  Workflow:  Initialize test data:  The test creates a predefined number of nodes. The structure of these objects is given by the repository that is being tested.  Insert, load and remove:  The test exchanges information with the servlet to save the previous nodes in database. For each node correctly-saved the servlet returns an identifier.  Recover the objects, with the previously recorded identifiers. Comparing each load-node with each memory-node create in the initialization process.  This method erases each node inserted using each identifier to locate the nodes to delete.  Auxiliary remove (optional):  When a failure point ocurrs, the test run this process in case some node has been inserted in the database to delete these nodes thereby maintaining the integrity of the process and not holding useless information.  Result:  All information of application sequence will be display on screen and, concurrently, the test will generate a xml file. It reports the same messages that the user sees during the performance. These messages identify transactions completed successfully in green and bugs in red. To the end, the report will be transformed into an html file and user’s default browser displays it.
20. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Overall conclusions 20  Conclusions:  Multi-repository management is easier:  They are fast and securely generated  Including automatic testing mechanisms  New repository viewer tool  The provider has not to develop their own viewer  It can be used as the basis to build another specific tool  Algorithms launching mechanism planned  For example, image manipulation operations will be submitted to the Grid from the viewer
21. CENTRO EXTREMEÑO DE TECNOLOGÍAS AVANZADAS gLibraRY/DRI, new features added Bogotá, 25/02/2009 INDEX Introduction 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 5 DRI Repository generator tool DRI Generic viewer DRI Test tool Conclusions Conventual de San Francisco, Sola 1, 10200 Trujillo Teléfono: 927 65 93 17 Fax: 927 32 32 37 www.ceta-ciemat.es