Digital Repositories Infrastructure

Transcript

1. www.epikh.eu The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Digital Repositories Infractucture Jorge Sevilla Cedillo(jorge.sevilla@ciemat.es) CETA-Ciemat University of Stellenbosch 10th June 2011
2. Stellenbosch, EPIKH Workshop , 10.06.2011 2 Outline • What is DRI? • Architecture • How it works? • API as library • API as Resful access • Jobs API.
3. Stellenbosch, EPIKH Workshop , 10.06.2011 3 What is DRI? • DRI, Digital Repositories Infrastructure – Provide store digital repositories on different stores.  Store metadata and digital content.  Abstraction  Easy commands – Extensible.  Add news modules. • Browsing • Storage  Default hibernate modules. • FTP, File and SE drivers. – Projects.  IMED. Portugal  BIOMED. Colombia.  AHT. Spain WEB DRI DB SE http
4. Stellenbosch, EPIKH Workshop , 10.06.2011 4 Architecture DRI-ENGINE Hppt REST-WS WEB-DRI Client Client DB Storage Browsing Storage Browsing Storage Browsing Storage Browsing SE DB FTP FILE DB SE AMGA Repository-1 Repository-2 Repository-3 Repository-N JSON DRINode DRINode Persistent Node DRI-JOB SERVICE Local glite Client Client
5. Desing Datamodel • Depends of the modules. – Each different module uses his own repository format. • A repository had.  Datamodel.  Configured instance of the browsing module.  Configured instance of the storage. • DRI manage several repositories.  Repositories and their modules are configured in DRI-Engine configuration. • Provided default tool hibernate-based.  Input: Standar hibernate xml schemas.  Ouput: standar hibernate repository. Location, Meeting title, dd.mm.yyyy 5
6. Datamodel - Storage – DRISchema  Datamodel schema • Entities • Attribute names. • Attribute Types. – DRINode  Id: entityName+number  Properties – DRIDigitalAsset  Id:  Type of asset.  Size  Digital Content Stellenbosch, EPIKH Workshop , 10.06.2011 6 [ { "entity":"Cliente", ← Entity Name «name":"String", ← String property "setPhone":"Set", ← Set of property "direccion":"Dirección", ← relation with other entity "setSubject":"Subject", ← relation with a set of entities "picture":"DigitalAsset", ← DRIDigitalAsset Property "age":"int" ← int property }, → Other entity← { "entity":"Pedido", "property1" :"float", … } … ] DRINode node = new DRINode("es.cetaciemat.dri_test.datamodel.Cliente"); node.setProperty(“name”,”Paco”); node.setProperty(“age”,”45”); node.setProperty(“picture”,new DigitalAsset("image/jpe")); • Storage Data Transfer Objects.
7. Datamodel - Browsing – DRITreeModel  Datamodel schema • Entities • Attribute names. • Attribute Types. – DRITreeNode  Id  Parent id  Display Name – Query:  Dialect  Query (String) – QueryResult  Array DRINodes Stellenbosch, EPIKH Workshop , 10.06.2011 7 [ • Browsing Data Transfer Objects.
8. Browser Module • Any modules which follow the browsing API. • Communicated with DTOs – DRITreeNode Object – Query Object • Feeding with repository configuration. • Several dialect support. • Provided DRIBrowsingDefault. – Hibernate browsing. – HQL and SQL dialect. Stellenbosch, EPIKH Workshop , 10.06.2011 8 Browsing XML REPO ENGINE DRITree Node Query
9. Storage Module • Any modules which follow the Storage API. – Digital data and metadata follow different ways • Metadata and digital asset store operations. • Datamodel configured from the repository. • Receives DTOs • DRIHibernateStorage – Hibernate persistent operation. – Different digital asset drivers.  Storage Elements.  FTP servers.  File System.  On-memory Stellenbosch, EPIKH Workshop , 10.06.2011 9 Storage ENGINE DB REPO DTOs Persistent format
10. DRI-Engine • Engine of the platform – Access to the modules.  Communicate browsing and storage module. – Management of the repositories.  Internal security.  Logging. Stellenbosch, EPIKH Workshop , 10.06.2011 10 Storage Browsing ENGINE Conf DTOs
11. API as Library • DRI can be integrated as library. – Configured by environment variables  Different repositories • Storage Module • Browsing Module Stellenbosch, EPIKH Workshop , 10.06.2011 11 DRINode node = new DRINode("es.cetaciemat.dri_test.datamodel.Cliente"); node.setProperty(“name”,”Paco”); node.setProperty(“age”,”45”); node.setProperty(“picture”,”image/jpeg”); Serializable id = DRIEngineFactory.getInstance().saveNode(session, repoId, node); DRI byte[] content = file.getBytes(); DRIEngineFactory.getInstance(). uploadDigitalContent(session, repoId, new DRIDigitalAsset("mime", 1), content, 0); DRI Medadata Digital Asset DB SE
12. Stellenbosch, EPIKH Workshop , 10.06.2011 12 API Public Methods (1) • startSession – DRISession session = DRIEngineFactory.getInstance().startSession(); • endSession – DRIEngineFactory.getInstance().endSession(session); • getEntitySchema – DRIEntitySchema schema = DRIEngineFactory.getInstance().getEntitySchema(session, repoId, entityName); • getRepositorySchema – Collection<DRIEntitySchema> schema = DRIEngineFactory.getInstance().getRepositorySchema(session,repoId); • getRepositories – DRIEngineFactory.getInstance().getRepositories(session); • getTreeModel – DRIEngineFactory.getInstance().getTreeNames(session, repoId); • getTreeNode – DRITreeModel treeNames = DRIEngineFactory.getInstance(). getTree(session,repoId,treeName);
13. Stellenbosch, EPIKH Workshop , 10.06.2011 13 API Public Methods(2) • saveNode – Serializable id = DRIEngineFactory.getInstance().saveNode(session, repoId, node); • loadNode – DriNode node= DRIEngineFactory.getInstance().loadNode(session,repoId,"es.cetaciemat.dri_test.datamodel.Cliente", 0, true); • removeNode – DriNode node= DRIEngineFactory.getInstance().removeNode(session,repoId,"es.cetaciemat.dri_test.datamodel.Cliente", 0); • uploadDigitalContent – DRIEngineFactory.getInstance().uploadDigitalContent(session, repoId,new DRIDigitalAsset("mime", 1),content,0); • downloadDigitalContet – DRIBinaryResponse binary = DRIEngineFactory.getInstance().downloadDigitalContent(session, repoId, da, 0, 0); • executeQuery – DRIQueryResult result=executeQuery(session,repiId,query);
14. API as Restful access • Web Service REST-Based (Representational State Transfer) • Resources Access with HTTP URIs. – URIs identify the operation. – PUT, GET, POST, DELETE. • Feeding with JSON object. • Ej: http://dri/repositories/imed/entities/patient  Head parameters. Ex: session token: ljlk4j4444hiua998as  Body in JSON Ex: { patientName: “Pepito”, patientAge: 65 } Stellenbosch, EPIKH Workshop , 10.06.2011 14 REST-WS JSON DRI-ENGINE DRINode
15. Stellenbosch, EPIKH Workshop , 10.06.2011 15 Https access(1) • startSession. PUT – http://host/dri-restws/resources/session  response object with http code which contains the session token • endSession. DELETE – http://host/dri-restws/resources/session  response object with http code • saveNode. PUT – http://host/dri-restws/resources/repositories/ {repoID}/entities/{entity}  response object with http code and ID  Head: • session token • "Content-Type", "application/json”  Body: Node in JSON format • loadNode. GET – http://host/dri-restws/resources/repositories/ {repoID}/entities/{entity}/ID  response object with http code and node in JSON format  Head: session token • removeNode. DELETE – http://host/dri-restws/resources/repositories/ {repoID}/entities/{entity} /ID  response object with http code  Head: session token • executeQuery. GET – http://host/dri-restws/resources/repositories/ {repoID}/query/{queryID} ?index=0&size=10  response object with http code and list of nodes in JSON format  Head: session token  Queryid id the identification of the predefined queries in the repository
16. Stellenbosch, EPIKH Workshop , 10.06.2011 16 Https access(2) • uploadDigitalContent. PUT – http://host/dri-restws/resources/repositories/ {repoID}/dc/{digAssetID}  response object with http code  Head: • session token • "Content-Type", "image/jpeg" • "Content-Transfer-Encoding", "binary"  Body: Digital Content, byte array • downloadDigitalContent. GET – http://host/dri-restws/resources/repositories/ {repoID}/dc/{digAssetIdID}  response object with http code and Response Object with the byte array  Head: session token • executeQuery. GET – http://host/dri-restws/resources/repositories/ {repoID}/query/{queryID} ?index=0&size=10  response object with http code and list of nodes in JSON format  Head: session token  Queryid id the identification of the predefined queries in the repository
17. DRI-JOB SERVICE Location, Meeting title, dd.mm.yyyy 17 • Submit jobs from personal computers. – Compose mode. – Your own scripts. • Monitoring. – Follow de job. – Get output • Drivers. – Local.  Background Forks. – Glite  Type  Executable  Dependencies  Parameters  Output  Extra Parameters DRI-JOB SERVICE Local glite JobType=Normal Executable=hostname Dependencies=[lib,lib2,li4] Parameters Output=hostname.err,hostname.output
18. Summary • Direct access from the client. – Access to the library. – Http access. • Store and manage medadata and digital asset. – Separate Management  Metadata  Digital asset. – Digital Asset linked to medatada • We are working in – DRI-JOB SERVICE – AMGA / SE modules. • You can integrate your own modules Location, Meeting title, dd.mm.yyyy 18 WEB DRI DB SE http
19. Stellenbosch, EPIKH Workshop , 10.06.2011 19 References • Contact to get the tutorials: – Architecture of Applications Departament uaa@ceta-ciemat.es – Jorge Sevilla Cedillo jorge.sevilla@ciemat.es – www.ceta-ciemat.es • Twitter – cetauaa, follow as.
20. 20 Thank you for your kind attention ! Any questions ? Stellenbosch, EPIKH Workshop , 10.06.2011