Tweet
Tweet

Slide 1

Slide 1 text

Your Data In the Major Leagues: A Practical Guide to REST Services Serdar Basegmez Managing Director, Developi Information Systems DEV-1383 IBM Connect 2017 Conference, 20-23 February 2017

Slide 2

Slide 2 text

• IBM Champion (2011 - 2017) • Developi Information Systems, Istanbul • OpenNTF / LUGTR / LotusNotus.com • Featured on… • Engage UG, IBM Connect, ICON UK, NotesIn9… Serdar Başeğmez

Slide 3

Slide 3 text

Agenda • RESTful Architecture • Practical Implications • Advantages of RESTful Services • Providing REST Services for IBM Domino • Consuming REST Services for IBM Domino • Wrap-up

Slide 4

Slide 4 text

RESTful Web Services Representational state transfer (REST) is an architectural style used for web development. Systems and sites designed using this style aim for fast performance, reliability and the ability to scale (to grow and easily support extra users). To achieve these goals, developers work with reusable components that can be managed and updated without affecting the system as a whole while it is running. Source: https://en.wikipedia.org/wiki/Representational_state_transfer

Slide 5

Slide 5 text

History

Slide 6

Slide 6 text

Old School Web Applications Source: https://speakerdeck.com/jeffschenck/rest-easy-api-security-done-right User Interface Business Logic Datastore Front-end Back-end ASP, PHP, CGI, Web Agents, JSP, etc. ← HTML, CSS, JavaScript Forms →

Slide 7

Slide 7 text

Web Applications Evolving User Interface Business Logic Datastore Front-end Back-end Async web apps, Ruby on Rails, Django, JSF, XPages, etc. ← HTML, CSS, JavaScript Forms, AJAX →

Slide 8

Slide 8 text

Web Applications Evolving User Interface Business Logic Datastore Front-end Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST →

Slide 9

Slide 9 text

Web Applications Evolving User Interface Business Logic Datastore Mobile Applications Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST → Front-end

Slide 10

Slide 10 text

Web Applications Evolving User Interface Business Logic Datastore Mobile Applications Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST → Front-end Microservice Microservice Microservice

Slide 11

Slide 11 text

RESTful, Everywhere! Solid Architecture Well-defined practices Widespread use in modern frameworks Easily consumable, even in micro environments

Slide 12

Slide 12 text

Stateless / Cacheable / Layered Every request processed independently Everything cacheable Client does not care who cooked the meal in the kitchen ⇣ Scalable, Robust, Resilient

Slide 13

Slide 13 text

The Conversation Makes Sense! Source: http://www.bizcoder.com/a-fresh-coat-of-rest-paint-on-a-soap-stack

Slide 14

Slide 14 text

The Conversation Makes Sense! GET /twink/contacts/DLEY-ACLH6Y HTTP/1.1 Host: homer.developi.info Cache-Control: no-cache { "zip": "13202", "state": "NY", "lastName": "Abbate", "middle": "J", "country": "US", "emailAddress": "Jessica.J.Abbate@trashymail.com", "number": "DLEY-ACLH6Y", "city": "Syracuse", "firstName": "Jessica" }

Slide 15

Slide 15 text

The Conversation Makes Sense! http://appserver.company.com/apps/contacts.nsf/ GiveMeTheContactWeNeedPleaseAgent?OpenAgent&id=1522 or… http://appserver.company.com/api/contacts/1522

Slide 16

Slide 16 text

Conventions on URLs GET http://appserver.company.com/api/contacts GET http://appserver.company.com/api/contacts/UK/London POST http://appserver.company.com/api/contacts Retrieve Contacts / Create a new Contact…

Slide 17

Slide 17 text

Conventions on URLs GET http://appserver.company.com/api/contacts/1522 PUT http://appserver.company.com/api/contacts/1522 DELETE http://appserver.company.com/api/contacts/1522 Retrieve/Update/Delete the Contact resource with id=1522…

Slide 18

Slide 18 text

URI GET PUT POST DELETE /contacts/ List Contacts Replace Contacts Create New Contact Delete Contacts /contacts/id Retrieve a Contact Replace a Contact N/A (generally) Delete a Contact Source: https://en.wikipedia.org/wiki/Representational_state_transfer Conventions on URLs

Slide 19

Slide 19 text

Unconventional uses in URLs GET https://api.twitter.com/1.1/statuses/show.json?id=1234567890 Retrieve the Tweet with id=1234567890…

Slide 20

Slide 20 text

RESTful Services for IBM Domino Applications

Slide 21

Slide 21 text

Motivation Putting stuff into a small device! Socializing with other developers! Opening to the wild… New animals out there! Enough! We are moving… All / Some / None of the above

Slide 22

Slide 22 text

RESTful Services on Domino IBM Domino Server Web Apps Agents IBM Notes Client RESTful Services As Consumer Browser Remote Applications As Provider Provider Consumer

Slide 23

Slide 23 text

Providing RESTful Services for IBM Domino Applications

Slide 24

Slide 24 text

Domino Access Services • Provided and Supported by IBM • Fully functional RESTful API for Domino Data • Access to Views/Folders/Documents/Collections/Fields/Calendar • Support for caching • Mail and FreeBusy services added by ExtLib OpenNTF version • Inherently uses standard security model (ACL, readers/authors…) • Server-level/Database-level/Design-level control over availability

Slide 25

Slide 25 text

Domino Access Services • No coding needed, Immediately available after a few settings. Enable on Server Enable For Database Enable For Views

Slide 26

Slide 26 text

Domino Access Services • Drawbacks: • No control over the data! • Reading a document —> Read All fields • Creating a document —> No checkpoints! • No place for business logic! • What about actions? The responsibility is on the consumer • Everything or Nothing • Exposes internals • You should trust consumers and the environment…

Slide 27

Slide 27 text

REST Components (ExtLib) • Provided and Supported by IBM • Access to Views/Folders/Documents/Collections/Fields/Calendar • Customizable component for RESTful access • Custom REST Service with SSJS or Java • Event model helps building business logic on top of REST model • Dojo support for single page model

Slide 28

Slide 28 text

REST Components (ExtLib) • Setup REST component(s) on your page. • Minimal coding, no administrator needed. Add to your XPage Add a Service Configure Options

Slide 29

Slide 29 text

REST Components (ExtLib) • Drawbacks: • Careful with the code organization… • You might inevitably end up with a spaghetti code! • Error handling is crucial • Prefer CustomRestService with a Java bean for more advanced stuff… • Difficult to follow RESTful URL Convention • e.g. https://someserver.domain.com/database.nsf/somepage.xsp/service/… • Careful with the security… • Do not trust security by obscurity…

Slide 30

Slide 30 text

Hardcoding (Web agents, XAgents…) • Obsolete way to create services, but still quite useful for some cases. • Great if you have pre-existing code (e.g. Lotusscript libraries, etc.) • Customizable, flexible and simple way to create any service

Slide 31

Slide 31 text

Hardcoding (Web agents, XAgents…) • Drawbacks: • Hardcode everything… • e.g. Header/parameter extraction • Careful with the code organization… • You might inevitably end up with a spaghetti code! • Error handling is crucial • Difficult to follow RESTful URL Convention • e.g. https://someserver.domain.com/database.nsf/xagent.xsp?… • e.g. https://someserver.domain.com/database.nsf/someagent?OpenAgent&…

Slide 32

Slide 32 text

Apache Wink Servlets • IBM Domino includes Apache Wink 1.1.2 • Create REST services using OSGi plugins. • Complete Java solution, extensible with custom providers • Conforms RESTful architecture standards using JAX-RS • Code reusability outside IBM Domino world.

Slide 33

Slide 33 text

Apache Wink Runtime Application Code Apache Wink Basic Architecture Wink Servlet (Customizable) HTTP/HTTPS Client Datastore Resource Resource Resource Resource Controllers Data Accessors Tools/Utilities Request Processor Helpers /BaseURI/* /BaseURI/Path-Patterns

Slide 34

Slide 34 text

@Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } { "zip": "13202", "state": "NY", "lastName": "Abbate", "middle": "J", "country": "US", "emailAddress": "Jessica.J.Abbate@trashymail.com", "number": "DLEY-ACLH6Y", "city": "Syracuse", "firstName": "Jessica" } Contact Resource Class Contact Resource Short JSON Representation

Slide 35

Slide 35 text

Apache Wink Servlets • Drawbacks: • Plugin only • Difficult if you are not familiar • Takes time to learn • Overkill? • Not suitable for small projects and simple needs • Apache Wink is old school • Not that bad, IBM still using Wink. But Apache took another way. • Alternatives: RESTEasy, Jersey, Apache CXF, etc. • Integrating a new module into Domino might be an issue

Slide 36

Slide 36 text

Demo

Slide 37

Slide 37 text

Providing RESTful Services on Domino Benefits Challenges Suggested When? Domino Access Services
 (DAS) No Backend Code Zero-setup Limited Control No Business Logic Exposes the Internals Simple internal integrations ExtLib Components
 for REST Less Backend Code Minimal Setup Partial/Full Customization Error Handling Spaghetti Code Risk URL Convention Simple needs for a limited scope Hardcoding
 (XAgents, Web agents, Servlets) Tailor-made (Almost) No Learning Curve Hardcoding Everything Spaghetti Code Risk URL Conventions Very specific needs for a limited scope Apache Wink Servlets Tailor-made Based on JAX-RS OSGi Benefits Learning Curve Barrier to Entry Large scope implementation, API Design

Slide 38

Slide 38 text

Consuming RESTful Services for IBM Domino Applications

Slide 39

Slide 39 text

How to Consume any RESTful Service? • It is just an HTTP Request… • Questions to ask: • Lotusscript or Java or SSJS? • On-demand or Background? • Who is going to be authenticated? • The remote service has an SDK?

Slide 40

Slide 40 text

Java or SSJS or Lotusscript? • Lotusscript • No internal support for Networking… • On Windows platform, we can use COM objects • Server-side JavaScript • Use Java! • Java • Core Java provides URLConnection class for basic operations • Open source libraries are preferred (e.g. Apache HttpComponents)

Slide 41

Slide 41 text

A Simple Example • Pulling exchange rates (When needed or on a schedule) Set rates = CreateObject("msxml2.DOMDocument" ) rates.async = False rates.validateOnParse = False rates.setProperty "ServerHTTPRequest" ,True If rates.load("http://www.tcmb.gov.tr/kurlar/today.xml" ) Then Set currencies =rates.getElementsByTagName("Currency" ) For i=0 To currencies.length - 1 Set node=currencies.item(i) If node.attributes.getNamedItem("Kod").value="USD" Then For j=0 To node.childnodes.length-1 If node.childnodes.item(j).NodeName="ForexBuying" Then forexBuyingUSD = node.childnodes.item(j).nodeTypedValue Elseif node.childnodes.item(j).NodeName="ForexSelling" Then forexSellingUSD = node.childnodes.item(j).nodeTypedValue End If Next ‘ Elseif for others… End If Next GetExchangeRates=True Else GetExchangeRates=False End If Using OLE Automation XML Parsing Windows only!

Slide 42

Slide 42 text

A Simple Example • Pulling exchange rates (When needed or on a schedule) public Double receiveEurUsdRate() throws Exception { CloseableHttpClient httpclient = HttpClients.createDefault(); HttpGet httpGet = new HttpGet("http://api.fixer.io/latest?base=USD"); CloseableHttpResponse response = httpclient.execute(httpGet); try { HttpEntity entity = response.getEntity(); // This is the worst practice ever! // We don't check anything! Everything can go wrong... JsonJavaObject ratesMap = (JsonJavaObject) JsonParser.fromJson(JsonJavaFactory.instanceEx, EntityUtils.toString(entity)); JsonJavaObject rates = ratesMap.getAsObject("rates"); // We can write values into a NotesDocument return rates.getAsDouble("EUR"); } finally { response.close(); } } Create an HTTP client Get a response Convert to JSON

Slide 43

Slide 43 text

A Simple Example • Pulling exchange rates (When needed or on a schedule) We can also call Java!

Slide 44

Slide 44 text

A Simple Example • Pulling exchange rates (When needed or on a schedule) • Lotusscript • Try not to do :) • Running from an Java Agent • Apache HttpComponents and IBM Commons should be imported. • Agent Security should be configured • Running from an XPage / Java or SSJS • Apache HttpComponents should be imported. • Time-outs and service hiccups should be considered • Values should always be cached!

Slide 45

Slide 45 text

When/How to Connect? • Web Application triggers the RESTful conversation. IBM Domino Server Web Apps RESTful Service Web Client 1. Click/Save/Update 2. REST Request 3. REST Response 4. Render Response Latency!!!

Slide 46

Slide 46 text

When/How to Connect? • Web Application performs the RESTful conversation at the browser. IBM Domino Server Web Apps RESTful Service Web Client 1. CSJS Action 2. AJAX Request 3. AJAX Response 4. Process Response Latency!!! CORS (Cross Origin Resource Sharing)?

Slide 47

Slide 47 text

When/How to Connect? • Notes Client performs the RESTful conversation IBM Domino Server Agent Manager RESTful Service Notes Client 1. Run local code (Java/LS) 2. REST Request 3. REST Response Latency!!! 4. Process Results

Slide 48

Slide 48 text

When/How to Connect? • Notes Client performs the RESTful conversation via a Server Agent IBM Domino Server Agent Manager RESTful Service Notes Client 1. Agent.RunOnServer() 2. REST Request 3. REST Response 4. Read Results Latency!!!

Slide 49

Slide 49 text

When/How to Connect? • RESTful conversation happens on background IBM Domino Server Agent Manager RESTful Service 1. Agent or DOTS run on a schedule 2. REST Request 3. REST Response 4. Process Results Latency!!! DOTS

Slide 50

Slide 50 text

When/How to Connect? • More complicated scenarios… IBM Domino Server REST Service RESTful Service RESTful Consumer 1. REST Call 3. REST Request 4. REST Response 6. Return response Latency!!! 2.Process the Request 5. Process More

Slide 51

Slide 51 text

Authentication • Who is going to be authenticated and how? • No authentication • Predefined credential or application key • Single Sign-on • Users with their own credentials on remote (OAuth, Saved password…)

Slide 52

Slide 52 text

Authentication • No authentication • No problem! • Predefined credential or application key • Every REST request should be configured with proper keys. • Credential or Application key should be secured. • Not suitable for CSJS Model

Slide 53

Slide 53 text

Authentication • Single Sign-on • Valid for integrations when SSO is available (e.g. IBM Connections) • Works within the same domain! • The remote system expects a token, generally in cookie form • CSJS model will work without any modification • When HTTP request passes through the server, cookie should be transferred

Slide 54

Slide 54 text

Authentication • Users with their own credentials on the remote system • Saved Password: • Easy for the developer, but not convenient! • Security risk, changing passwords, two-factor authentication, etc. • Many applications do not allow credentials via API. • OAuth • Domino has no native support, but possible with some effort • Need Encryption for user keys (Not yet supported in XPages) • IBM Social Business Toolkit SDK brings the support for OAUTH (well, sort of…)

Slide 55

Slide 55 text

Why not SDK? • Many remote services provide a Java SDK or Wrapper • Wrapper vs SDK • Wrapper: Well-defined data model (e.g. POJO classes) • SDK: No need to deal with low level operations • Generally supported by the provider or the community • But… • Compatibility issues (especially on Java Agents) • e.g. IBM Watson SDK requires Java 7 • JVM Security issues • Too much dependency, Poor documentation, etc.

Slide 56

Slide 56 text

Demo

Slide 57

Slide 57 text

Consuming RESTful Services Usage Examples XPages
 Java - SSJS Through SSJS or Java beans Called when needed Sending message to Slack Integrations to IBM Watson Social Media Interaction Agents Java - Lotusscript Scheduled agents Can be called by Notes Client Periodically pulling exchange rates Pushing data to remote service Training IBM Watson AI Using SDKs XPages - OSGi Access to services using external libraries Called when needed Scheduled using DOTS Accessing IBM Connections Integration to Box

Slide 58

Slide 58 text

Wrap-up

Slide 59

Slide 59 text

Summary RESTful Services Architecture Providing and Consuming RESTful services for Domino Applications Scenarios around RESTful Services Architecture Examples

Slide 60

Slide 60 text

Takeaway Download and play with demos Experiment simple services Get yourself familiar with RESTful Services Download OpenNTF projects Study on RESTful design practices Have a Pet Project

Slide 61

Slide 61 text

Notices and disclaimers Copyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights — Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Slide 62

Slide 62 text

Notices and disclaimers continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Slide 63

Slide 63 text

Resources • Serdar Başeğmez: Demo for this session
 https://github.com/sbasegmez/IC17RestDemo • Serdar Başeğmez: Apache Wink Template and Demo
 https://github.com/sbasegmez/RestAssuredDemo • Graham Acres / Serdar Başeğmez: The Journey to Becoming a Social Application Developer (IBM Connect 2014)
 https://speakerdeck.com/sbasegmez/bp308-the-journey-to-becoming-a-social-application-developer • Stephan H. Wissel: Custom REST service in XPages using a service bean
 https://wissel.net/blog/d6plinks/SHWL-9Q55QL • Eric McCormick: Series on JSON Data with Java in XPages
 https://edm00se.io/json-with-java-in-xpages • Thomas Ladehoff: REST Services with the XPages Extension Library
 https://www.assono.de/blog/d6plinks/REST-Services-with-the-XPages-Extension-Library • Paul Withers: XPages OSGi Plugins series
 http://www.intec.co.uk/xpages-osgi-plugins-1-an-introduction/ • John Cooper: Domino OSGI (Part 1) - Configuring Eclipse for XPages OSGI Plugins
 http://developmentblog.johnmcooper.co.uk/2014/05/configuring-eclipse-for-xpages-osgi-plugins-part1.html • Toby Samples: JAX-RS or THE way to do REST in Domino series
 https://tobysamples.wordpress.com/2015/04/28/jax-rs-or-the-way-to-do-rest-in-domino-part-1/ • Jesse Gallagher: Eclipse Tutorial for Domino Developers
 https://github.com/jesse-gallagher/eclipse-tutorial-oct2015/wiki/Java

Slide 64

Slide 64 text

Thank you Serdar Basegmez @serdar_basegmez http://lotusnotus.com