Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Engage 2019: Your Data in the Major Leagues: A Practical and Updated Guide to RESTful Domino v10

Engage 2019: Your Data in the Major Leagues: A Practical and Updated Guide to RESTful Domino v10

Engage 2019 session with Tom Van Aken (https://twitter.com/vanakentom):

Your Data in the Major Leagues: A Practical and Updated Guide to RESTful Domino v10

Things are moving fast with many new tools and toys in our hands. While Domino v10 creates a whole new world of possibilities, we still need to interact with other internal and external applications. One of the key technologies to do that is the RESTful services and we have an updated toolbox now. Regardless of your experience, come to this session to see what options are available to you, what pitfalls you may experience and how to break down the borders between your applications and others. We will present the basic concepts and best practices, allowing you to walk away with the techniques that you will need to take your applications to the next level. Last but not the least, we will show you practical examples on how you provide RESTful services for other applications and how you can use REST data to enrich your applications using the the new capabilities.

Serdar Basegmez , Developi Information Systems
Tom Van Aken, GroupWave


May 15, 2019

More Decks by sbasegmez

Other Decks in Programming


  1. #engageug Your Data in the Major Leagues: A Prac6cal and

    Updated Guide to RESTful Domino v10 Serdar Basegmez Developi serdar_basegmez vanakentom Tom Van Aken GroupWave
  2. #engageug Serdar Basegmez • IBM Champion Alumni (2011 - 2018)

    • Developi InformaCon Systems, London • Notes/Domino/XPages Developer, half-blooded admin! • Member Director at OpenNTF Board • Blog: LotusNotus.com [English / Turkish] • Blogger and Podcaster on ScienCfic SkepCcism / CriCcal Thinking serdar_basegmez
  3. #engageug Tom Van Aken • First Cme IBM Champion in

    2019 • Groupwave, KonCch - Belgium • Notes Domino / Xpage developer • Blog: vanakentom.wordpress.com [Dutch] vanakentom
  4. #engageug Agenda • RESTful Architecture • PracCcal ImplicaCons and Advantages

    of RESTful Services • Ways to Provide REST Services for IBM Domino • Consuming REST Services within IBM Domino • Wrap-up
  5. #engageug RESTful Web Services Source: https://en.wikipedia.org/wiki/Representational_state_transfer Representa)onal state transfer (REST)

    is an architectural style used for web development. Systems and sites designed using this style aim for fast performance, reliability and the ability to scale (to grow and easily support extra users). To achieve these goals, developers work with reusable components that can be managed and updated without affec)ng the system as a whole while it is running.
  6. #engageug Source: https://speakerdeck.com/jeffschenck/rest-easy-api-security-done-right User Interface Business Logic Datastore Front-end Back-end

    ASP, PHP, CGI, Web Agents, JSP, etc. ← HTML, CSS, JavaScript Forms → Old School Web ApplicaCons
  7. #engageug Web ApplicaCons Evolving User Interface Business Logic Datastore Front-end

    Back-end Async web apps, Ruby on Rails, Django, JSF, XPages, etc. ← HTML, CSS, JavaScript Forms, AJAX →
  8. #engageug Web ApplicaCons Evolving User Interface Business Logic Datastore Front-end

    Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST →
  9. #engageug [Web] ApplicaCons Evolving User Interface Business Logic Datastore Mobile

    Applications Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST → Front-end
  10. #engageug [Web] ApplicaCons Evolving User Interface Business Logic Datastore Mobile

    Applications Back-end Modern Web frameworks, Angular.js, React.js, etc. ← HTML, CSS, JavaScript ← REST → Front-end Microservice Microservice Microservice
  11. #engageug [Web] ApplicaCons Evolving Back-end Modern Web frameworks, Angular.js, React.js,

    etc. ← HTML, CSS, JavaScript ← REST → User Interface Business Logic Datastore Mobile Applications Front-end External Apps Microservice Microservice Microservice Integration
  12. #engageug The ConversaCon Makes Sense! GET /twink/contacts/DLEY-ACLH6Y HTTP/1.1 Host: homer.developi.info

    Cache-Control: no-cache { "zip": "13202", "state": "NY", "lastName": "Abbate", "middle": "J", "country": "US", "emailAddress": "[email protected]", "number": "DLEY-ACLH6Y", "city": "Syracuse", "firstName": "Jessica" }
  13. #engageug URI GET PUT POST DELETE /contacts/ List Contacts Replace

    Contacts Create New Contact Delete Contacts /contacts/id Retrieve a Contact Replace a Contact N/A (generally) Delete a Contact Source: https://en.wikipedia.org/wiki/Representational_state_transfer
  14. #engageug Some InspiraCon • New Front-ends and Enhanced UX •

    JS Frameworks (e.g. Angular, React), Mobile Apps, etc. • Grids, quick searches, becer type-aheads, etc. • Richer experiences with chatbots, AI, etc. • IntegraCon for third party sites/applicaCons • Pull exchange rates, get weather info • Get customers from CRM, create leads • Upload to Box, Send to Slack… • AutomaCng processes using APIs • Push data to the AccounCng system… • Access to ERP, check the inventory, iniCate a new PO User Experience Business Processes Integration
  15. #engageug Providing RESTful Services Domino Access Services Extension Library REST

    Components Hardcoded Solutions (Web Agents, XAgents, Servlets) Advanced J2EE (JAX-RS) SmartNSF Node.js #OpenNTF #Domino10
  16. #engageug Domino Access Services IBM Supported Fully Func6onal REST API

    Access to Data / Calendar / Mail* / Freebusy* / Directory** * Mail and Freebusy access is part of the core product since 9.0.1FP10 / ** Directory access is enabled with the ExtLib version of OpenNTF Caching / Na6ve Security (ACL, Reader/Author fields, etc.) Server-level / Database-level / Design-level control Enable on Server Enable For Database Enable For Views
  17. #engageug Domino Access Services • OpenAPI definiCons (Swagger) are hosted

    on OpenNTF! • hcps://github.com/OpenNTF/das-api-specs
  18. #engageug Domino Access Services • Drawbacks: • Weak control over

    the data! • No checkpoints on CRUD, No coding involved… • Weak control over the access. • No place for business logic! • Consumer has to implement all the business logic. • Exposes internals • You should trust consumers and the environment…
  19. #engageug REST Components (ExtLib) • Provided and Supported by IBM

    • Customizable component version of the DAS • Computed/Filtered columns, Custom search, etc. • Event model helps building business logic on top of REST model • Custom REST Service • Write your own SSJS or Java bean • Write your CSJS rouCnes for async access • Remote Service / JSON-RPC • Dojo support for single page model
  20. #engageug REST Components (ExtLib) • Setup REST component(s) on your

    page. • Minimal coding, no administrator needed. Add to your XPage Add a Service Configure Options
  21. #engageug REST Components (ExtLib) • Drawbacks: • Easy to slip

    into a spageh code! • Java beans are your friends… • Not opCmized for performance and scalability • Difficult to follow RESTful URL ConvenCon https://someserver.domain.com/database.nsf/somepage.xsp/service/…
  22. #engageug Hardcoding (Web agents, XAgents, Servlets…) • Old school way;

    sCll quite useful for some cases. • Great if you have pre-exisCng code (e.g. Lotusscript libraries, etc.) • Customizable, flexible and simple way to create any service
  23. #engageug Hardcoding (Web agents, XAgents, Servlets…) • Drawbacks: • Hardcode

    everything… • e.g. Header/parameter extracCon • Very easy to slip into a spageh code! • Error handling / proper tesCng needed • Difficult to follow RESTful URL ConvenCon https://someserver.domain.com/database.nsf/xagent.xsp?… https://someserver.domain.com/database.nsf/someagent?OpenAgent&…
  24. #engageug J2EE (JAX-RS) Methods • JAX-RS: ‘Java-ish’ way to define

    RESTful services • Create JAX-RS based REST services on top of OSGi plugins. • Complete Java soluCon, extensible with custom providers • OpCons • Apache Wink 1.1.2 (DAS uses!) • Jakarta EE • RestEasy, Apache CXF, Jersey, etc. • Code reusability outside IBM Domino world.
  25. #engageug J2EE (JAX-RS) Methods • Drawbacks: • Plugin only •

    Difficult if you are not familiar, Takes Cme to learn • Overkill? • Not suitable for small projects and simple needs • Tool selecCon is criCcal. • Apache Wink is old school • IntegraCng alternaCves might be difficult
  26. #engageug SmartNSF • OpenNTF Plugin • Beta-7 version. • Developers:

    • ChrisCan Güdemann • MarCn Jinoch • Define your REST Services from DDE! • Uses Groovy • Auto-generate OpenAPI definiCons Looking for Contributors!
  27. #engageug Node.js • Just arrived: DQL + Domino AppDev Pack:

    • Domino Query Language • AppDev Pack: Proton Task / domino-db.js • Countless opportuniCes in the most popular framework • Different possibiliCes for the architecture • Wide range of opCons for tooling https://insights.stackoverflow.com/survey/2019
  28. #engageug Node.js IBM Domino Server Proton NSF Node.js Layer domino-db.js

    GRPC Application ……… Application REST New Add-on Domino v10 Routers for the RESTful service Might be on the same box or not MyRoutes.js
  29. #engageug Providing RESTful Services on Domino Benefits Challenges Suggested When?

    Domino Access Services
 (DAS) Zero-setup True RESTful No Programming No Business Logic Exposes the Internals Simple data exchange ExtLib Components
 for REST Less Backend Code Minimal Setup Partial/Full Customization Error Handling Spaghetti Code Risk URL Convention Simple needs for a limited scope Hardcoding
 (XAgents, Web agents…) Tailor-made Any Language is OK Hardcoding Everything Spaghetti Code Risk URL Conventions Very specific needs for a limited scope J2EE / JAX-RS Tailor-made OSGi Benefits True RESTful Learning Curve Barrier to Entry Environment Setup Large scope implementation, API Design Node.js Tailor Made Node.js Benefits True RESTful Learning Curve Still Improving Environment Setup Modernization Projects SmartNSF Beta Less Backend Code True RESTful Learning Curve Limited Support Simple needs for a limited scope
  30. #engageug Consuming RESTful Services …or Teaching your app to browse

    the Internet… Language Preference? Is it On-demand or Background? Server-side or Client-side? SDK? Authentication?
  31. #engageug Language Preference • Server-side JavaScript • Already uses Java.

    Eliminate the middleman… • Java • XPages Managed Beans, Java Agents, OSGi Plugins, etc. • ConnecCng a remote • URLConnecCon (Core Java) • HcpComponents (Apache Commons) • JSON Processing • JsonJavaObject (IBM Commons) • Jackson, Gson, etc.
  32. #engageug ConnecCng Java • Pulling exchange rates via Java (Agent

    or Managed Beans) public Double receiveEurUsdRate() throws Exception { CloseableHttpClient httpclient = HttpClients.createDefault(); HttpGet httpGet = new HttpGet("http://api.fixer.io/latest?base=USD"); CloseableHttpResponse response = httpclient.execute(httpGet); try { HttpEntity entity = response.getEntity(); // This is the worst practice ever! // We don't check anything! Everything can go wrong... JsonJavaObject ratesMap = (JsonJavaObject) JsonParser.fromJson(JsonJavaFactory.instanceEx, EntityUtils.toString(entity)); JsonJavaObject rates = ratesMap.getAsObject("rates"); // We can write values into a NotesDocument return rates.getAsDouble("EUR"); } finally { response.close(); } } * Networking is a restricted operation for Agents! Agent Security should be configured. Create an HTTP client Get a Response Convert to JSON
  33. #engageug ConnecCng SSJS • Pulling exchange rates on an XPage

    acCon <xp:button id="button1" value="What is the exchange rate?"> <xp:eventHandler event="onclick" submit="true" refreshMode="partial" refreshId="computedField1"> <xp:this.action><![CDATA[#{javascript: viewScope.EurUsdRate = test.ExchangeRate.receiveEurUsdRate() }]]></xp:this.action> </xp:eventHandler> </xp:button> * This code will run multiple times for each page cycle. Always cache values! Call Java (via Managed Bean)
  34. #engageug Reminder for Java • Consuming any REST Service (When

    needed or on a schedule) • Running from an Java Agent • Import Apache HcpComponents/IBM Commons • Configure Agent Security • Running from an XPage / Java or SSJS • Apache HcpComponents should be imported per NSF. • Time-outs and service hiccups should be considered • Values should always be cached!
  35. #engageug Language Preference • Using SDK/Wrapper • Saves Cme and

    effort • Ower provided and supported by the vendor/community. • Box, IBM Watson, Salesforce, Quickbooks, etc. • AdapCng to Domino • CompaCbility issues. • Java version mismatch (prior to v9.0.1 FP9), Java security issues, etc.
  36. #engageug Language Preference • Lotusscript • No internal support for

    Networking… • On Windows plaporm, we can use COM objects • Call Java via LS2J N O M O R E !
  37. #engageug New CapabiliCes of Domino v10! • Create HTTP requests…

    • NotesHTTPRequest • Manipulate JSON… • NotesJSONNavigator • NotesJSONElement • NotesJSONArray • NotesJSONObject
  38. #engageug NotesHTTPRequest • GET • POST • PUT • PATCH

    (V10.0.1) • DELETERESOURCE (V10.0.1) • Preferstrings • False: As Byte Array • True: As Text
  39. #engageug Parsing JSON • NotesJSONNavigator • getElementByName • getElementByPointer •

    gepirstElement / getnextElement / getNthElement • preferUTF8 • False : gets String values • True : gets Byte Array in UTF-8 • NotesJSONElement • NotesJSONArray
  40. #engageug Work has to be done • CreateJsonNavigator causes problems

    with CR/LF and internaConal characters • SoluCon=save file as UTF-8 and read as binary • JSONNavigator doesn’t handle internaConal characters well • Will be solved by FP2. • Only JSON parsing, No creaCon of JSON (yet?) • AlternaCve: ls.snapps.json… libraries • No REST compliant Agent NotaCon • Makes it less suitable for Providing REST services
  41. #engageug Consuming RESTful Services Usage Examples XPages
 Java - SSJS

    Through SSJS or Java beans Call when needed Sending message to Slack Integrations to IBM Watson Social Media Interaction Agents Java - Lotusscript Scheduled agents Call by Notes Client Periodically pulling exchange rates Pushing data to remote service Training IBM Watson AI Using SDKs Access to services using libraries Call when needed Schedule / Trigger Accessing IBM Connections Integration to Box
  42. #engageug Architecture… RESTful Remote Web Browser Domino Server Web App

    Agents XPages Notes Client On Demand REST Conversation Scheduled REST Conversation On Demand REST Conversation On Demand REST Conversation
  43. #engageug Architecture Decision… • Who iniCates a RESTful conversaCon? When?

    How? • Trigger: On demand vs. Schedule vs. Event-based • How dynamic is the remote data? Can we use caching? • The remote can be accessed by the server or the client? • Intranet services cannot be accessed by remote users. • What about security? Who has the password for the remote? • SSO / OAuth can be a challenge in many cases. • Predefined credenCals might be exposed in some cases.
  44. #engageug AuthenCcaCon • Key quesCon, depending on the remote service.

    • Who is going to be authenCcated and how? • Predefined credenCal or applicaCon key • Single Sign-on • Users’ own credenCals on remote (OAuth, Password…)
  45. #engageug Some Tweaks • SSL Issue • TLS 1.2 is

    required in most cases. • Technote: hcps://www-01.ibm.com/support/docview.wss? uid=swg21985289 • HTTP Proxy/Gateway • Proxy through the Domino Server for CSJS requests (Mostly for security) • OpCons • IBM Domino provides HTTP-proxy servlet • ProgrammaCcally create a gateway using “Remote Service” component • TesCng • Postman / MockBin / RequestBin / hcpbin / JSON View
  46. #engageug Resources • Serdar Başeğmez: Node.js demo for this session

    hcps://github.com/sbasegmez/Engage19Demo • Tom Van Aken: Rest calls and JSON Parsing in LotusScript
 hcps://vanakentom.wordpress.com/2019/01/15/rest-calls-and-json-parsing-in-lotusscript/ • Serdar Başeğmez: Demo for IBM Connect 2017 session
 hcps://github.com/sbasegmez/IC17RestDemo • Serdar Başeğmez: Apache Wink Template and Demo for Icon UK 2016
 hcps://github.com/sbasegmez/RestAssuredDemo • Graham Acres / Serdar Başeğmez: The Journey to Becoming a Social ApplicaCon Developer (IBM Connect 2014)
 hcps://speakerdeck.com/sbasegmez/bp308-the-journey-to-becoming-a-social-applicaCon-developer • Stephan H. Wissel: Custom REST service in XPages using a service bean
  47. #engageug Resources (cont.) • Eric McCormick: Series on JSON Data

    with Java in XPages
 hcps://edm00se.io/json-with-java-in-xpages • Thomas Ladehoff: REST Services with the XPages Extension Library
 hcps://www.assono.de/blog/d6plinks/REST-Services-with-the-XPages-Extension-Library • Paul Withers: XPages OSGi Plugins series
 hcp://www.intec.co.uk/xpages-osgi-plugins-1-an-introducCon/ • John Cooper: Domino OSGI (Part 1) - Configuring Eclipse for XPages OSGI Plugins
 hcp://developmentblog.johnmcooper.co.uk/2014/05/configuring-eclipse-for-xpages-osgi-plugins-part1.html • Toby Samples: JAX-RS or THE way to do REST in Domino series
 hcps://tobysamples.wordpress.com/2015/04/28/jax-rs-or-the-way-to-do-rest-in-domino-part-1/ • Jesse Gallagher: Eclipse Tutorial for Domino Developers
  48. #engageug Resources (cont.) • Ulrich Krause: NotesJsonNavigator, NotesJsonElement, NotesJsonArray, NotesJsonObject

 hcps://www.eknori.de/2019-01-01/notesjsonnavigator-notesjsonelement-notesjsonarray-notesjsonobject-example/ • John Dalsgaard: REST Services in IBM Domino/XWork
 hcps://www.dalsgaard-data.eu/blog/rest-services-in-ibm-dominoxwork • Oliver Busse: First dive into Domino and Node.js
 hcp://oliverbusse.notesx.net/hp.nsf/blogpost.xsp? documentId=2E52&SessionID=00997C8066130BED21F9B372F01DADDA32214EAE • Oliver Busse: Domino, Proton, IAM, OAuth (Series)
 hcp://oliverbusse.notesx.net/hp.nsf/blogpost.xsp?documentId=2FEA • Sven Hasselbach: node.js, domino-db & Docker
 hcp://hasselba.ch/blog/?p=2670 • Jesse Gallagher: That Java Thing, Part 1: The Java Problem in the Community
  49. #engageug Resources (cont.) • Graham Acres / Heiko Voigt: Engage

    2019 Demo of Domino v10, Proton and Node.js session 
 hcps://github.com/c3ug/engage2019democode • Troy Reimer: OpenNTF Project: LotusScript implementaCon of a reader and writer for JSON