#engageug Six Polite Ways to Design a RESTful API for your Domino Applica

#engageug Serdar Basegmez HCL Master (2020) IBM Champion Alumni (2011 - 2018) Developi InformaGon Systems, London Notes/Domino/XPages/Java Developer, half-blooded admin! Member Director at OpenNTF Board Blog: LotusNotus.com / TwiWer: @serdar_basegmez Also blogging/speaking/podcasGng on scienGfic skepGcism / criGcal thinking

#engageug Agenda • Why should you care? • RESTful APIs • PracGcal ImplicaGons of RESTful Architectures • Designing an API • Ways to Provide REST Services for HCL Domino • Tools • Wrap-up

#engageug Why Should You Care? • New Front-ends and Enhanced UX • Faceli`ing, but keeping data and business logic in NSF. • JS Frameworks (e.g. Angular, React), Mobile Apps, etc. • Richer experiences with chatbots, AI, etc. • IntegraGon for third party sites/applicaGons • IT becomes a huge jungle, we can’t walk alone! • Financial Systems, AI Systems, CRM, S/M AutomaGon • CollaboraGve Apps, Office 365 • AutomaGng processes using APIs • Domino Apps not independent from Business Processes • AccounGng/Sales/MarkeGng/ERP Processes User Experience Business Processes Integration

#engageug RESTful Web Services Source: https://en.wikipedia.org/wiki/Representational_state_transfer Representa/onal state transfer (REST) is an architectural style used for web development. Systems and sites designed using this style aim for fast performance, reliability and the ability to scale (to grow and easily support extra users). To achieve these goals, developers work with reusable components that can be managed and updated without affec/ng the system as a whole while it is running.

#engageug History

#engageug Source: https://speakerdeck.com/jeffschenck/rest-easy-api-security-done-right User Interface Business Logic Datastore Front-end Back-end ASP, PHP, CGI, Web Agents, JSP, etc. ← HTML, CSS, JavaScript Forms → Old School Web Applica

#engageug Web Applica

#engageug Web Applica

#engageug [Web] Applica

#engageug [Web] Applica

#engageug [Web] Applica

#engageug RESTful, Everywhere! Solid Architecture Well-defined practices Widespread use Easily consumable Scalable

#engageug The Conversa

#engageug The Conversa

#engageug The Conversa

#engageug Conventions on URLs GET hTp://appserver.company.com/api/contacts GET hTp://appserver.company.com/api/contacts/UK/London POST hTp://appserver.company.com/api/contacts Retrieve Contacts / Create a new Contact…

#engageug Conventions on URLs GET hTp://appserver.company.com/api/contacts/1522 PUT hTp://appserver.company.com/api/contacts/1522 DELETE hTp://appserver.company.com/api/contacts/1522 Retrieve/Update/Delete the Contact resource with id=1522…

#engageug URLs and Resources GET POST PUT / PATCH DELETE /contacts/ List
 Contacts Create
 Contact N/A N/A /contacts/id Retrieve
 Contact N/A Update / Replace
 Contact Delete
 Contact Source: https://en.wikipedia.org/wiki/Representational_state_transfer

#engageug Different Conventions GET hTps://api.twiTer.com/1.1/statuses/show.json?id=1234567890 Retrieve the Tweet with id=1234567890…

#engageug Designing RESTful APIs… What does it mean?

#engageug Example: Collabora

#engageug Collabora

#engageug Collabora

#engageug Collabora

#engageug Designing RESTful API • Contract Design • URLs (Resources + Verbs) • Data formats • Rules, Success and Error Responses • Tip: • Start with the end product. • Write a documentaGon first, before coding. • No “correct way”, only “best pracGces” • Follow ConvenGons * Icons made by Flat Icons from www.flaticon.com

#engageug Designing RESTful API • Security Design • AuthenGcaGon / AuthorizaGon • EncrypGon • ApplicaGon Security • Tips • The scope of the implementaGon is the key. • Managing security is o`en overlooked. • Security starts with the design phase. * Icons made by Flat Icons from www.flaticon.com

#engageug Designing RESTful API • Versioning • EnumeraGon, Lifecycle • DocumentaGon • OpenAPI Specs • Generate Server stubs, client libraries • InteracGve documentaGon • TesGng • Automated tesGng * Icons made by Flat Icons from www.flaticon.com /v1/api/xyz /v2/api/xyz /api1/xyz /api2/xyz

#engageug RESTful Services for HCL Domino Applica

#engageug 6 Polite Ways of RESTful Domino 1. Domino Access Services 2. Using Extension Library Components 3. Hardcoding Services (Web Agents, XAgents, Servlets) 4. OSGi Plugins to implement JAX-RS 5. Node.js ApplicaGons with App.Dev.Pack 6. UnconvenGonal implementaGons • SmartNSF, JAX-RS inside NSF, JAX-RS outside Domino • Hybrid Methods

#engageug Domino Access Services HCL Supported Fully Func

#engageug Domino Access Services • OpenAPI definiGons (Swagger) are hosted on OpenNTF! • hWps://github.com/OpenNTF/das-api-specs

#engageug Domino Access Services • Drawbacks: • Weak control over the data! • No checkpoints on CRUD, No coding involved… • No place for business logic! • Exposes internals • Not opGmized for some use cases • You should trust consumers and the environment…

#engageug REST Components (ExtLib) • Provided and Supported by HCL • Customizable component version of the DAS • Computed/Filtered columns, Custom search, etc. • Event model helps building business logic on top of REST model • Custom REST Service • Write your own SSJS or Java bean • Write your CSJS rouGnes for async access • Remote Service / JSON-RPC • Dojo support for single page model

#engageug REST Components (ExtLib) • Setup REST component(s) on your page. • Minimal coding, no administrator needed. Add to your XPage Add a Service Configure Options

#engageug REST Components (ExtLib) • Drawbacks: • Easy to slip into a spages code! • Not opGmized for performance and scalability • Difficult to follow RESTful URL ConvenGon https://someserver.domain.com/database.nsf/somepage.xsp/service/…

#engageug Hardcoding (Web agents, XAgents, Servlets…) • Old school way; sGll quite useful for some cases. • Great if you have pre-exisGng code (e.g. Lotusscript libraries, etc.) • Also, useful to prototype quick ideas. • Customizable, flexible and simple way to create any service.

#engageug Hardcoding (Web agents, XAgents, Servlets…) • Drawbacks: • Hardcode everything… • e.g. Header/parameter extracGon • Very easy to slip into a spages code! • Error handling / proper tesGng needed • Difficult to follow RESTful URL ConvenGon https://someserver.domain.com/database.nsf/xagent.xsp?… https://someserver.domain.com/database.nsf/someagent?OpenAgent&…

#engageug Java (JAX-RS) with OSGi Plugins • JAX-RS: ‘Java-ish’ way to define RESTful services • Create JAX-RS based REST services on top of OSGi plugins. • Complete Java soluGon, extensible with custom providers • OpGons • Apache Wink 1.1.2 (DAS uses!) • Jakarta EE (Official Java) • RestEasy, Apache CXF, Jersey, etc. • Code reusability outside HCL Domino world.

#engageug JAX-RS Architecture JAX-RS Runtime Application Code Servlet (Customizable) HTTP/HTTPS Client Datastore(s) Resource Resource Resource Resource Controllers Data Accessors Tools/Utilities Request/Response Wrappers Context Helpers /BaseURI/* /BaseURI/Path-Patterns

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } { "zip": "13202", "state": "NY", "lastName": "Abbate", "middle": "J", "country": "US", "emailAddress": "Jessica.J.Abbate@trashymail.com", "number": "DLEY-ACLH6Y", "city": "Syracuse", "firstName": "Jessica" } Contact Resource Class Contact Resource Short JSON Representation

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } The base URI for the resource In the demo, the root path of the plugin is “/twink”. So this class is enabled for requests made to: /twink/contacts/*

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } This method responds to GET requests. No path defined, so this is the default responder.

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } This method also responds to GET requests. But it the request path will be elected based on this format.

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } Parameters will be injected into methods. /contacts?start=X&count=Y /contacts/someId JAX-RS servlet will handle type conversion. It supports ordinary java objects, enums, primitives, etc.

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } There are lots of options of returning response. ResponseBuilders and some other helpers make it quite easy.

#engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) { List contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } } Error handling is handled by the JAX-RS engine as well. You can inject your own errors.

#engageug @Path("/contacts") public class ContactResource { ………… @POST() @Consumes(MediaType.APPLICATION_JSON) public Response postContactJson(String body) { Contact contact = ModelUtils.buildContactfromJson(body); accessor.saveNewContact(contact); String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @POST() @Consumes(MediaType.MULTIPART_FORM_DATA) public Response postContactForm(BufferedInMultiPart formData) { Contact contact = ModelUtils.buildContactfromMultipart(formData); accessor.saveNewContact(contact); String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } This methods respond to POST requests. This time the selection depends on the incoming data type. Client marks the request with Content-Type header and Request processor will select the appropriate method here.

#engageug @Path("/contacts") public class ContactResource { ………… @POST() @Consumes(MediaType.APPLICATION_JSON) public Response postContactJson(String body) { Contact contact = ModelUtils.buildContactfromJson(body); accessor.saveNewContact(contact); String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @POST() @Consumes(MediaType.MULTIPART_FORM_DATA) public Response postContactForm(BufferedInMultiPart formData) { Contact contact = ModelUtils.buildContactfromMultipart(formData); accessor.saveNewContact(contact); String result = ModelUtils.toJson(contact).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } } Wink injects the incoming data into the method automatically. JAX-RS libraries provide their own implementation to process different data formats (Multipart, Atom, XML, JSON, etc.)

#engageug JAX-RS Methods On Apache Wink… hWps://speakerdeck.com/sbasegmez/iconuk-2016-rest-assured... On Jakarta EE… (by Jesse Gallagher) hWps://github.com/OpenNTF/org.opennx.xsp.jakartaee

#engageug JAX-RS Methods • Drawbacks: • Plugin only • Difficult if you are not familiar, Takes Gme to learn • Overkill? • Not suitable for small projects and simple needs • Tool selecGon is criGcal. • Apache Wink is old school • IntegraGng alternaGves might be difficult

#engageug Node.js • DQL + Domino AppDev Pack: • Domino Query Language • AppDev Pack: Proton Task / domino-db.js • Countless opportuniGes in the most popular framework • Different possibiliGes for the architecture • Wide range of opGons for tooling https://insights.stackoverflow.com/survey/2019

#engageug Node.js Support for Domino Apps HCL Domino Server Proton Node.js Layer domino-db.js GRPC Application ……… Application REST New Add-on Domino v10 Routers for the RESTful service Might be on the same box or not MyRoutes.js NSF

#engageug DQL: Domino Query Language • Available since Domino V10 • OpGmized query access to NSF data • Domino server extracts and opGmizes data for DQL access John Curtis, “Factory Tour DQL Performance Explanation”, https://jdcurtis.blog/2019/07/17/factory-tour-dql-performance-explanation/

#engageug IAM: Iden

#engageug Other Implementa

#engageug SmartNSF • OpenNTF Plugin • Beta-7 version. • Developers: • ChrisGan Güdemann • MarGn Jinoch • Define your REST Services from DDE! • Uses Groovy • Auto-generate OpenAPI definiGons Looking for Contributors!

#engageug JAX-RS inside NSF • IniGal work: MarGn Pradny • Included in Jakarta EE by Jesse Gallagher • JAX-RS 2.1 support (Apache Wink: JAX RS 1.1.1) • No need to deal with OSGi • Directly create annotated classes inside NSF • OpenAPI SpecificaGon annotaGons supported https://www.pradny.com/2019/05/using-jax-rs-inside-nsf-follow-up.html

#engageug Integrate Domino into Open Liberty • ImplementaGon: Jesse Gallagher • Inspired by Sven Hasselbach • Run Java EE server • e.g. Open Liberty, open source variant of Websphere Liberty • Run Domino HTTP Stack side by side • Deploy Java EE projects connecGng to Domino data naGvely. • HTTP/2, Servlet 4, Websockets, etc. • High performance, scalability https://frostillic.us/blog/posts/2019/1/3/be9501a07279b40d85258377007b0dfd

#engageug Bonus…

#engageug Hybrid Approaches • Wrap services using a suitable front-end • e.g. Express, Node-RED, etc. • Add some magic • Extra processing, computaGon, security, etc. • Wrap missing parts from exisGng assets • Combine best of all worlds • Consistency across the soluGon • OpGmized ImplementaGon Internal Domino Server Node.js JAX-RS Service ExtLib REST Service DAS - Data API Proton domino-db.js JavaScript Services Wrapped Services Data / Application Layer Non-Domino Services API Clients API Clients API Clients

#engageug Tooling - OpenAPI • OpenAPI SpecificaGons / DocumentaGon • Swagger Tools - swagger.io • Stoplight Studio - stoplight.io • Apicurio - apicur.io

#engageug Tooling - Tes

#engageug Thank you! Serdar Basegmez Developi serdar_basegmez lotusnotus.com

#engageug Resources • Serdar Başeğmez: Node.js demo for this session
 hWps://github.com/sbasegmez/Engage19Demo • Tom Van Aken: Rest calls and JSON Parsing in LotusScript
 hWps://vanakentom.wordpress.com/2019/01/15/rest-calls-and-json-parsing-in-lotusscript/ • Serdar Başeğmez: Demo for IBM Connect 2017 session
 hWps://github.com/sbasegmez/IC17RestDemo • Serdar Başeğmez: Apache Wink Template and Demo for Icon UK 2016
 hWps://github.com/sbasegmez/RestAssuredDemo • Graham Acres / Serdar Başeğmez: The Journey to Becoming a Social ApplicaGon Developer (IBM Connect 2014)
 hWps://speakerdeck.com/sbasegmez/bp308-the-journey-to-becoming-a-social-applicaGon-developer • Stephan H. Wissel: Custom REST service in XPages using a service bean
 hWps://wissel.net/blog/d6plinks/SHWL-9Q55QL

#engageug Resources (cont.) • Eric McCormick: Series on JSON Data with Java in XPages
 hWps://edm00se.io/json-with-java-in-xpages • Thomas Ladehoff: REST Services with the XPages Extension Library
 hWps://www.assono.de/blog/d6plinks/REST-Services-with-the-XPages-Extension-Library • Paul Withers: XPages OSGi Plugins series
 hWp://www.intec.co.uk/xpages-osgi-plugins-1-an-introducGon/ • John Cooper: Domino OSGI (Part 1) - Configuring Eclipse for XPages OSGI Plugins
 hWp://developmentblog.johnmcooper.co.uk/2014/05/configuring-eclipse-for-xpages-osgi-plugins-part1.html • Toby Samples: JAX-RS or THE way to do REST in Domino series
 hWps://tobysamples.wordpress.com/2015/04/28/jax-rs-or-the-way-to-do-rest-in-domino-part-1/ • Jesse Gallagher: Eclipse Tutorial for Domino Developers
 hWps://github.com/jesse-gallagher/eclipse-tutorial-oct2015/wiki/Java

#engageug Resources (cont.) • Ulrich Krause: NotesJsonNavigator, NotesJsonElement, NotesJsonArray, NotesJsonObject example
 hWps://www.eknori.de/2019-01-01/notesjsonnavigator-notesjsonelement-notesjsonarray-notesjsonobject-example/ • John Dalsgaard: REST Services in IBM Domino/XWork
 hWps://www.dalsgaard-data.eu/blog/rest-services-in-ibm-dominoxwork • Oliver Busse: First dive into Domino and Node.js
 hWp://oliverbusse.notesx.net/hp.nsf/blogpost.xsp?documentId=2E52 • Oliver Busse: Domino, Proton, IAM, OAuth (Series)
 hWp://oliverbusse.notesx.net/hp.nsf/blogpost.xsp?documentId=2FEA • Sven Hasselbach: node.js, domino-db & Docker
 hWp://hasselba.ch/blog/?p=2670 • Jesse Gallagher: That Java Thing, Part 1: The Java Problem in the Community
 hWps://frosGllic.us/blog/thread.xsp?thread=That+Java+Thing

#engageug Resources (cont.) • Graham Acres / Heiko Voigt: Engage 2019 Demo of Domino v10, Proton and Node.js session 
 hWps://github.com/c3ug/engage2019democode • Troy Reimer: OpenNTF Project: LotusScript implementaGon of a reader and writer for JSON
 hWps://opennx.org/main.nsf/project.xsp?r=project/JSON%20LotusScript%20Classes • Bansilal Haudakari: EffecGve API Design 
 hWps://www.slideshare.net/BansilalHaudakari/effecGve-api-design • Heiko Voigt: How to set up the Domino 10 OAUTH2 Provider - IAM Basics 
 hWps://www.harbour-light.org/2019/02/18/how-to-set-up-the-domino-10-oauth2-provider-iam-basics/ • Mar