Engage 2023: Taking Domino Apps to the next level by providing a Rest API

Session De13 from Engage User Group Conference 2023 with Tom Van Aken

"Providing RESTful APIs has become an increasingly common requirement in recent times. Whether you're just starting a new project or looking to level up your existing Notes/Domino application, finding the right approach is the key to success. By giving you some real-world scenario's and examples, this session will arm you with the knowledge and best practices to help you pick the right tools for the right job when it comes to performance, scalability, and security.
We'll also discuss pros and cons of the brand new Domino REST API (Project Keep) and custom development with Java or LotusScript. Don't miss this opportunity to uncover the hidden gems and avoid the pitfalls of integrating external systems with your Domino platform. Join us and be ready to take your platform to new heights!"

(25/April/2023)

Transcript

1. #engageug DE13 Taking Domino Apps to the Next Level by Providing a REST API Tom Van Aken, Groupwave Serdar Basegmez, Developi Engage 2023
2. #engageug Who is speaking to you today? Working at Groupwave Table tennis youth coach 
 Notes/Domino Developer since R3.5 IBM Champion (2019) HCL Ambassador (2020-2023) @vanakentom vanakentom.wordpress.com linkedin.com/in/tomvanaken Tom Van Aken Developer/Half-blooded Admin Ex-Istanbulite, New(ish) Londoner Developi UK, OpenNTF Board Notes/Domino since 1999 IBM Champion Alumni (2011-2018) HCL Ambassador (2020-2023) @serdar_basegmez lotusnotus.com linkedin.com/in/sbasegmez Serdar Basegmez
3. #engageug Our session today ๏What is in our session? ๏ REST API Overview ๏ Providing a REST API from a Domino Server ๏ Building a REST API for Existing Applications ๏ Best Practices for REST API Development ๏ Q&A
4. #engageug REST API Overview
5. #engageug What is REST API? “Representational state transfer (REST) is a software architectural style that describes the architecture of the Web.” (Source: Wikipedia) “An application programming interface (API) is a way for two or more computer programs to communicate with each other.” (Source: Wikipedia)
6. #engageug RESTful, Everywhere! ๏Solid Architecture ๏Well-defined practices ๏Widespread use ๏Intuitive Design ๏Easily consumable ๏Scalable Source: https://openclipart.org/detail/221722/cloud-network
7. #engageug Back-end User Interface Business Logic Datastore Mobile Applications Front-end External Apps Microservice Microservice Microservice Integration Architectures are Evolving User Interface Business Logic Datastore Front-end Back-end
8. #engageug Why Should You Care? User Experience Business Processes Integration ➡ Facelifting front-end using JS frameworks (Angular, React, etc.) ➡ Richer experiences with chatbots, AI, mobile apps, wearables etc. ➡ Financial Systems, AI Systems, CRM, S/M Automation ➡ Collaborative Apps, Office 365 ➡ Domino Apps not independent from Business Processes ➡ Accounting/Sales/Marketing/ERP Processes
9. #engageug Providing a REST API from a Domino Server
10. #engageug Choose the Path, You must! ๏Domino REST API (a.k.a. Project Keep) ๏XPages / Java ๏Classical Notes Development / LotusScript ๏App. Dev. Pack
11. #engageug Using Domino REST API (or Project Keep)
12. #engageug Domino REST API (Project Keep) ๏Official REST API for the Domino apps ๏ Connects Domino to Volt MX ๏Supersedes Domino Access Services ๏Much Broader approach ๏ Schema-based forms ๏ Selective data with Views ๏ Async Agent trigger ๏ Design API ๏Modern Security https://opensource.hcltechsw.com/Domino-rest-api/topicguides/introducingrestapi.html
13. #engageug Domino REST API: Overview ๏Control what to expose ๏ Schemas and Scopes ๏ Tweaking field names ๏ “NID” → “storyId” ๏ “NCreationDate” → “date” ๏ Same data, different audience ๏Agent Processing ๏ Run or Queue agents by API ๏ Provide context
14. #engageug Domino REST API: Overview ๏Security ๏ Authentication with JWT / External IdP ๏ Domino login (can be disabled) ๏ OAuth access ๏ Opt-in & Overlay security ๏ Access denied by default, ๏ Domino Security > Scope Configuration ๏OData support ๏ Salesforce, Excel, SAP, etc. ๏Hosting static apps
15. #engageug Domino REST API: Architecture ๏Runs over Domino Server ๏ Install to Server / Run as Docker ๏ Dev/test instance on Client ๏ “restapi” task ๏ Separate HTTP listener ๏Java 8 and Eclipse Vert.x ๏ EventBus to worker verticles ๏ New verticles can be added (Extensibility)
16. #engageug Domino REST API: Summary – Go to opensource.hcltechsw.com/Domino-rest-api – Start using it today and provide feedback! – Implements latest open standards – Security by default – Extensible – Well documented – What not to expect: – Mostly Data, with simple business logic – More to come
17. #engageug XPages / Java Path .xsp
18. #engageug XPages / Java Path ExtLib REST Components Quick and Dirty services for existing (XPages) Apps or XPages Jakarta EE Support OpenNTF project by Jesse Gallagher Using JAX-RS in Java classes
19. #engageug REST Components (ExtLib) ๏A customisable wrapper of the Domino Access Services (DAS) ๏ Custom columns, Custom search, etc. ๏ Business logic on top of the REST model via events ๏Custom REST Service ๏ Write your own SSJS or Java bean ๏Dojo Support for single-page apps ๏Minimal coding, no administrator needed Add to your XPage Add a Service Configure Options
20. #engageug REST Components (ExtLib) ๏Viable for… ๏ Quick and dirty solutions ๏ Prototyping ๏ Single service cases ๏Drawbacks: ๏ Easy to slip into a spaghetti code! ๏ Not optimised for performance and scalability ๏ Challenging to follow the RESTful URL Convention https://someserver.domain.com/database.nsf/somepage.xsp/service/…
21. #engageug Java (JAX-RS) ๏JAX-RS: Jakarta RESTful Web Services ๏ ‘Java-ish’ way to define RESTful services ๏ Use special “annotations” in Java classes. ๏ Extensible with custom providers ๏XPages Jakarta EE Support ๏ OpenNTF project by Jesse Gallagher ๏ Provide JAX-RS support for XPages apps
22. #engageug JAX-RS Inside NSF JAX-RS Runtime Application Code Services Servlet HTTP/HTTPS Client Datastores (NSFs) Resource Resource Resource Resource Controllers Data Accessors Tools/Utilities Request/Response Wrappers Context Helpers /db.nsf/xsp/app/* /db.nsf/xsp/app/story Java Classes in NSF Jakarta EE Plugin /db.nsf/xsp/app/author /db.nsf/xsp/app/tag
23. #engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList(@QueryParam("start") int start, @QueryParam("count") int count) { List<Contact> contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() @Produces(MediaType.APPLICATION_JSON) public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { return contact; } } } JAX-RS Development The base URI for the resource XPages Jakarta EE Support will assign a path underneath the NSF. So this class is enabled for requests made to: /db.nsf/xsp/app/contacts/ /db.nsf/xsp/app/contacts/id
24. #engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList(@QueryParam("start") int start, @QueryParam("count") int count) { List<Contact> contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() @Produces(MediaType.APPLICATION_JSON) public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { return contact; } } } i8 This method responds to GET requests. No path is defined, so this is the default responder. This method also responds to GET requests. But the request path will be used to select the right one
25. #engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList(@QueryParam("start") int start, @QueryParam("count") int count) { List<Contact> contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() @Produces(MediaType.APPLICATION_JSON) public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { return contact; } } } JAX-RS Development Parameters will be injected into methods. /contacts?start=X&count=Y /contacts/id JAX-RS servlet will handle type conversion. It supports ordinary Java objects, enums, primitives, etc.
26. #engageug @Path("/contacts") public class ContactResource { private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession()); @GET() public Response getContactList(@QueryParam("start") int start, @QueryParam("count") int count) { List<Contact> contactList = accessor.pullContacts(start, count); String result = ModelUtils.toJson(contactList).toString(); return Response.ok(result, MediaType.APPLICATION_JSON).build(); } @Path("/{id}") @GET() @Produces(MediaType.APPLICATION_JSON) public Response getContact(@PathParam("id") String id) { Contact contact = accessor.findContact(id); if(null == contact) { throw new WebApplicationException(Response.Status.NOT_FOUND); } else { return contact; } } } JAX-RS Development There are lots of options of returning response. ResponseBuilders and some other helpers make it quite easy. JAX-RS will handle error responses “contact” will be automatically converted to JSON
27. #engageug @Path("/contacts") public class ContactResource { @Inject DominoAccessor accessor; @POST() @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.APPLICATION_JSON) public Response postContactJson(String body) { Contact contact = ModelUtils.buildContactfromJson(body); accessor.saveNewContact(contact); return contact; } @POST() @Consumes(MediaType.MULTIPART_FORM_DATA) @Produces(MediaType.APPLICATION_JSON) public Response postContactForm(BufferedInMultiPart formData) { // This is for Apache Wink! Contact contact = ModelUtils.buildContactfromMultipart(formData); accessor.saveNewContact(contact); return contact; } } JAX-RS Development POST requests handled based on the incoming data type JAX-RS libraries provide their own implementation to process different data formats (Multipart, Atom, XML, JSON, etc.) Supports dependency injection
28. #engageug XPages / Java Path Summary ๏An excellent option for XPages developers ๏Potential to transfer to non-Domino platforms ๏Disadvantages: ๏ JAX-RS has a learning curve ๏ Easy to use. Needs conceptual understanding ๏ Too many moving parts ๏ Development experience is not the best
29. #engageug Classical Notes Development / LotusScript
30. #engageug At a glance
31. #engageug Lotusscript Web Agents - Advantages ๏'Out of the Box' domino - no extra's required ๏Both CRUD and ACTION operations ๏Small learning curve (if you know LotusScript) ๏Reuse of existing LotusScript code
32. #engageug Lotusscript Web Agents - Disadvantages ๏No standard URLS /v2/appointments?OpenAgent&id=20230418_1515_6QY5MV ๏Only GET and POST methods ๏Limitations on authentication ๏Limitation on response size
33. #engageug AppDev-Pack
34. #engageug HCL Domino AppDev Pack ๏Domino AppDev Pack: ๏ Proton Task / domino-db.js / com.hcl.domino.db ๏ Node.js + Java support for Domino data access ๏ gRPC protocol to provide native support ๏Wide range of options for tooling and architecture ๏Potential implementations in other languages ๏Identity and Access Management
35. #engageug Node.js Support for Domino Apps HCL Domino Server Proton Node.js Layer domino-db.js GRPC Application ……… Application REST Add-on Installed Routers for the RESTful service Might be on the same box or not MyRoutes.js NSF
36. #engageug Domino AppDev Pack Summary ๏AppDev Pack vs Domino REST API ๏ GRPC vs HTTP ๏ Who has control? Domino team or “other” team? ๏ Language and Framework Preference ๏Disadvantages ๏ No significant development in 2 years ๏ Only data access ๏ Difficult to install
37. #engageug Building a REST API for Existing Applications
38. #engageug Extending your Applications LotusScript
39. #engageug Extending your apps Lotusscript Web Agents Server configuration Writing your endpoints
40. #engageug Server configuration ๏Configure your web server as usual ๏Create a separate Internet site. ๏ Change security settings depending on requirements. eg: SSO/SAML for web apps vs. functional users for API ๏Set Allowed Methods ๏ For LotusScript API: GET, POST, HEAD and OPTIONS
41. #engageug Basic Agent settings ๏Agent settings ๏ ‘Run as Web user’ for credentials authenticated users ๏ Set agent name or alias as endpoint URL
42. #engageug Lotusscript agent – the request ๏Get Request information through document context. PATH_TRANSLATED URL path information /demo/cars.nsf/helloworld QUERY_INFO_DECODED URL query parameters (after ?) OpenAgent&color=red REQUEST_METHOD Method used for the request GET or POST CONTENT_TYPE Content type of the request application/json CONTENT_REQUEST Content, only for POST requests
43. #engageug Lotusscript agent – the response ๏Print statements to write your response Print {Content-Type:application/json} Print {status:200} ๏NotesJson(…) classes to create JSON response ๏ NotesJsonNavigator ๏ NotesJsonElement ๏ NotesJsonArray ๏ NotesJsonObject
44. #engageug Lotusscript agent – example Set context = session.Documentcontext 'Make sure the response is set to json Print {Content-Type:application/json} 'Get the selected color using a url parameter Dim qs As String qs = context.Query_String_Decoded(0) Dim param_color As String param_color = StrRight(qs, "&color=") 'A JSON Navigator is used to return a response Dim jsNav As NotesJSONNavigator Call jsNav.appendElement(param_color, "color") 'Status 200 sets the response status to OK Print {status:200} ‘Return the Response to the rest client Print jsNav.stringify
45. #engageug Demo time #engageug
46. #engageug Extending your Applications Domino REST API (Keep)
47. #engageug Using Domino REST API Recent News News By Person News fi elds needed List Categories News By Categories ๏Collaboration Today API
48. #engageug Using Domino REST API ๏For Curators…
49. #engageug Using Domino REST API: Schemas Fields represented on the schema Formulas to de fi ne simple rules Modes to re fi ne func ti onal access
50. #engageug Using Domino REST API: Scopes The applica ti on creates app-id and app-secret to enable server-to-server access for a scope Scopes enables schema to be accessible and consumable
51. #engageug KeepConfig CT Domino Database Domino REST API App Flow Data 
 (Data Note) Design (Design Note) News (title, link, person…) Categories (text, order…) Persons (name, avatar…) Forms (News, Persons…) Views (ModeratedNews…) Agents (RetrieveAbstract) Scope: CT-Visitor Scope: CT-Curator Apps OAuth Curator Site News Site 3rd Party App Curator Schema JSON File Form:Mode (News:draft) Agent (RetrieveFeeds) View (NewsAll) Form:Mode (News:default) Form:Mode (person:default) Visitor Schema JSON File Form:Mode (News:dql) View (NewsModerated) View (Persons) View (categories)
52. #engageug Best Practices for REST API Development
53. #engageug Unexpected Login Page…
54. #engageug Best Practices – Unauthorized HTML login page with status 200 by default Return 401 (Unauthorised) status instead Notes.ini parameter: 
 DOMINO_FORCE401_WITH_HTML_LOGIN_PAGE=1 This is a server-wide setting!
55. #engageug Best Practices – Unauthorized ๏Create a form in domcfg.nsf ๏ Set content type to application/json ๏ Put JSON response content on the form { 
 "status": "401", 
 "error": "You are not authorized“ 
 }
56. #engageug Best Practices – Unauthorized ๏Add mapping in domcfg.nsf
57. #engageug CORS
58. #engageug Best Practises : CORS ๏Cross Origin Resource Sharing ๏ Get information from different domains ๏ Not allowed by default ๏ The web (data) server must allow requests from other domains
59. #engageug CORS explained
60. #engageug CORS explained
61. #engageug CORS : How to fix on Domino HTTP Server ๏Set DSAPI filter on Internet Site document (for api) ๏ ncorsext (Windows) or corsext (other platforms) ๏ OPTIONS must be set as Allowed Method
62. #engageug CORS : How to fix on Domino HTTP Server ๏ Create “cors-rules.json” file in “<domino data folder>/domino/cors” { "version": "1.0", "rules": [ { "resource": { "path": "/demo/cars", "startsWith": true }, "allowOrigins": ["http://localhost:1234"], "allowMethods": ["GET", "POST"], "allowCredentials": true } ] }
63. #engageug CORS : How to fix for Domino REST API ๏Domino REST API does not use the Domino HTTP server ๏ Update or create a file named “security.json” ๏ Add domains in the CORS section. ๏ Subdomains are supported automatically. { "CORS": { "localhost": false, "hcl.com": false, ".local": false, "acme.com": true } }
64. #engageug Recommended Tools
65. #engageug Tooling - Testing
66. #engageug Q&A
67. #engageug ๏ HCL Domino REST API Documentation: CORS 
 https://opensource.hcltechsw.com/Domino-rest-api/references/ usingdominorestapi/keepapplications.html?h=cors ๏ HCL Domino Documentation: CORS 
 https://help.hcltechsw.com/domino/12.0.2/admin/conf_cors.html ๏ Tom Van Aken: REST API in LotusScript Series 
 https://vanakentom.wordpress.com/rest-api-in-lotusscript/ ๏ Jesse Gallagher: Code-First REST APIs With XPages Jakarta EE Support 
 https://frostillic.us/blog/posts/2022/8/25/code-first-rest-apis-with-xpages-jakarta- ee-support ๏ Paul Withers: OpenNTF Webinar: Getting to Know Domino REST API 
 https://www.youtube.com/watch?v=Gwd9rnAsJFk ๏ Jesse Gallagher: OpenNTF Webinar - XPages Jakarta EE Support In Practice 
 https://www.youtube.com/watch?v=-v8-8oFHKxY Resources