© 2013 IBM Corporation BP207 Meet the Java Application Server You Already Own – IBM® Domino® Bruce Elgort | Elguji / OpenNTF / Clark College Serdar Basegmez | Developi / OpenNTF

2 © 2013 IBM Corporation Agenda Introduction to OSGi and DOTS Preparing Runtime and Development Environment Your First DOTS Tasklet Running and Debugging Tasklets Deploying Tasklets Tips, Tricks and Good Practices Q&A

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. 3

4 Cast Who are we?

Serdar Basegmez IBM Collaboration Solutions Champion (2011-2013) Owner of Developi Information Systems (Istanbul, Turkey) Founder and Co-leader of LUGTR – Turkish Lotus User Group Bilingual Blogger at LotusNotus.com (Turkish/English) OpenNTF Guy in Turkey – Contributor in XSnippets and CollaborationToday.info – Member Director at OpenNTF Board IBM Notes/Domino Design Partner Writer at The View Magazine Expertise on XPages, IBM Domino, IBM Connections, etc. 5

6 Elguji Software, Inc. OpenNTF Chairman Taking Notes Podcast @ belgort

7 Prelude Introduction

What’s all the fuss about DOTS about? XPages is great! – We all know it... But... – XPages are based on interaction. – Users demand, they reply back. What if… – Processing takes too long? – You need housekeeping? – You have scheduled reports? – You need to send reminders? – Your application needs batch processing? – You require unattended tasks? You have Agents, right? – We have a much better idea... • Much better idea… 8

Some Basics XPages ≈ Java IBM Domino – Java Application Server (almost) – JSF Container – OSGi platform Positioning Java on XPages – For classical Domino developers: • Formula SSJS • Lotusscript Java (Truth hurts!) – Performance – Reusability – Modularity – Separation of Powers (UI-Data-Logic) OSGi is the key technology! 9

What is OSGi? Open Services Gateway Initiative – OSGi™ Alliance (IBM involved) – Developed first in 1999 – Continues to be improved Module System and Service Platform – OSGi Bundles (Applications/components) – Bundles can be remotely... • Installed / Uninstalled • Started / Stopped • Updated – No reboot required for operation! 10

OSGi within IBM Notes®/Domino®... IBM Domino Server 8.5.2 – OSGi introduced in HTTP task – XPages runtime as OSGi plugin – Extension Points IBM Domino Server 8.5.3+ – OSGi support extended to; • XPages Extension Library, • REST APIs, • Social Enabler • DOTS (Domino OSGi Tasklet Service) • Domino Servlet Container • OpenSocial Container • ... 11 8.5.2 8.5.3 9+

Why OSGi matters Extending toolbox – Designer components – Server-side implementation for components Providing services for non-UI consumers – REST Services – Social APIs Adapting standards – OAUTH – JDBC Running Tasklets – DOTS 12

Meet DOTS – your new best friend Domino OSGi Tasklet Services: Next Generation Agents for Domino DOTS provides ability to develop and run OSGi level server Tasklets for Domino – Easy deployment. – Tooling with Eclipse IDE and its Java and plugins editors and Java debugger. – Leverage existing OSGi assets (within the company or open source). – Tasklets can be run: • Manually (local and remote calls) • Scheduled or on server start • Triggered (Hooks through data events) Not supported on Domino 9 SE Public Beta! OpenNTF project by IBM since April, 2011 (replaced JAVADDIN project) Included in IBM Domino 9 Social Edition Public Beta 13

What is a Tasklet? Component to process a step in your business logic – Lightweight addin task. – DOTS Tasklets are simply Java methods within OSGi bundles. – Declared as OSGi extension point Tasklet is functionally less capable than a Server Task 14 Tasks DOTS Tasklets Loaded once, run continuously Loaded once, run when needed Manages own threads DOTS manages threading Should be native (or encapsulated) Runs inside DOTS container Unmanaged security Managed security Hard to develop Easy to develop

DOTS Service Architecture 15 Source: Lotusphere 2012 – SHOW112 session by David Taieb, Paul Fiore, Elizabeth Sawyer

DOTS vs. Agents DOTS Tasklets outperforms over Java Agents! – Why? 16

DOTS vs. Agents DOTS Tasklets outperforms over Java Agents! 17 AMGR launches a Thread JVM Initializes Prepare Java and JNI interfaces Load Agent Bytecode Run! Run! Java Agent: DOTS tasklet: Everything is ready to run > tell amgr run "test\XPagescrash.nsf" 'LongJobAgent' 09.11.2012 19:38:39 JVM: Java Virtual Machine initialized. 09.11.2012 19:38:39 AMgr: Start executing agent 'LongJobAgent' in 'test\XPagescrash.nsf' 09.11.2012 19:38:39 Agent Manager: Agent printing: 181349 09.11.2012 19:41:02 Agent Manager: Agent printing: 2227 09.11.2012 19:41:02 Agent Manager: Agent printing: Finished in 143 secs... - 09.11.2012 19:41:02 AMgr: Agent 'LongJobAgent' in 'test\XPagescrash.nsf' completed execution > load dots > Listening for transport dt_socket at address: 8001 09.11.2012 19:42:40 Domino OSGi Tasklet Container started ( profile DOTS ) > 181349 > 2227 09.11.2012 19:43:22 [DOTS] (annotated) Finished in 41 secs...

DOTS vs. Agents Amgr is old and complicated – Security model problems – Limited functionality (like issues on class loading, reflections) Range – Agents are database-dependent – DOTS runs server-wide More flexible scheduling – Precise timings like 45 seconds – Programmatic control over scheduling DOTS provides wide option of database hooks 18 Not supported on Domino 9 SE Public Beta!

Examples on DOTS IBM SmartCloud™ for Social Business (a.k.a. LotusLive) – Some part of back-end processing has been implemented using DOTS CollaborationToday.info – Feed Aggregation is implemented using DOTS Other possible uses... – Workflow Engine implementations – Social Media Monitoring – Database Monitoring – Application Architecture Assessment 19

Episode 1 Preparation 20

Preparing Runtime Environment DOTS should be installed into IBM Domino server – IBM Domino 9.0 Social Edition Public Beta does not need installation! – For IBM Domino 8.5.2 or 8.5.3; • Download DOTS package from OpenNTF • You will also need some Eclipse plugins from Eclipse 3.6.2+ 21

Preparing Runtime Environment 22 Prepare directory structure under Domino folder – [Domino Program Files]\osgi-dots\rcp\eclipse\plugins – [Domino Program Files]\osgi-dots\shared\eclipse\plugins These plugins will be copied from Eclipse installation [Eclipse]\plugins

Preparing Runtime Environment 23 DOTS package contains executable files and Java libraries – Copy JAR files into folders you have created – Copy executable and libraries into Domino program folder Copy to [Domino]\osgi-dots Copy to [Domino]\osgi-dots\shared\eclipse\plugins

Preparing Development Environment DOTS Tasklets consist of methods in Java classes as OSGi bundles. Eclipse IDE is needed for development and debugging. – In theory, Domino Designer can be used with lots of unsupported configuration changes. Configuration is easy. – You will see steps for setting up Eclipse to develop OSGi plugins for Domino environment. • Eclipse (Development Environment) OSGi on Domino (Runtime) – Resulting Eclipse environment is useful to develop... • DOTS • XPages extensions • Custom validators, data sources, converters, etc. • Servlets • J2EE applications for Domino 24

Preparing Development Environment Download and extract Eclipse IDE... – http://www.eclipse.org/downloads – Eclipse 3.6.2 (Eclipse IDE for Java EE Developers) needed at least. Download and extract necessary OpenNTF project releases – Domino OSGi Tasklet Service : http://tiny.cc/DOTS – Domino Debug Plug-in from XPages SDK for Eclipse RCP: http://tiny.cc/XPagesSDK Prepare IBM Domino Server 8.5.2+ – DOTS should be installed properly. – A local IBM Domino Server is strongly recommended. – Use 32-bit for Eclipse and Domino Server for development/testing 25

Development Environment Basics Target Platform: – Simulate the runtime environment (e.g. JRE settings, classpaths, plug-ins...) Notes Java API: – DOTS Tasklets use Notes objects (e.g. Session). – Needed for development only. OSGi Framework – Comes with the Debug Plug-in. – Contains information about OSGi runtime (e.g. workspace)

Configuring Development Environment Launch Eclipse IDE, – Window\Open Perspective Plug-in Development

Configuring Development Environment Add Target Platform – Window\Preferences Plug-in Development Target Platform Add... 28

Configuring Development Environment Setup new Target Platform – Name it and add OSGi Directories from Domino Server – Mark it as Active 29

Configuring Development Environment Create Notes Java API project – File New Plug-in Project 30

Preparing Development Environment Import Notes.jar into the Notes API Project 31

Configuring Development Environment Configure Notes Java API project – Open Manifest file, add Notes.jar to the class path... 32

Configuring Development Environment Configure Notes Java API project – Add all packages to the «Exported Packages»... – Save the Manifest file (Ctrl+S or File Save) 33

Configuring Development Environment Install Domino Debug Plugin from XPages SDK for Eclipse – Help Install New Software... – Add Local Point to the update site folder extracted from OpenNTF Package – Accept warnings and restart Eclipse... 34

Configuring Development Environment That’s it... You are now ready to develop OSGi bundles on Domino! – To confirm what we have done... 35

You are Ready! You are ready to develop your first Tasklet... Here are some resources for you! – Lotusphere 2012 – Show 112 Session by David Taieb, Paul Fiore and Elizabeth Sawyer • Available on Greenhouse – Lotusphere Community – YouTube: OSGi Tasklet Service for IBM Lotus Domino by Niklas Heidloff • http://youtu.be/CRuGeKkddVI – NotesIn9 Episode #93 – «Introduction to DOTS» by Serdar Basegmez • http://notesin9.com/index.php/2012/12/04/notesin9-093-introduction-to-dots/ – XPages Portable Command Guide by Martin Donnelly, Maire Kehoe, Tony McGuckin, Dan O'Connor • Chapter #3: Working with the Console 36

Episode 2 Develop your First Tasklet! 37

Tasklet Development Basics A Tasklet project is a Plug-in Project contains a number of Java classes – Same project may contain lots of Tasklets. Extension Points – Extension point to be defined for «com.ibm.dots.task» Imported Packages – Notes Java API to be linked for compile-time. OSGi Runtime Configuration – OSGi on Domino will start bundles from its own repository. – A link between Eclipse workspace and OSGi Framework is needed. • Eclipse will tell DOTS that «I have bundles to install!» Debug Configuration – Eclipse needs to know how it will connect to the target JVM for debug 38

Developing your first Tasklet Create a new Plug-in Project – File New Plug-in Project 39

Developing your first Tasklet Import DOTS extensions – Open Manifest file and click extensions 40

Developing your first Tasklet Import DOTS extensions – Deselect «Show only extension points from the required plug-ins» – Find & pick «com.ibm.dots.task» extension from the list – Correct the warning in the Manifest.mf tab, then Save & Close Manifest view. 41

Developing your first Tasklet Import «lotus.domino» package into your project 42 S

Developing your first Tasklet Create a Java Class for your Tasklet 43

Developing your first Tasklet Welcome to your first DOTS Tasklet! 44 Runs when your tasklet unloaded Runs when your tasklet runs

Developing your first Tasklet Now add some code 45

Developing your first Tasklet We will define how it will run! – Open Plugin.xml (Plug-in Manifest) from the Package Explorer... – These markups declare your Tasklet 46 Name of your class An id for your tasklet

Running your first Tasklet So far, we have created a Java class, written some code and Eclipse compiled. DOTS task running on server will load and run our tasklet, but how? – Normal way, we should deploy the plugin into server. – For development/testing, we will tell DOTS to load my bundle directly from Eclipse PDE. There are two ways: – Run only, – Run and Debug 47

Running your first Tasklet Create a «Run Configuration» 48

Running your first Tasklet Domino OSGi PDE Configuration should be made for the first time Eclipse created a special file in DOTS workspace – So DOTS task will load bundles from our run configuration... 49

Running your first Tasklet Now, you can launch DOTS task in the server – Also check for tasklist to see that our Tasklet has been loaded. – ...and run! 50

Debugging your first Tasklet For debugging, Eclipse has to connect to DOTS task. – Eclipse will know what runs on DOTS More settings needed now... – Close DOTS task • «tell dots quit» – Set Notes.ini parameters for debugging DOTS: • «set config DOTS_DEBUGADDRESS=8001» • «set config DOTS_DEBUGSUSPEND=y» – Reload DOTS task • «load dots» • DOTS task will not start until Debugger connects. 51 Security Alert! These settings create a security breach and should be used with caution...

Debugging your first Tasklet Server is listening. Now, Eclipse has to know where to connect – Create a new Debug Configuration – Use the port we have set before! 52

Debugging your first Tasklet You can now set a breakpoint and start debugging. – Eclipse will switch to debug perspective as DOTS reach your breakpoint. – Your changes will be applied immediately! 53

A couple of tips on Debugging & Testing... Suspending DOTS... – «set config DOTS_DEBUGSUSPEND=y» – This setting will prevent DOTS task starting until a debugger connects. – Suspending DOTS task would be useful to debug scheduled Tasklets or those which starts on server start. But if your debugger cannot connect for some reason, server will be unstable! You can also enable debugging for DOTS temporarily... – load dots -debugaddress= -debugsuspend= Eclipse utilizes ‘hot-replace’ feature of JVM – In Debug mode, we can change the source code and bundle will be refreshed upon compiling! 54

A couple of tips on Debugging & Testing... Refresh bundles... – Without debug, there will be no «hot-replace». • Meaning: Recompiling classes will have no effect until DOTS task restarted. – OSGi allows you to refresh bundles without restart, here is how: • Find out the «bundle-id» for the plug-in • Refresh the bundle – Doesn’t work for scheduled tasklets. 55

Scheduling Tasklets DOTS looks for extension point configuration in «plugin.xml» to find out how to run tasklets. 56 Tasket id Add this!

Magic Word: Annotation So far, we have declared our tasklets with an XML file. – A Java class «doRun» method A Tasklet DOTS provides annotated declarations within Java classes – Remember, you still need to create an extension point in «plugin.xml» 57 @RunOnStart public void runOnStart( IProgressMonitor monitor ){ logMessage("Annotated onStart method"); } @Run( id="manual") public void runManual( String[] args, IProgressMonitor monitor ){ logMessage("Annotated run method with id=manual"); } @RunOnStart @RunEvery( every=60, unit=RunUnit.second ) @HungPossibleAfter( timeInMinutes=1 ) public void runEvery60seconds( IProgressMonitor monitor ){ logMessage( "Called from annotated method every 60 seconds"); }

Episode 3 Tasklet Going Production! 58

Deploying Tasklets - Basics What we develop in Eclipse IDE is a «Plug-in» Plug-ins can be deployed into OSGi by... – Referencing PDE tool to the Eclipse Run Configuration • For testing & development... We have seen this... – Putting JAR file into OSGi plug-ins folder, • Needs file-level connection to the server and (sometimes) begging to your Admin! – Referencing an update site • DOTS provides profiles! Before deploying Tasklets, make sure: – Delete PDE runtime configuration (pde.launch.ini) – Disable suspend feature in debugging (DOTS_DEBUGSUSPEND) 59

Export Plug-in to a JAR file – Right Click Export Deployable plug-ins and fragments... Deploying Tasklets as Plug-in 60

Select your plug-in project and a destination Place exported JAR file into plugins folder, then restart DOTS task – [Domino Data Folder]\domino\workspace-dots\applications\eclipse\plugins Deploying Tasklets as Plug-in 61

DOTS Profiles A DOTS profile is a specific DOTS container. – Each profile defines «Tasklet Container» that has different settings for; • Remote Controller • Security Configuration • Specific update sites/folders to load plug-ins DOTS support multiple profiles for different needs. Profiles can be stored in XML files or Notes Databases – It can be any Notes Database like names.nsf. Every profile has a unique name (Message Queue Name – mqName) – Profiles accept separate tell commands, like «tell profileName run XYZ» – DOTS loaded with a default profile named as «DOTS» 62

Creating Profile Repository as NSF As a best practice, store DOTS Profiles in a Notes database – Create an empty database and secure it with ACL. On Server, configure DOTS to use Profiles database – tell dots quit – set config OSGI_CONFIGURATION_DB=dotsprofiles.nsf – load dots – tell dots profileCreate dots DOTS will create necessary design elements By using DOTS profiles, we can import Tasklets from NSF-based update sites... 63

Update Sites and Features Update Site is used to organize and export features. – So they can be installed into eclipse-based platforms. Feature is a collection of plug-ins. – Made of plug-ins that can be installed/updated together. Update site, therefore is a directory structure. Plug-ins can be deployed via: – File system – Archive file – HTTP/HTTPs – Virtual file system • e.g. Domino OSGi creates VFS for NSF databases 64

Deploying Tasklets as Update Site Create a Feature project in Eclipse 65

Deploying Tasklets as Update Site Create an Update Site project in Eclipse 66

Deploying Tasklets as Update Site Configure the Update Site project 67

Creating Update Site Database Create an empty database from Eclipse Update Site template. Import update site we just created. Confirm our feature and plug-in has been imported into database. 68

Configure Profile to use our Update Site Open OSGi profile we have created. – Add update site database into Sites section. – You may also select Features you want to include. Restart DOTS task and check for tasklist. When you need to update your plug-in, load a new version to the update site database and refresh the bundle. 69

Security with Profiles You might define additional Java security to the profile. One reminder: – File-based update sites or plug-ins have unlimited security by default. – NSF-based update sites have a default set of permissions. Check DOTS documentation! 70

Multiple Profiles You may create more than one profiles. – For testing – Tasklets that you use occasionally – Tasklets with different security needs – Heavily loaded Tasklets You can create a second profile from the server console. Second profile should be loaded separately. 71

Episode 4 Tips, Tricks and Good Practices! 72

Tips and Tricks on Programmability Logging – Try not to use «System.out.println». (No output to log.nsf) – Instead, use «logMessage(...)» and «logException(...)» runWhen argument helps – If you overload doRun method (run the same class for manual and scheduled operations at the same time), runWhen will help you to identify which. Parameterization – Manual tasks can get parameters through Console or Plug-in manifest (plugin.xml) • «tell dots run XYZ param1 param2». • «args[n]», «getNthArgument(n)» or «getKeyedArgument(key)» can be used. – For other tasks, you have multiple options. • «notes.ini» settings • Argument Resolver (refer to DOTS samples) • Profile parameterization extension via DXL (refer to DOTS readme.pdf) 73

Tips and Tricks on Programmability Profile Configuration Document: 74

Tips and Tricks on Programmability Life-cycle for a scheduled Tasklet – When a tasklet runs on schedule, DOTS class loader will create an instance from your tasklet class. This instance will not be disposed until DOTS process stops. – This is a great advantage! 75

Tips and Tricks on Programmability Multiple run-mode for the same Tasklet and different Java Instances – When building multi-purpose tasklets, • Accomplish tasks with more than one functions via schedules & manual calls – Each run will create a new Java object independent from each other • Scheduled calls persistent objects • Manual calls will be unloaded from memory afterwards An example: Feed Reader for CollaborationToday.info – A managed queue for feeds to be consumed – Lots of tasklets • Scheduled tasklet to read the next feed for new stories every 2 minutes • Scheduled tasklet to check if there is a new feed definition every 60 minutes • Manual tasklet to force refresh queue (when we have added a new feed) • Manual tasklet to force one or more feeds to be refreshed immediately – All can be defined in a single class but will run on different Java objects!!! 76

Tips and Tricks on Programmability Instantiation Problem – You can employ singleton pattern to create a global object within bundle. • Activator class can be used to initialize your objects... • Be careful about synchronization. • Domino-based objects will be recycled after each run. Don’t persist those objects! – In Feed Reader example; • We can design a bundle-level Queue manager, • Initialize it on bundle start (and even save it when bundle stops). 77

Tips and Tricks on Programmability Feed Reader – Singleton Approach – Still, careful about lifecycles… NotesSession should be used carefully! 78 Bundle Bundle.start Initialize the Queue Bundle.stop Save the Queue QueueManager (singleton object) readNextFeed() refreshQueue() readFeed(id) Tasklet Class @RunEvery( every=2, unit=RunUnit.minute ) Sched1() qm.readNextFeed() @RunEvery( every=60, unit=RunUnit.minute ) Sched2() qm.refreshQueue() @Run( id="refreshQueue" ) Manual1(args) qm.refreshQueue() @Run( id="refreshFeed" ) Manual21(args) qm.readFeed(args[0])

Tips and Tricks on Programmability Remote Controller – DOTS might accept remote commands from other applications (e.g. XPages apps) – Remote commands are accepted from localhost only. – Listener should be configured from Profile document. 79 Source: DOTS Readme File

Tips and Tricks on Programmability Progress Monitoring – «IProgressMonitor monitor» argument passed on tasklet methods. – Very useful for tasklets running long time. – Progress monitor can be used to... • Inform DOTS task manager about how much tasklet has completed. • Aware of situation that might break execution. 80 Vector> views = db.getViews(); monitor.beginTask( "List the views", views.size() ); Init with # of steps for ( Object view : views ){ if ( monitor.isCanceled() ){ break; Break if canceled. } logMessage("\t" + ((View)view).getName() ); try { Thread.sleep( 1000 ); } catch (InterruptedException e) { e.printStackTrace(); } monitor.worked( 1 ); One step completed }

Problems you may encounter using DOTS OSGi Context – DOTS runs in a different OSGi level than HTTP. – So there is almost no connection between DOTS and XPages • You can start DOTS tasks from XPages, using remote controller socket • Refer to Readme documentation for configuration and a basic example – Some possibilities • Communicate over Notes Document • Using REST API to send/receive information between DOTS and XPages Extension Manager events – Problems with queuing – This feature has been removed in IBM Domino SE 9.0 Public Beta Class Loading issues – Some Open Source APIs might have issues with OSGi class loading (e.g. Rome project) 81

Q&A 82

Thank you! 83 Twitter: @belgort Blog: http://bruceelgort.com/ Twitter: @sbasegmez Blog: http://lotusnotus.com

84 © 2013 IBM Corporation Legal disclaimer © IBM Corporation 2013. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.