Engage 2018: A Domino R5 Application Upgrade Project: What Could Possibly Go Wrong?

Engage 2018 session with Graham Acres (https://twitter.com/gacres99):

A Domino R5 Application Upgrade Project: What Could Possibly Go Wrong?

Are there old Domino R5 applications out there still? Yes, there are! And of course they are fully supported by the original developer, are fully documented, and someone remembers the password for the cert.id, right? No? Oh well, this is Domino! No Problem!

Join us as we relate our experience of doing just such a project, our successes and the things we learned the hard way, including all the things we knew about R5 but had long forgotten. We took a very comprehensive old Domino R5 application and turned it into a modern XPages application with a full Java back end. If you are planning any type of Domino application upgrade or modernization project, you will come away with a long list of things to keep in mind as you make your plans.

Speakers:
Graham Acres , Brytek Systems Inc.
Serdar Basegmez , Developi Information Systems

Transcript

1. #engageug Dev04 A Domino R5 Applica.on Upgrade Project: What Could Possibly Go Wrong? Serdar Basegmez, Developi Informa.on Systems Graham Acres, Brytek Systems Inc.
2. #engageug Serdar Basegmez • IBM Champion (2011 - 2018) • Developi InformaCon Systems, Istanbul • Notes/Domino/XPages Developer, half-blooded admin! • Member Director at OpenNTF Board • LUGTR, LotusNotus.com • EngageUG, IBM Connect, ICON UK, NotesIn9… • Blogger and Podcaster on ScienCfic SkepCcism serdar_basegmez 2
3. #engageug • IBM Champion (2017, 2018) • Brytek Systems Inc., Vancouver • Notes Developer since 1992 (v2.1) • Core Team Member, Cross Canada CollaboraCon User Group • Lotusphere, Connect, C3UG, NotesIn9, LUGs • Cyclist, Minor Hockey Coach, I built a greenhouse! Graham Acres gacres99 3
4. #engageug Disclaimer! • Many of these emoCons were experienced during this project • We will convey as many of them as possible during this presentaCon • Very few of them may be afributed to Notes and Domino, but to a nameless developer lost to the sands of Cme… 4
5. #engageug Our Story • Let’s Introduce the App • Convincing the Customer to Upgrade • The Plan • What Actually Happened • Facepalms, Lots of Facepalms! • Lessons Learned 5
6. #engageug Introducing the ApplicaCon http://www.sandsmuseum.com/cars/elise/experience/maintenance/frontclam/clamremoval.html 6
7. #engageug The App • Yes, there are sCll R5 apps out there • NDA • Built in 1999 • Register / Search for Financial Instruments • Notes for business logic and administraCve processing • Web front end for clients • 8 templates, 10 NSFs, plus numerous archives • IntegraCon with SQL Server, RightFax, proprietary scanning somware • PDF GeneraCon using Adobe Acrobat 4! 7
8. #engageug Convincing the Customer to Upgrade • Introduced in 2010 to adjust tax calculaCons • Limited exposure amer that first project • Talked for 3 years about an upgrade • RightFax upgrade no longer supported R5 • Server and workstaCon OS upgrades • Replace or Upgrade? • Budget to replace was 3x more than upgrade budget • Actually agreed to start in 2016 • Started in 2017 • Should be upgraded in 2018 8
9. #engageug What is the Plan?
10. #engageug The Plan • Build your team and your toolbox • Team needs to learn the app • Build a new 9.0.1 server • Migrate all templates • Test Notes funcConality for defects • Test data for encrypted fields (cert.id password) • Rewrite of web app • Rewrite of PDF module • Keep everything else the same 10
11. #engageug The Toolbox… • 3 CiCes, 4 ConCnents, 3 Timezones • CollaboraCon is tough. • Slack / Skype / Trello / Box • IBM Watson Workspace • Hammer… 11
12. #engageug Sharing Code… • Bitbucket for the Repositories • Source Control issues… • Designer is very bad for source control… • First, tried a single repo, but failed a couple of Cmes and lost some work. • We switched to our own repos, Slack helped to keep in sync • Good, because all commits have been audited by other • Bad, because it’s Cme-consuming. 12
13. #engageug Learn The App • No documentaCon • No comments, anywhere • All filenames had a space in them • No conCnuity from dev to dev • Massive LotusScript libraries (a lot of “copy of”s) 13
14. #engageug From the Current Help For optimal access to the System, the minimum computer system requirements are as follows: • Pentium-class PC computer, 32 mb of RAM • Windows 95 or Windows NT operating system • Modem (internal or external) with a minimum speed of 33.6 kbps, and connection to an uninterrupted phone line • Laser printer, letter-size tray • Internet Connection Software • Web browser software - Netscape Navigator 3.0+ has been tested and proven to be compatible 14
15. #engageug Learn The Server • “What was the password of Cert.id file?” • Time zone change • OS upgrades • Security in the extreme 15
16. #engageug Back to The App • Forms • ValidaCon steps in web process meant separate Forms, controlled by WebQuerySave agents: massive useless forms need to be purged • Different business processes, but fields that hold the same data, with different field names (we will come back to this!) • Navigators! • Pages • No Framesets 16
17. #engageug Write Your Agents Step by Step 17
18. #engageug The Scanner • A separate, really old workstaCon afached to a scanner • Proprietary scanner somware • A small app with a LotusScript agent to import the scanned files and metadata • No working floppy drive, No working network drives • New scanner and workstaCon meant analysis of the LotusScript, but how to get at it? 18
19. #engageug GeneraCng PDF Files in 1999 • An agent to create PDF… • Used every worst pracCce one can possibly think of. • Too many assumpCons. • Too many dependencies on external libraries. • Dependency to some files and folders that should exist. • Of course, NOTHING DOCUMENTED! • Rewrite needed because of the Acrobat version… 19
20. #engageug Our Painful Experience… 20
21. #engageug What Actually Happened • Remember our plan: • Move templates to 9.0.1 server and test • Rewrite the Web app in XPages • Rewrite the PDF module • Test the data in a new Notes domain (cert.id password) 21
22. #engageug The Notes Side of Things • Zero issues! • All code (including the warts) ran fine • No password on the Cert.id • No encrypted data, so just migrate it 22
23. #engageug The Web ApplicaCon Rewrite • The old applicaCon was fantasCc :) • 2-4 Web Query Agents • 3-4 Web Form Designs • Script Libraries • Spitng out temporary documents everywhere! 23 Editable Form Read Only Form A Silly Question Thanks! Temporary Document ⚙ Web Query Save ⚙ Web Query Save ⚙ Web Query Save ⚙ Web Query Save ⚙ Web Query Save Temporary Document (For Each Process!!!)
24. #engageug All That Code • ConverCng mission-criCcal parts to Java… • LS code, formulas, logic, etc. • MulCple copies of the same code? • A task coded in an agent, separate version for PDF. • A duplicate, slightly modified for some rare cases. • Another, modified in a different database… • All in use, for the same task, 4 different ways! 24
25. #engageug XPagifying… • Development Speed is the Gold! • We’ll have lots of garbage to deal with! • A General UI/UX Framework based on Bootstrap 3 • Java Beans and Caching Beans for the back-end • Minimal use of SSJS (to enforce the use of Java) 25
26. #engageug CreaCng a PDF: How hard it can be? • We preferred Java with Apache PDFBox • Open source, compaCble, easy to use, familiar to us. • But not hassle-free! • Needed to use Java agent because of the way they use. So maintained some duplicate code. • We couldn’t use the latest version because of an unidenCfied issue with resources. • Developed a simple layout helper. Took some Cme but it paid back amerwards. 26
27. #engageug 27
28. #engageug Facepalms! • Found a computed field, just for a formafed date Cme to be used in an agent. 28
29. #engageug Facepalms! • Taking a TIFF, insert it into a new PDF, so you can add it to another PDF… 29
30. #engageug Facepalms! • Use [Back] in Netscape to create a new transacCon 30
31. #engageug Facepalms! • No remote access to the client’s server environments. Onsite for everything 31
32. #engageug Technotes, Belated… • IdenCfied a bug introduced in R6 of Notes where an @Formula agent to clear the contents of a Private on first use Folder results in an error that appears to have never been fixed. • IdenCfied a memory leak in the PDF Agent. JAR files afached into Java agents leak the memory and they crash the server amer a while. 32
33. #engageug Two Weeks Ago… • The ProducCon R5 server ran out of storage space 33
34. #engageug Lessons Learned… 34
35. #engageug Lessons Learned • Tradeoff of adapCng the old design vs creaCng a new design • New design means migraCon of exisCng data • Results in a cost of Cme, tesCng, funcConality • Would introduce heavy work tesCng on the Notes side • Keeping the old design means keeping the old warts and working with/around them • UlCmately this took longer than originally planned 35
36. #engageug Lessons Learned • Resourcing / availability • Project management, meeCngs, calls • Dev team 11 hr Cme difference! • Missing deadlines • Be prepared to work with people who aren’t fully knowledgeable on the products they are responsible for that integrate with Domino • Security, and firewall requirements • Scope: Who is responsible for install/config of servers and clients? 36
37. #engageug Lessons Learned • Analysis step is full of surprises… • Do not learn the applicaCon from the code! • What the applicaCon does is not the problem. • Ask users how they do things! • What bufons they use, which fields they fill in, etc. • Try to determine what design elements are in use and what were no longer used • Spare Cme for the analysis up front, or pay for it later 37
38. #engageug Lessons Learned • Notes is solid! 38
39. #engageug QuesCons Thank you!! @serdar_basegmez @gacres99 39