Introduction to OSGi

In this presentation, we'll briefly introduce the fundamentals of OSGi - a dynamic module system for the Java Virtual Machine.

We'll also introduce an example project, hosted on GitHub, showing how to employ Gradle to create an OSGi-powered JavaFX application written in the Scala language.

Transcript

1. Gianluca Costa Introduction to OSGi http://gianlucacosta.info/
2. Introduction • Elegance always matters, especially when creating software – and modularity lies at its very heart • In this presentation, we'll briefly introduce the fundamentals of OSGi - a dynamic module system for the Java Virtual Machine • The author would like to thank: – Professor Paolo Bellavista, for his kind and valuable advice and suggestions – Neil Bartlett, author of the beautiful, free, Creative Commons-licensed book OSGi in practice
3. Outline • Why the traditional Java class loading mechanism is far from being perfect • Overview of the module system provided by OSGi • Details of OSGi's class loading • Creating OSGi bundles • First steps in Apache Felix • Example multi-module application with Scala + JavaFX + OSGi + Gradle
4. Section 1 Why OSGi?
5. Modularity • Modularity is paramount for developing complex systems, as it fosters: – Separation of concerns – Parallel development – Design simplicity – Code reuse – Ease of maintenance and repair
6. What is a module? • The main defining attributes of a module are: – Self-contained: from the outside, a module must appear as a logical whole – Highly cohesive: a module should have just one, well-defined purpose, and achieve it – Loosely coupled: a module should know nothing about the implementation of the modules it depends on – it must only rely on their public interface • The JVM employs JAR files - archives containing bytecode and resources. Are JARs modules?
7. JAR hell • JAR files are not modules! For several reasons: – JARs do not exist at runtime: when searching for a class, the JVM performs a linear scan of the CLASSPATH, starting from its left-most item → this is a very inefficient and unreliable process – in particular, a class might load not a companion class, but a totally unrelated - yet preceding - class in the CLASSPATH – JARs, by default, contain no explicit declaration about dependencies (except a file-system based, very specific, Class-Path declaration)
8. JAR hell (2) – JARs are not versioned – it's the developer that must find a way to track versions – for example, stamping them in every JAR file name – There is no information hiding between JARs: once a class is declared public, it can be accessed by all the other classes in its own JAR, but also by any other class in the system – this can lead to potential side-effects • So, apart from the fact that they keep files together, for example simplifying Internet downloads, JAR files are not modules – at all
9. What about Gradle, Maven, Ivy...? • Build systems are brilliant – they automate and extremely simplify the build process, hiding an immense deal of complexity – e.g., they introduce artifact versioning, track down transitive dependencies and perform file-based JAR versioning • Alas, such versioning is static, a contract respected by the tools only: the JVM keeps working with its linear CLASSPATH, making JAR files lose their boundaries • Anyway, build systems are precious compile-time complementary tools for OSGi
10. JVM – Class loading • The JVM loads classes by means of class loaders, instances of java.lang.ClassLoader subclasses • ClassLoader has 2 duties: 1)Find the bytes of a class – which could reside anywhere 2)Turn them into a Class • We can only override methods to customize task (1) • Task (2) is performed by defineClass(), a native final method – so we are unable to change it
11. Class loaders – The delegation tree • Class loaders are organized in a hierarchical structure: each class loader has a parent, except the native class loader, which is the root of the tree • A class loader first asks its parent to load a class – and tries to locate such class only if the parent fails (parent-first delegation) • This is due to security reasons – the loading of core Java classes is enforced to be carried out by the native class loader, and not by a possibly malicious one
12. Java SE – Typical class loader tree Native class loader Extension class loader Application class loader Java core classes (rt.jar) Extension JARs (in $JAVA_HOME/lib/ext) These libraries are available to all apps App-specific JARs
13. Java EE traditional class loader tree Native class loader Extension class loader Application class loader Java core classes (rt.jar) Extension JARs (in $JAVA_HOME/lib/ext) Server-level shared JARs EAR 1 class loader WAR 1 c.l. EJB 1 c.l. EAR 2 class loader WAR 2 c.l. EJB 2 c.l. (...)
14. JAR hell: unpredictable, unreliable • Not only might a class happen to reference an unknown class contained in another JAR file, in lieu of the expected companion... • ...but such misbehavior might occur at any moment, perhaps just under certain conditions • What's more, the resulting errors are usually not trivial (LinkageError, ClassNotFoundException, NoClassDefFoundError) • Finally, different compile-time versions of a JAR just can't coexist at runtime
15. OSGi • OSGi is “The dynamic module system for Java™” • “The OSGi Alliance is a worldwide consortium of technology innovators that advances a proven and mature process to create open specifications that enable the modular assembly of software built with Java technology. Modularity reduces software complexity; OSGi is the best model to modularize Java.” - https://www.osgi.org/
16. OSGi bundles • Bundle = a module in OSGi. Bundles are plain JAR files with just additional metadata, declared in the standard META-INF/MANIFEST.MF file • Therefore, OSGi is fully backward-compatible • Bundle metadata: – Symbolic name (required) – Version (optional, but advised) – Description – List of imported and exported packages – …
17. Immediate benefits of OSGi • Each bundle has its own class loader • Dependencies are explicit, by means of import/export declarations for packages • Bundle encapsulation: only exported packages are visible to other bundles • Multiple versions of the same bundle – and even of the same package – can coexist at once within an application • OSGi is dynamic: bundles can be installed, updated and uninstalled without restarting applications
18. OSGi: from trees to graphs • Traditional Java class loaders compose a tree – with Java EE apps structured like self-contained silos – involving scalability issues, in addition to all the drawbacks imposed by a CLASSPATH within the boundaries of each app • The OSGi framework reads the declarative import/export package statements and dynamically connects (wires) bundles – bypassing the inefficient, non-scalable global sequential scan • It's the same as trying to retrieve an element by indexing a hash map in lieu of scanning a huge list
19. The OSGi graph – Further advantages • There is no more a node-subnode relationship between class loaders – only a network of peers, that can be providers of some packages and consumers of other packages • In case of missing dependencies, an error occurs immediately, with detailed explanations • In the graph, dependency links express the consumer/provider relationship for single packages: that is, bundle A can depend on bundle B for package P1, and depend on bundle C for package P2
20. Part 2 Bundles in detail
21. Fine-grained versioning • Bundles are versioned, just like Gradle or Maven artifacts... • ...but – which is most important - packages are versioned as well! • A package version is arbitrary, provided that it's well-formed • In OSGi, dependencies are mainly expressed in terms of packages – any bundle could provide a package required by another bundle
22. Version syntax • A version string is subject to the following syntax: MAJOR[.MINOR[.MICRO[.QUALIFIER]]] • MAJOR, MINOR and MICRO must be numeric, and they are sorted accordingly, as expected • QUALIFIER is a string: whenever the other components are all equal, QUALIFIER is checked via string comparison. There are a few qualifier patterns: – Development phase: alphaXX/betaXX/final – Timestamp: YYYY-MM-DD_HHMM => scales better
23. Import / export – Headers & Versions • An exported version must be exact or missing (defaulting to: 0.0.0) → it is like a point in the space • An imported version must be a range or missing (defaulting to: any version) → it is like a segment • This design greatly increases flexibility, as a module does not necessarily require recompilation whenever a new version of an imported package is available - provided that only the implementation has changed, not the shared contract
24. Version ranges • Version ranges are expressed as mathematical ranges: – [V1,V2] → from V1 to V2, both included – (V1, V2) → from V1 to V2, both excluded – [V1, V2) → from V1 to V2, only V1 included – (V1, V2] → from V1 to V2, only V2 included • Special cases: – V1 (where a range is expected) → V1 or any later – [V1,V1] → V1 only
25. Bundle encapsulation • All the packages in a bundle are private by default, to minimize namespace pollution and to foster encapsulation • Bundles must keep implementation as hidden as possible, only exposing interface information
26. OSGi – Class loading in detail Bundle A Bundle class loader Parent class loader Bundle B Bundle C (1) java.* and boot delegation (2) Imported packages (3) Required bundles Internal types (4) Internal classes Fragment X (5) Fragments
27. OSGi – The application class loader Every resolved bundle has its own class loader, not relying on the application class loader - which is still employed by OSGi in 2 cases: 1)Only the native class loader – at the top of the standard class loader tree - can load java.* packages – otherwise, a SecurityException would be thrown 2)A few classes might assume that parent delegation is in use – to avoid breaking compatibility, OSGi defines a bootdelegation parameter – packages that will be loaded just by the parent class loader. It should be used for very particular requirements
28. The system bundle • The parent class loader transparently loads java.* and bootdelegation classes, as usual • However, the JRE and its CLASSPATH include several packages – for example, javax.swing.* - which are not automatically loaded: in OSGi, they must be explicitly imported – and it is the system bundle that exports them • System bundle = meta-bundle encapsulating the OSGi framework at runtime; its id is always 0 • The packages exported by the system bundle are not standard, but they can be enforced via properties: org.osgi.framework.system.packages[.extra]
29. OSGi – Loading packages • Class loading is delegated, whenever possible, to the class loaders of other bundles, transitively, following the dependency trail (called wiring) • Imported packages prevail over local packages, for: – Optimization: a class is globally loaded just once – and not once per bundle – Coherence: because class identity = <class FQN> + <class loader>, having a versioned FQN loaded by just one class loader prevents cases of ClassCastException • A bundle can even export its imported packages
30. Version resolution • Different versions of the same package might be available – but a bundle can export at most one • Whenever an import requested by a bundle – via the Import-Package header - must be resolved: 1)If one matching version - that is, one version belonging to the range of the import declaration – is available, it is automatically wired 2)If two or more matching versions are available, the highest of them is chosen 3)Otherwise, it's the bundle with the lowest id that provides the package
31. Required bundles • Via the Require-Bundle header, a bundle can directly depend on another bundle, not on packages • Depending on bundles is definitely discouraged: – The efficient granularity at the heart of OSGi is lost – The architecture becomes fragile, as splitting a bundle could cause unresolved runtime missing dependencies, not eagerly detected by the OSGi framework • If a package cannot be found via import/export wiring, such package is looked for in the Export- Package header of the required bundles
32. Internal types • A bundle is a regular JAR file, so it includes types and resources • Packages can be private or exported – but they can all be used by the bundle itself, of course • Packages are private by default, unless they are listed in the Export-Package header – NOTE: always check your tools, as they might actually export packages by default – Furthermore, tools might create a Private-Package pseudo-header, ignored by the OSGi framework
33. Fragments • Fragment = set of types and resources that cannot exist on its own – it needs to be attached to a bundle, providing additional types and resources. • Fragments are especially valuable in a few common situations: – Multi-platform support: usually, fragments integrate a cross-platform bundle by adding a few platform- specific implementations – Resource bundles: to provide small customer-specific variations – for example, logos, icons, or localization
34. Bundle dependencies • Dependency = Set of assumptions made by a software component about its context • Bundles can require 3 types of dependencies: 1)Execution environment specification 2)Package imports 3)Bundle requirements • OSGi even supports simultaneous bundle resolution, thus enabling circular dependencies
35. Environment specification • Bundle-RequiredExtensionEnvironment header • Describes the expected Runtime Enviroment, for example JavaSE-1.8. • JRE classes are extended, version after version, with new methods and properties – which, if called in the context of an older JRE, would cause an unpredictable NoSuchMethodError • The solution is twofold: 1)Compile your code using exactly the target JDK 2)Enforce the requested JRE with the OSGi header
36. Package imports • The header for declaring required packages is: Import-Package: <package>[;version=<range>] [,<package>[;version=<range>]][,....] • version is optional and, if omitted, means “any version” • Employing a punctual version means “that version or later”
37. Bundle requirements • Requiring a bundle is really discouraged • Anyway, the header syntax is: Require-Bundle: <bundle symbolic name>[;bundle-version=<version>][,<bundle symbolic name>[;bundle-version=<version>]][,...] • Versioning works like package versioning • Bundle-Version is the related header, which has an important side-effect: it enables multiple, different versions of a bundle to be loaded at the same time
38. Bundle lifecycle INSTALLED RESOLVED STARTING ACTIVE STOPPING UNINSTALLED Install Resolve/ Install Start Stop Refresh/Update Refresh/ Update Uninstall Uninstall
39. Bundle lifecycle - States • INSTALLED: the bundle has been loaded, but nothing is known about its dependencies • RESOLVED: the 3 kinds of dependencies have been resolved and the bundle is ready to run • ACTIVE: the bundle is exporting packages, and its services are running • UNINSTALLED: the bundle has been removed from the environment • STARTING, STOPPING: temporary states, where the bundle activator is running
40. Further details on the lifecycle • A bundle can move from INSTALLED to RESOLVED if its dependencies are RESOLVED or they can be transitively resolved at that moment • Calling Start on an INSTALLED bundle moves it to RESOLVED, then to STARTING, and finally to STARTED - if dependencies are satisfied, of course • Update loads a new version of a bundle • Refresh re-wires bundle dependencies after one or more bundles have been updated • The state of a bundle can also be affected by the state of bundles on which it depends
41. BundleActivator • Every bundle can have a bundle activator – a class implementing the BundleActivator interface and registered with the following manifest header: – Bundle-Activator: <Activator FQN> • Its start() and stop() methods are called in the STARTING and STOPPING phases – so, they should be as fast as possible; however, for long- running tasks, an activator can spawn threads • The bundle activator can reside in a private package • A new activator instance is created for every STARTING / STOPPING pair
42. BundleContext • BundleContext is a handle to access the framework • It includes methods for programmatically inspecting and manipulating the current bundles: – Read metadata – Change lifecycle state – Install new bundles – … • A common way of obtaining a BundleContext is via BundleActivator's start(BundleContext context) method
43. BundleListener • A bundle listener is called on lifecycle events triggered by any bundle • It must implement the BundleListener interface • It can be registered an unregistered using BundleContext's methods: – addBundleListener(...) – removeBundleListener(...)
44. OSGi = Modularity + Services • Up to now, OSGi might appear as a brilliant, package-level runtime extension of the versioning ideas introduced by build systems • Actually, OSGi is much more - it provides a full- fledged lifecycle management for services and a rich API to publish, locate and consume them • We won't deal with services in this presentation, but most books, as well as the reference documentation, describe them in great detail
45. Section 3 Creating OSGi bundles
46. Bundle = JAR + OSGi metadata • To create a bundle, one needs to add standard key: value metadata to the manifest file, located at: – META-INF/MANIFEST.MF within the JAR • Bundle-SymbolicName is the only mandatory header • Bundle-SymbolicName + Bundle-Version uniquely identify a bundle in the OSGi runtime • The manifest file should never be manually edited, as it must follow very specific JVM constraints • A tool is therefore needed: Bnd
47. Bnd • Bnd can be: – A command-line program – An IDE extension – A plugin for a build system (Gradle, Maven, ...) • No matter its form, bnd reads a list of directives (for example, in a .bnd file) and generates a full OSGi jar, with a compliant manifest file. • You can include the usual OSGi headers, as well as syntactic shortcuts
48. Bnd: quick reference • The full documentation of this tool is available on its website. We'll only deal with its basic principles • Bnd supports all the OSGi headers, but shortcuts as well: – * in a package name is a pattern-making character matching anything – for example, javafx.* – A leading ! in a package name/pattern excludes it • The Export-Package, Private-Package and Import-Package directives are paramount, and should be learnt; however, always check that your Bnd tools actually support them in a standard way
49. Bnd and Gradle • Gradle supports OSGi by providing the osgi plugin, which internally employs Bnd • To make a Gradle project OSGI-aware, use: – apply plugin: 'osgi' • Bundle generation can be customized via Bnd: jar { manifest { instruction '<Directive>', ['<value>'[,'value2'...]] [, instruction '<Directive>', …] } }
50. Bnd in the JVM • The JVM is now hosting more and more languages that, while introducing innovative syntax, still are able to compile to bytecode • In particular, Scala (http://scala-lang.org/) is an extremely flexible, elegant, blended language, supporting both the OOP and the functional paradigm • Bnd detects packages not in the source code, but in the bytecode – which makes OSGi compatible with the vast ecosystem of the new languages
51. Section 4 Introduction to Apache Felix
52. OSGi implementations • Equinox: the reference implementation, a fundamental element of the Eclipse IDE • Apache Felix: another widespread open source implementation • … and many more! ^__^! • As long as the standard framework is employed, the choice of the specific implementation depends on other factors (robustness, performances, simplicity, compatibility with other products, …) and does not have to be permanent
53. Why Apache Felix • OSGi-compliant • Minimalist, simple, easy to learn • Employed in other enterprise-level Apache products (Karaf, ServiceMix, ...) • Open source
54. Running Apache Felix • Felix is distributed as a zip bundle, which can be downloaded from its official website • Felix can be started: – Embedded in a Java application - it is a standard JAR – As a standalone jar: cd to its directory, and run: java -jar bin/felix.jar NOTE: running directly from within bin will NOT work: Felix will search the current directory for a bundle subdir, for retrieving startup bundles
55. Gogo – Felix's command line • When running Felix as an application, you will see a command line • Type help to see the list of available commands • Type help <command> to see specific help • Every OSGi shell may have specific commands - for example, in Equinox you might have to employ ss to show the list of bundles in the system, whereas in Felix's Gogo one employs lb
56. Common Gogo tasks • List all the current bundles → lb • Install a bundle → install <URL> • Start an installed/resolved bundle → start <id> • Stop a running bundle → stop <id> • Update a bundle → update <id> [<new URL>]
57. Example project • A small demonstrative project, called OSGi-Test, is available on GitHub. It's a multimodule Java SE application employing a variety of technologies: – OSGi – Scala – JavaFX 8 – Gradle • The repository is hosted on GitHub: https://github.com/giancosta86/OSGi-Test
58. Running the example project • The suggested way to run the application is MoonDeploy, a minimalist, open source tool to automate browser-based software deployment (à la Java Web Start): – https://github.com/giancosta86/moondeploy – In this case, you just have to click and open the file App.moondeploy from the project's download area • Otherwise, you can download the zip file, extract it and run one of the scripts from the bin sub-directory • For further information, please refer to the example project's page on GitHub
59. Example project - Enhancements • The GUI bundle directly depends on a Scala object provided by the hotswap-message bundle: this purposely demonstrates how reloading a bundle can reload its client bundles without stopping the app • However, a drawback exists: a new main window is created! Since we want to depend on a static object, we need to reload and re-initialize the client bundle • A more elegant solution would be to introduce an OSGi service, whose interface and implementation can be decoupled, in order to be able to change the latter without interfering with clients
60. OSGi in the real world • OSGi is not an abstract concept - it is a general- purpose, widespread technology. Examples are: – Eclipse: Equinox, OSGi's reference implementation, is developed as a core component of the IDE – IntelliJ and NetBeans equally employ modular, elegant OSGi-based kernels – Enterprise-level solutions, such as Apache Karaf and Apache ServiceMix, rely on OSGi – The OSGi Alliance is a consortium of technology innovators actively employing OSGi – Scala's core (scala-library) is an OSGi bundle
61. Final considerations • This presentation briefly introduced a minimal, basic part of OSGi, without any claim of completeness but still showing how OSGi can prove an effective technology even at its fundamental core, greatly enhancing class loading – in terms of conceptual clarity, robustness and performances • Much more could be said about: – Its dynamic aspects, revolving around the concept of service and the classes ServiceReference, ServiceTracker and ServiceTrackerCustomizer – The use of annotations, to reduce boilerplate code
62. Further references • OSGi - Official website: https://www.osgi.org/ • OSGI – Wiki: http://wiki.osgi.org/ • Equinox: http://www.eclipse.org/equinox/ • Apache Felix: http://felix.apache.org/ • Apache Karaf: http://karaf.apache.org/ • Apache ServiceMix: http://servicemix.apache.org/ • OSGi in practice: http://njbartlett.name/osgibook.html • Professor Paolo Bellavista's website: http://www.lia.deis.unibo.it/Staff/PaoloBellavista/