Introduction to OSGi - Speaker Deck

Slide 1

Slide 1 text

Gianluca Costa Introduction to OSGi http://gianlucacosta.info/

Slide 2

Slide 2 text

Introduction ● Elegance always matters, especially when creating software – and modularity lies at its very heart ● In this presentation, we'll briefly introduce the fundamentals of OSGi - a dynamic module system for the Java Virtual Machine ● The author would like to thank: – Professor Paolo Bellavista, for his kind and valuable advice and suggestions – Neil Bartlett, author of the beautiful, free, Creative Commons-licensed book OSGi in practice

Slide 3

Slide 3 text

Outline ● Why the traditional Java class loading mechanism is far from being perfect ● Overview of the module system provided by OSGi ● Details of OSGi's class loading ● Creating OSGi bundles ● First steps in Apache Felix ● Example multi-module application with Scala + JavaFX + OSGi + Gradle

Slide 4

Slide 4 text

Section 1 Why OSGi?

Slide 5

Slide 5 text

Modularity ● Modularity is paramount for developing complex systems, as it fosters: – Separation of concerns – Parallel development – Design simplicity – Code reuse – Ease of maintenance and repair

Slide 6

Slide 6 text

What is a module? ● The main defining attributes of a module are: – Self-contained: from the outside, a module must appear as a logical whole – Highly cohesive: a module should have just one, well-defined purpose, and achieve it – Loosely coupled: a module should know nothing about the implementation of the modules it depends on – it must only rely on their public interface ● The JVM employs JAR files - archives containing bytecode and resources. Are JARs modules?

Slide 7

Slide 7 text

JAR hell ● JAR files are not modules! For several reasons: – JARs do not exist at runtime: when searching for a class, the JVM performs a linear scan of the CLASSPATH, starting from its left-most item → this is a very inefficient and unreliable process – in particular, a class might load not a companion class, but a totally unrelated - yet preceding - class in the CLASSPATH – JARs, by default, contain no explicit declaration about dependencies (except a file-system based, very specific, Class-Path declaration)

Slide 8

Slide 8 text

JAR hell (2) – JARs are not versioned – it's the developer that must find a way to track versions – for example, stamping them in every JAR file name – There is no information hiding between JARs: once a class is declared public, it can be accessed by all the other classes in its own JAR, but also by any other class in the system – this can lead to potential side-effects ● So, apart from the fact that they keep files together, for example simplifying Internet downloads, JAR files are not modules – at all

Slide 9

Slide 9 text

What about Gradle, Maven, Ivy...? ● Build systems are brilliant – they automate and extremely simplify the build process, hiding an immense deal of complexity – e.g., they introduce artifact versioning, track down transitive dependencies and perform file-based JAR versioning ● Alas, such versioning is static, a contract respected by the tools only: the JVM keeps working with its linear CLASSPATH, making JAR files lose their boundaries ● Anyway, build systems are precious compile-time complementary tools for OSGi

Slide 10

Slide 10 text

JVM – Class loading ● The JVM loads classes by means of class loaders, instances of java.lang.ClassLoader subclasses ● ClassLoader has 2 duties: 1)Find the bytes of a class – which could reside anywhere 2)Turn them into a Class ● We can only override methods to customize task (1) ● Task (2) is performed by defineClass(), a native final method – so we are unable to change it

Slide 11

Slide 11 text

Class loaders – The delegation tree ● Class loaders are organized in a hierarchical structure: each class loader has a parent, except the native class loader, which is the root of the tree ● A class loader first asks its parent to load a class – and tries to locate such class only if the parent fails (parent-first delegation) ● This is due to security reasons – the loading of core Java classes is enforced to be carried out by the native class loader, and not by a possibly malicious one

Slide 12

Slide 12 text

Java SE – Typical class loader tree Native class loader Extension class loader Application class loader Java core classes (rt.jar) Extension JARs (in $JAVA_HOME/lib/ext) These libraries are available to all apps App-specific JARs

Slide 13

Slide 13 text

Java EE traditional class loader tree Native class loader Extension class loader Application class loader Java core classes (rt.jar) Extension JARs (in $JAVA_HOME/lib/ext) Server-level shared JARs EAR 1 class loader WAR 1 c.l. EJB 1 c.l. EAR 2 class loader WAR 2 c.l. EJB 2 c.l. (...)

Slide 14

Slide 14 text

JAR hell: unpredictable, unreliable ● Not only might a class happen to reference an unknown class contained in another JAR file, in lieu of the expected companion... ● ...but such misbehavior might occur at any moment, perhaps just under certain conditions ● What's more, the resulting errors are usually not trivial (LinkageError, ClassNotFoundException, NoClassDefFoundError) ● Finally, different compile-time versions of a JAR just can't coexist at runtime

Slide 15

Slide 15 text

OSGi ● OSGi is “The dynamic module system for Java™” ● “The OSGi Alliance is a worldwide consortium of technology innovators that advances a proven and mature process to create open specifications that enable the modular assembly of software built with Java technology. Modularity reduces software complexity; OSGi is the best model to modularize Java.” - https://www.osgi.org/

Slide 16

Slide 16 text

OSGi bundles ● Bundle = a module in OSGi. Bundles are plain JAR files with just additional metadata, declared in the standard META-INF/MANIFEST.MF file ● Therefore, OSGi is fully backward-compatible ● Bundle metadata: – Symbolic name (required) – Version (optional, but advised) – Description – List of imported and exported packages – …

Slide 17

Slide 17 text

Immediate benefits of OSGi ● Each bundle has its own class loader ● Dependencies are explicit, by means of import/export declarations for packages ● Bundle encapsulation: only exported packages are visible to other bundles ● Multiple versions of the same bundle – and even of the same package – can coexist at once within an application ● OSGi is dynamic: bundles can be installed, updated and uninstalled without restarting applications

Slide 18

Slide 18 text

OSGi: from trees to graphs ● Traditional Java class loaders compose a tree – with Java EE apps structured like self-contained silos – involving scalability issues, in addition to all the drawbacks imposed by a CLASSPATH within the boundaries of each app ● The OSGi framework reads the declarative import/export package statements and dynamically connects (wires) bundles – bypassing the inefficient, non-scalable global sequential scan ● It's the same as trying to retrieve an element by indexing a hash map in lieu of scanning a huge list

Slide 19

Slide 19 text

The OSGi graph – Further advantages ● There is no more a node-subnode relationship between class loaders – only a network of peers, that can be providers of some packages and consumers of other packages ● In case of missing dependencies, an error occurs immediately, with detailed explanations ● In the graph, dependency links express the consumer/provider relationship for single packages: that is, bundle A can depend on bundle B for package P1, and depend on bundle C for package P2

Slide 20

Slide 20 text

Part 2 Bundles in detail

Slide 21

Slide 21 text

Fine-grained versioning ● Bundles are versioned, just like Gradle or Maven artifacts... ● ...but – which is most important - packages are versioned as well! ● A package version is arbitrary, provided that it's well-formed ● In OSGi, dependencies are mainly expressed in terms of packages – any bundle could provide a package required by another bundle

Slide 22

Slide 22 text

Version syntax ● A version string is subject to the following syntax: MAJOR[.MINOR[.MICRO[.QUALIFIER]]] ● MAJOR, MINOR and MICRO must be numeric, and they are sorted accordingly, as expected ● QUALIFIER is a string: whenever the other components are all equal, QUALIFIER is checked via string comparison. There are a few qualifier patterns: – Development phase: alphaXX/betaXX/final – Timestamp: YYYY-MM-DD_HHMM => scales better

Slide 23

Slide 23 text

Import / export – Headers & Versions ● An exported version must be exact or missing (defaulting to: 0.0.0) → it is like a point in the space ● An imported version must be a range or missing (defaulting to: any version) → it is like a segment ● This design greatly increases flexibility, as a module does not necessarily require recompilation whenever a new version of an imported package is available - provided that only the implementation has changed, not the shared contract

Slide 24

Slide 24 text

Version ranges ● Version ranges are expressed as mathematical ranges: – [V1,V2] → from V1 to V2, both included – (V1, V2) → from V1 to V2, both excluded – [V1, V2) → from V1 to V2, only V1 included – (V1, V2] → from V1 to V2, only V2 included ● Special cases: – V1 (where a range is expected) → V1 or any later – [V1,V1] → V1 only

Slide 25

Slide 25 text

Bundle encapsulation ● All the packages in a bundle are private by default, to minimize namespace pollution and to foster encapsulation ● Bundles must keep implementation as hidden as possible, only exposing interface information

Slide 26

Slide 26 text

OSGi – Class loading in detail Bundle A Bundle class loader Parent class loader Bundle B Bundle C (1) java.* and boot delegation (2) Imported packages (3) Required bundles Internal types (4) Internal classes Fragment X (5) Fragments

Slide 27

Slide 27 text

OSGi – The application class loader Every resolved bundle has its own class loader, not relying on the application class loader - which is still employed by OSGi in 2 cases: 1)Only the native class loader – at the top of the standard class loader tree - can load java.* packages – otherwise, a SecurityException would be thrown 2)A few classes might assume that parent delegation is in use – to avoid breaking compatibility, OSGi defines a bootdelegation parameter – packages that will be loaded just by the parent class loader. It should be used for very particular requirements

Slide 28

Slide 28 text

The system bundle ● The parent class loader transparently loads java.* and bootdelegation classes, as usual ● However, the JRE and its CLASSPATH include several packages – for example, javax.swing.* - which are not automatically loaded: in OSGi, they must be explicitly imported – and it is the system bundle that exports them ● System bundle = meta-bundle encapsulating the OSGi framework at runtime; its id is always 0 ● The packages exported by the system bundle are not standard, but they can be enforced via properties: org.osgi.framework.system.packages[.extra]

Slide 29

Slide 29 text

OSGi – Loading packages ● Class loading is delegated, whenever possible, to the class loaders of other bundles, transitively, following the dependency trail (called wiring) ● Imported packages prevail over local packages, for: – Optimization: a class is globally loaded just once – and not once per bundle – Coherence: because class identity = + , having a versioned FQN loaded by just one class loader prevents cases of ClassCastException ● A bundle can even export its imported packages

Slide 30

Slide 30 text

Version resolution ● Different versions of the same package might be available – but a bundle can export at most one ● Whenever an import requested by a bundle – via the Import-Package header - must be resolved: 1)If one matching version - that is, one version belonging to the range of the import declaration – is available, it is automatically wired 2)If two or more matching versions are available, the highest of them is chosen 3)Otherwise, it's the bundle with the lowest id that provides the package

Slide 31

Slide 31 text

Required bundles ● Via the Require-Bundle header, a bundle can directly depend on another bundle, not on packages ● Depending on bundles is definitely discouraged: – The efficient granularity at the heart of OSGi is lost – The architecture becomes fragile, as splitting a bundle could cause unresolved runtime missing dependencies, not eagerly detected by the OSGi framework ● If a package cannot be found via import/export wiring, such package is looked for in the Export- Package header of the required bundles

Slide 32

Slide 32 text

Internal types ● A bundle is a regular JAR file, so it includes types and resources ● Packages can be private or exported – but they can all be used by the bundle itself, of course ● Packages are private by default, unless they are listed in the Export-Package header – NOTE: always check your tools, as they might actually export packages by default – Furthermore, tools might create a Private-Package pseudo-header, ignored by the OSGi framework

Slide 33

Slide 33 text

Fragments ● Fragment = set of types and resources that cannot exist on its own – it needs to be attached to a bundle, providing additional types and resources. ● Fragments are especially valuable in a few common situations: – Multi-platform support: usually, fragments integrate a cross-platform bundle by adding a few platform- specific implementations – Resource bundles: to provide small customer-specific variations – for example, logos, icons, or localization

Slide 34

Slide 34 text

Bundle dependencies ● Dependency = Set of assumptions made by a software component about its context ● Bundles can require 3 types of dependencies: 1)Execution environment specification 2)Package imports 3)Bundle requirements ● OSGi even supports simultaneous bundle resolution, thus enabling circular dependencies

Slide 35

Slide 35 text

Environment specification ● Bundle-RequiredExtensionEnvironment header ● Describes the expected Runtime Enviroment, for example JavaSE-1.8. ● JRE classes are extended, version after version, with new methods and properties – which, if called in the context of an older JRE, would cause an unpredictable NoSuchMethodError ● The solution is twofold: 1)Compile your code using exactly the target JDK 2)Enforce the requested JRE with the OSGi header

Slide 36

Slide 36 text

Package imports ● The header for declaring required packages is: Import-Package: [;version=] [,[;version=]][,....] ● version is optional and, if omitted, means “any version” ● Employing a punctual version means “that version or later”

Slide 37

Slide 37 text

Bundle requirements ● Requiring a bundle is really discouraged ● Anyway, the header syntax is: Require-Bundle: [;bundle-version=][,[;bundle-version=]][,...] ● Versioning works like package versioning ● Bundle-Version is the related header, which has an important side-effect: it enables multiple, different versions of a bundle to be loaded at the same time

Slide 38

Slide 38 text

Bundle lifecycle INSTALLED RESOLVED STARTING ACTIVE STOPPING UNINSTALLED Install Resolve/ Install Start Stop Refresh/Update Refresh/ Update Uninstall Uninstall

Slide 39

Slide 39 text

Bundle lifecycle - States ● INSTALLED: the bundle has been loaded, but nothing is known about its dependencies ● RESOLVED: the 3 kinds of dependencies have been resolved and the bundle is ready to run ● ACTIVE: the bundle is exporting packages, and its services are running ● UNINSTALLED: the bundle has been removed from the environment ● STARTING, STOPPING: temporary states, where the bundle activator is running

Slide 40

Slide 40 text

Further details on the lifecycle ● A bundle can move from INSTALLED to RESOLVED if its dependencies are RESOLVED or they can be transitively resolved at that moment ● Calling Start on an INSTALLED bundle moves it to RESOLVED, then to STARTING, and finally to STARTED - if dependencies are satisfied, of course ● Update loads a new version of a bundle ● Refresh re-wires bundle dependencies after one or more bundles have been updated ● The state of a bundle can also be affected by the state of bundles on which it depends

Slide 41

Slide 41 text

BundleActivator ● Every bundle can have a bundle activator – a class implementing the BundleActivator interface and registered with the following manifest header: – Bundle-Activator: ● Its start() and stop() methods are called in the STARTING and STOPPING phases – so, they should be as fast as possible; however, for long- running tasks, an activator can spawn threads ● The bundle activator can reside in a private package ● A new activator instance is created for every STARTING / STOPPING pair

Slide 42

Slide 42 text

BundleContext ● BundleContext is a handle to access the framework ● It includes methods for programmatically inspecting and manipulating the current bundles: – Read metadata – Change lifecycle state – Install new bundles – … ● A common way of obtaining a BundleContext is via BundleActivator's start(BundleContext context) method

Slide 43

Slide 43 text

BundleListener ● A bundle listener is called on lifecycle events triggered by any bundle ● It must implement the BundleListener interface ● It can be registered an unregistered using BundleContext's methods: – addBundleListener(...) – removeBundleListener(...)

Slide 44

Slide 44 text

OSGi = Modularity + Services ● Up to now, OSGi might appear as a brilliant, package-level runtime extension of the versioning ideas introduced by build systems ● Actually, OSGi is much more - it provides a full- fledged lifecycle management for services and a rich API to publish, locate and consume them ● We won't deal with services in this presentation, but most books, as well as the reference documentation, describe them in great detail

Slide 45

Slide 45 text

Section 3 Creating OSGi bundles

Slide 46

Slide 46 text

Bundle = JAR + OSGi metadata ● To create a bundle, one needs to add standard key: value metadata to the manifest file, located at: – META-INF/MANIFEST.MF within the JAR ● Bundle-SymbolicName is the only mandatory header ● Bundle-SymbolicName + Bundle-Version uniquely identify a bundle in the OSGi runtime ● The manifest file should never be manually edited, as it must follow very specific JVM constraints ● A tool is therefore needed: Bnd

Slide 47

Slide 47 text

Bnd ● Bnd can be: – A command-line program – An IDE extension – A plugin for a build system (Gradle, Maven, ...) ● No matter its form, bnd reads a list of directives (for example, in a .bnd file) and generates a full OSGi jar, with a compliant manifest file. ● You can include the usual OSGi headers, as well as syntactic shortcuts

Slide 48

Slide 48 text

Bnd: quick reference ● The full documentation of this tool is available on its website. We'll only deal with its basic principles ● Bnd supports all the OSGi headers, but shortcuts as well: – * in a package name is a pattern-making character matching anything – for example, javafx.* – A leading ! in a package name/pattern excludes it ● The Export-Package, Private-Package and Import-Package directives are paramount, and should be learnt; however, always check that your Bnd tools actually support them in a standard way

Slide 49

Slide 49 text

Bnd and Gradle ● Gradle supports OSGi by providing the osgi plugin, which internally employs Bnd ● To make a Gradle project OSGI-aware, use: – apply plugin: 'osgi' ● Bundle generation can be customized via Bnd: jar { manifest { instruction '', [''[,'value2'...]] [, instruction '', …] } }

Slide 50

Slide 50 text

Bnd in the JVM ● The JVM is now hosting more and more languages that, while introducing innovative syntax, still are able to compile to bytecode ● In particular, Scala (http://scala-lang.org/) is an extremely flexible, elegant, blended language, supporting both the OOP and the functional paradigm ● Bnd detects packages not in the source code, but in the bytecode – which makes OSGi compatible with the vast ecosystem of the new languages

Slide 51

Slide 51 text

Section 4 Introduction to Apache Felix

Slide 52

Slide 52 text

OSGi implementations ● Equinox: the reference implementation, a fundamental element of the Eclipse IDE ● Apache Felix: another widespread open source implementation ● … and many more! ^__^! ● As long as the standard framework is employed, the choice of the specific implementation depends on other factors (robustness, performances, simplicity, compatibility with other products, …) and does not have to be permanent

Slide 53

Slide 53 text

Why Apache Felix ● OSGi-compliant ● Minimalist, simple, easy to learn ● Employed in other enterprise-level Apache products (Karaf, ServiceMix, ...) ● Open source

Slide 54

Slide 54 text

Running Apache Felix ● Felix is distributed as a zip bundle, which can be downloaded from its official website ● Felix can be started: – Embedded in a Java application - it is a standard JAR – As a standalone jar: cd to its directory, and run: java -jar bin/felix.jar NOTE: running directly from within bin will NOT work: Felix will search the current directory for a bundle subdir, for retrieving startup bundles

Slide 55

Slide 55 text

Gogo – Felix's command line ● When running Felix as an application, you will see a command line ● Type help to see the list of available commands ● Type help to see specific help ● Every OSGi shell may have specific commands - for example, in Equinox you might have to employ ss to show the list of bundles in the system, whereas in Felix's Gogo one employs lb

Slide 56

Slide 56 text

Common Gogo tasks ● List all the current bundles → lb ● Install a bundle → install ● Start an installed/resolved bundle → start ● Stop a running bundle → stop ● Update a bundle → update []

Slide 57

Slide 57 text

Example project ● A small demonstrative project, called OSGi-Test, is available on GitHub. It's a multimodule Java SE application employing a variety of technologies: – OSGi – Scala – JavaFX 8 – Gradle ● The repository is hosted on GitHub: https://github.com/giancosta86/OSGi-Test

Slide 58

Slide 58 text

Running the example project ● The suggested way to run the application is MoonDeploy, a minimalist, open source tool to automate browser-based software deployment (à la Java Web Start): – https://github.com/giancosta86/moondeploy – In this case, you just have to click and open the file App.moondeploy from the project's download area ● Otherwise, you can download the zip file, extract it and run one of the scripts from the bin sub-directory ● For further information, please refer to the example project's page on GitHub

Slide 59

Slide 59 text

Example project - Enhancements ● The GUI bundle directly depends on a Scala object provided by the hotswap-message bundle: this purposely demonstrates how reloading a bundle can reload its client bundles without stopping the app ● However, a drawback exists: a new main window is created! Since we want to depend on a static object, we need to reload and re-initialize the client bundle ● A more elegant solution would be to introduce an OSGi service, whose interface and implementation can be decoupled, in order to be able to change the latter without interfering with clients

Slide 60

Slide 60 text

OSGi in the real world ● OSGi is not an abstract concept - it is a general- purpose, widespread technology. Examples are: – Eclipse: Equinox, OSGi's reference implementation, is developed as a core component of the IDE – IntelliJ and NetBeans equally employ modular, elegant OSGi-based kernels – Enterprise-level solutions, such as Apache Karaf and Apache ServiceMix, rely on OSGi – The OSGi Alliance is a consortium of technology innovators actively employing OSGi – Scala's core (scala-library) is an OSGi bundle

Slide 61

Slide 61 text

Final considerations ● This presentation briefly introduced a minimal, basic part of OSGi, without any claim of completeness but still showing how OSGi can prove an effective technology even at its fundamental core, greatly enhancing class loading – in terms of conceptual clarity, robustness and performances ● Much more could be said about: – Its dynamic aspects, revolving around the concept of service and the classes ServiceReference, ServiceTracker and ServiceTrackerCustomizer – The use of annotations, to reduce boilerplate code

Slide 62

Slide 62 text

Further references ● OSGi - Official website: https://www.osgi.org/ ● OSGI – Wiki: http://wiki.osgi.org/ ● Equinox: http://www.eclipse.org/equinox/ ● Apache Felix: http://felix.apache.org/ ● Apache Karaf: http://karaf.apache.org/ ● Apache ServiceMix: http://servicemix.apache.org/ ● OSGi in practice: http://njbartlett.name/osgibook.html ● Professor Paolo Bellavista's website: http://www.lia.deis.unibo.it/Staff/PaoloBellavista/