Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Platform Cloud Service "Black Belt" Training: Custom Policies Development

API Platform Cloud Service "Black Belt" Training: Custom Policies Development

This is the Custom Policies Development chapter of the training that I delivered on Munich, 2018.

Ricardo Ferreira

April 10, 2018
Tweet

More Decks by Ricardo Ferreira

Other Decks in Programming

Transcript

  1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 2
  2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 3
  3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Creating Custom Policies? • Although creating custom policies might be the "Way To Go" for most use cases, you have to make sure that creating one won't bring you more issues than peace of mind. • Any custom policy by itself is a project implementation that you'll need to maintain, evolve and keep developers/testers onboard. It is a piece of code that will need to evolve as the product itself evolves, and that sometimes is not that pleasant. • Likewise, custom policies can make your project dependent on code, which means that sometimes you won't be able to upgrade your gateway because your custom policy is tied to some dependency specific to that version. Top 3 Reasons why Someone Should Create a Custom Policy
  4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Creating Custom Policies? • Here is the top 3 reasons why someone should create a custom policy: – The behavior is simply way too complex: it happens when the behavior is complex enough to be implemented by simply using the Groovy policy. Here, code complexity can be measured by the amount of lines of code; by the algorithm's/patterns that are required for it or, if the code relies on external libraries in order to work properly. – The behavior is repeatable and reusable: if the behavior to be implemented is not tied to a specific problem and rather; it is applicable for many other contexts, you should definitely create a custom policy for it. The rule of thumb here is making sure that the custom policy might be reusable across multiple APIs and not just only one. – The Groovy policy restricts certain code: The Groovy policy has some restrictions about packages/classes/methods from the JDK. Therefore, you might not be able to deploy the code and building a custom policy will be your only option. Top 3 Reasons why Someone Should Create a Custom Policy
  5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 6
  6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The Policy SDK is the set of libraries, resources and techniques that allows software developers to create their own custom policies, which in turn will be deployed in the API Platform to be used within the APIs. • The Policy SDK is the foundation of any policy in the product, whether if it is a built-in policy or a custom policy. All policies are built depending of the same libraries and packaged/deployed using the same techniques. • In terms of layers, the Policy SDK comprises the following components: Understanding the details about the Policy SDK Implementation Libraries UI Development Framework Deployment Engine + +
  7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The implementation libraries are the set of JAR files that you will need to compile your code properly. By default, you will need at least 3 JAR files: – oracle.apiplatform.policies.sdk.jar – oracle.apiplatform.sharedlib.jar – oracle.apiplatform.shared.jar – oracle.apiplatform.utils.jar • These JAR files are required to properly compile the config and runtime code of your policy. They can be obtained from the API Gateway installer. • Make sure to always use the most recent version of these JAR files, when your API Gateway get upgraded. The Policy SDK keeps changing all the time. Understanding the Implementation Libraries
  8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • Within the API Gateway installer, the implementation libraries can be found inside the following file: – oracle.apiplatform.sharedlib.ear • Extract the contents of this file into a temporary folder. Once you do that, you will find the implementation libraries in the following location: – $TEMP_FOLDER/APP-INF/lib Understanding the Implementation Libraries
  9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • To better understand the usage of certain classes/interfaces from the Policy SDK, it is a best practice to read the Online Java API Reference for APIPCS: – https://docs.oracle.com/en/cloud/paas/api-platform-cloud/api-platform- javadoc/toc.htm Understanding the Implementation Libraries
  10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The UI development framework is the set of built-in implementations such as JavaScript Callback Handlers, CSS, HTML DIV's that allows developers to build UI's for their custom policies. • It allows the development of both the read/write and read-only views of a custom policy. Understanding the UI Development Framework
  11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The policy UI design is based on the dynamic injection of HTML code built by the developer. The HTML markup must be enclosed inside a DIV who's ID is "policy-ui-content". This ID ensures that the framework will retrieve the content for the markup and populate in a dialog box. • If an HTML UI element needs to have a value stored in the backend model, then it must use declarative bindings to associate the UI element with the backend model. Understanding the UI Development Framework
  12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The UI development framework is based on an open-source web framework called Knockout.js. This is a very lightweight MVC implementation that uses HTML-based code for the presentation layer and JavaScript code for all the rest. Therefore, it is very important to get familiar with this framework. • There are many examples available online where you can get familiar with the UI controls, as well as understanding how to implement them correctly. – http://knockoutjs.com/examples/helloWorld.html – You can try these examples live using jsFiddle !!! • Keep in mind that whatever behavior you expect to implement for the UI, it has to be implemented using JavaScript. That might restrict things a little bit. Understanding the UI Development Framework
  13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The UI development framework has a built-in communication model where the UI state is passed back and forth. It's based on specific callback functions that are used to initialize and complete the execution of the policy UI. Understanding the UI Development Framework • Be careful while building the JSON payload. It has to be a valid content otherwise either the UI or the policy code will fail. • Usually, it is hard to troubleshoot errors originated from bad JSON parsing. It may take you hours.
  14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • The deployment engine is composed by the set of mechanisms that will pick the custom policy JAR file and install into the gateway. It starts with the REST API that are available for that, but by far is not simply that. The API Platform Mgmt Tier exposes a REST API that is used to install the custom policy, but what this REST API does is merely register it into its database. • After that, policies are pulled to the gateway by the gateway controller. The gateway manager downloads the EAR file (from the policy JAR) and apart from that it makes a few changes, ultimately generating a new EAR file. • The new EAR will contain: – Lifecycle Listener: class the bootstrap the policy runtime – Concrete Classes: All the concrete classes for the DAF action Understanding the Deployment Engine
  15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Introduction to the Policy SDK • A subclass of ApipPolicyLifecycleListener is auto generated by the gateway controller. This class has the following responsibilities: – Reads the metadata.json from the EAR – Register the ApipAction to DAF – Register the PolicyRuntimeFactory – Register Policy Security Handler – Register the Policy DAF Translator – Translate Policy to Action Config – Plumbing of the OCSG native calls • Remember that a custom policy is a layer on top of OCSG DAF… Understanding the Deployment Engine
  16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 17
  17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying • In a nutshell; create a custom policy is no different than creating any Java project, where you will start by creating the necessary artifacts in your IDE. • However, different from regular Java projects (Servlets/JSP, EJB's, Spring) an custom policy project must contain: – Metadata: File that provides generic metadata about the policy. It contains the location of the necessary artifacts such as Configuration, UI and the Runtime. – Configuration: Implementation that covers all the design-time features, such as policy validation and helper classes used for internationalization. Also may contain constants. – UI Layer: Set of web resources used to render the provider-specific UI for edit/views. – Runtime: Contains the code used in runtime. It is where you will write de code for the policy behavior, and whatever needs to happen when a message comes in for an API. First Steps to Create a Custom Policy: Project Structure
  18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying Code related to the Configuration Layer First Steps to Create a Custom Policy: Project Structure Code related to the Runtime Layer Policy metadata configuration file Code related to the UI Layer
  19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying Ant build script that will do the magic  First Steps to Create a Custom Policy: Project Structure Third-party dependencies for the policy
  20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying First Steps to Create a Custom Policy: metadata.json Policy unique name. Must start with "o:" Version + Revision must be unique per build Identifier of the showing category Set of constraints to control UI behavior Location of UI resources Location of the L10n file Config/Runtime services
  21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying First Steps to Create a Custom Policy: Policy Categories @implementations.policyCategory.security @implementations.policyCategory.trafficMgmt @implementations.policyCategory.interfaceMgmt @implementations.policyCategory.routing @implementations.policyCategory.other
  22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying • direction: represents where a policy can be inserted: one of the following 4 possibilities: REQUEST, RESPONSE, REQUEST_AND/OR_RESPONSE, REQUEST_AND_RESPONSE • singleton: represents whether the policy is allowed only once. It is true by default. • requires: a policy might require other policies to be inserted before or in the other pipeline (i.e. this policy requires other policies in order to work). • before: a policy might need to be inserted before other policies if they are present (i.e. this policy is before those policies). For example, the API Request policy must be before the Service Request policy • after: a policy might need to be inserted after other policies if they are present (i.e. this policy is after those policies). For example, the Service Request policy must be after the API Request policy. First Steps to Create a Custom Policy: Policy Constraints
  23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Creating/Packaging/Deploying First Steps to Create a Custom Policy: Packaging Structure • The packaging structure of a custom policy is way different from other Java packages you ever seen. The following structure must be used: customPolicy.jar metadata.json config.jar runtime.ear ui.zip config.jar runtime.jar
  24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • The management tier of the API Platform provides a REST API to request the creation of a custom policy: • Keep in mind that you have to have the proper role in order to invoke this REST API. Specifically, you need to be part of the "APIPlatformSysAdmins". • Also, if you try to deploy the custom policy into a customer-ready APIP pod, you might need to import the certificate of the management tier. That is the certificate installed in the OTD that front the management tier servers. – Tip: If you are using cURL to deploy, you can skip this step by using the '-k' option. Creating/Packaging/Deploying First Steps to Create a Custom Policy: Deploy curl -X POST -u apics-system-admin:password -H 'Content-Type:application/octet-stream' http://server:port/apiplatform/administration/v1/policies --data-binary customPolicy.jar
  25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • If you plan to develop your custom policies using the Eclipse IDE, you may leverage the Ant script developed by the A-Team to automate certain tasks. • The script provides the following facilities: – Allows the customization of the mgmt tier endpoint through a properties file. – Automatically manages the revision of your custom policies, so you don't need to worry about updating the metadata.json every time you run a build. – Compiles your code using the product specifications, as well as include debugging flags to ensure that your policy will be "Debuggable" using JDWP (Java Debug Wire Protocol) – Packages the custom policy using the JAR structure that the REST API expects. – Automatically deploys your custom policy by invoking the REST API for you  – Make sure to bypass trust certificates, when the HTTP connection is based on SSL. Creating/Packaging/Deploying Using the A-Team Automated Script to Package/Deploy
  26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • Starting from the 18.1.5 release, there is an official product documentation about how to develop custom policies: – https://docs.oracle.com/en/cloud/paas/api-platform-cloud/apiplatform- custompolicies/developing-and-deploying-custom-policies.html • Although there might have some inconsistencies there, it is still the best way to learn what can be done about custom policy development. Most of what is written there came from the A-Team and thus, you might see an similar content there. • Also, the policy generator that you will work out in the lab is being officially delivered as part of the API Gateway installer. That means supportable code. Creating/Packaging/Deploying Product Documentation about the Development of Custom Policies
  27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 28
  28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 29 Hands-On with Custom Policies
  29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    Why Create Custom Policies? Introduction to the Policy SDK Creating/Packaging/Deploying Hands-On with Custom Policies Debugging Custom Policies 1 2 3 4 5 Agenda 30
  30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • As a best practice, always prepare your environment to allow debugging of the code you write. Verifying if your custom policy is working properly may be time consuming, since the code you write is spread over multiple layers. • For the runtime layer, you need to enable remote debugging in gateway's JVM. Use the following command line to enable that: -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n • This enables JDWP over the port 9999. That means that you can use your IDE to connect to this port for a debugging session. Most Java IDEs support this type of debugging. – Make sure that the port is visible outside the Docker Container/VM where the gateway is running. This is also true if the gateway sits behind a firewall (i.e.: JCS Security Rules) Debugging Custom Policies Setting Up Debugging for the Runtime Layer
  31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • On Eclipse, go to "Debug Configurations" and create a "Remote Java App" connection. Once connected, you can simply add breakpoints to your code* Debugging Custom Policies Setting Up Debugging for the Runtime Layer * Make sure that your code is generated with the appropriate debugging flags. In particular, the bytecode must be generated using line number attributes.
  32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • If you need to debug the UI layer, then you must use the debugging tools from your preferred web browser. This section is going to show how to do this using Google Chrome. • On Google Chrome, press the F12 key to open the Developer Tool: Debugging Custom Policies Setting Up Debugging for the UI Layer
  33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • Open the file "editPolicyDialog.js" and set a breakpoint as shown below: Debugging Custom Policies Setting Up Debugging for the UI Layer
  34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

    • From that point on, keep pressing the F11 key until you hit into a file named VMnnnn.js, where 'n' are numbers. This is your JavaScript implementation  Debugging Custom Policies Setting Up Debugging for the UI Layer