Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Integrating Jython with Java by Jim Baker and Shashank Bharadwaj

Integrating Jython with Java by Jim Baker and Shashank Bharadwaj

PyCon 2013

March 17, 2013
Tweet

More Decks by PyCon 2013

Other Decks in Programming

Transcript

  1. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Integrating Jython

    with Java Jim Baker Shashank Bharadwaj March 16, 2013
  2. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Jython status

    What’s new in Jython Jython 2.7 beta 1 just released! We will be sprinting on Jython 2.7 during the sprints
  3. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Why Jython?

    Very compliant implementation of Python: Jython 2.5 tracked CPython 2.5 Jython 2.7 tracks CPython 2.7 Java ecosystem is huge and growing This size is even beneficial Pick the domain, there’s likely to be a library for it Some of them are quite good
  4. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Why not

    Jython? No C extension API, ctypes, Cython, etc Certainly possible however
  5. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Other possibilities

    But first, other integrations are possible JEPP - embed CPython within the JVM using JNI JPype - embed JVM within CPython using JNI Bridging between CPython and Java using interprocess communication Or using IPC between CPython and Jython
  6. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Example: execnet

    Bridges CPython and Jython efficiently: import execnet gw = execnet.makegateway("popen//python=jython") channel = gw.remote_exec(""" from java.util import Vector v = Vector() v.add(’aaa’) v.add(’bbb’) for val in v: channel.send(val) """) for item in channel: print (item)
  7. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Issues with

    alternatives But the integrations are very shallow: Callbacks Extending Java classes in Python etc etc
  8. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Jython compatibility

    High degree of compatibility, with the regrtest to prove it Keep in mind: different GC model which impacts finalization/bad code No GIL of course And threading is currently efficient! Sequential consistency (* almost)
  9. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Talk examples

    Metacircular approach: use as examples snippets from Jython tests and libraries Models CPython - tests and libraries use Python too Tests cover lots of corner cases Testing Jython with Jython itself is a very pleasant way to test Occasional disadvantages, such as requiring a substantial amount of the language and runtime be availableh Aside: Jython could see more unit tests of the runtime
  10. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Java from

    Jython Python code can directly import Java classes, as if they were Python classes Construct Java objects from Java classes, as if they were Python classes Work with Java objects. Your programs can call Java methods, or have Java call back into Python. And this “magic”/integration works because we have built a proxy on the fly
  11. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Apache POI:

    Read and write MS Office docs from contextlib import closing from org.apache.poi.hssf.usermodel import * from java.io import FileInputStream with closing(FileInputStream("data.xls")) as fis, \ closing(HSSFWorkbook(fis)) as wb: sheet = wb.getSheetAt(0) num_rows = sheet.getPhysicalNumberOfRows() for i in xrange(num_rows): row = sheet.getRow(i) if row: num_cols = sheet.getRow(i).\ getPhysicalNumberOfCells() for j in xrange(num_cols): cell = row.getCell(j) # then do something interesting!!!
  12. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Python and

    Java type equivalence Java type Python type char str boolean bool byte, short, int, long int java.lang.String unicode byte[], char[] str java.lang.Class JavaClass x[] array.array org.python.core.PyObject unchanged Foo PyJavaType proxying Foo
  13. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Achieving equivalence

    Jython uses the following methods tojava java2py Usually used transparently Sometimes significant overhead because of boxing/unboxing Not always because boxing/unboxing is a common idiom in Java, so JVMs can often optimize this away (* significant but it depends!)
  14. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Example: datetime.py

    if _sys.platform.startswith(’java’): def __tojava__(self, java_class): if java_class not in (Calendar, Date, Object): return Py.NoConversion calendar = Calendar.getInstance() calendar.clear() calendar.set( self.year, self.month - 1, self.day) if java_class == Calendar: return calendar else: return Date(calendar.getTimeInMillis())
  15. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Use of

    overloaded methods What happens when you call overloaded methods? How does Jython pick the one you want? Sometimes the runtime picks wrong Force it by using constructors from java.lang - Boolean, Integer, Double, etc
  16. Integrating Jython with Java Jim Baker, Shashank Bharadwaj JSR-223: Scripting

    for the Java platform Part of the Jython release since 2.5.1 JSR 223 is intentionally shallow - common operations we expect to see in scripting scenarios Can still be incredibly effective in driving Jython at the top level Just like we would expect in a good script
  17. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Example code

    public void testEvalString() throws ScriptException { ScriptEngineManager manager = new ScriptEngineManager(); ScriptEngine pythonEngine = manager.getEngineByName("python"); assertNull(pythonEngine.eval("x = 5")); assertEquals(Integer.valueOf(5), pythonEngine.eval("x")); } Source: /tests/java/org/python/jsr223/ScriptEngineTest.java
  18. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Jython, testing

    Jython You can even test JSR 223 support from Jython itself: def test_factory(self): engine = ScriptEngineManager().\ getEngineByName("python") f = engine.factory language_version = ".".join(str(comp) for comp in sys.version_info[0:2]) impl_version = ".".join(str(comp) for comp in sys.version_info[0:3]) self.assertNotEqual(f.scriptEngine, engine) self.assertEqual(f.engineName, "jython") self.assertEqual(f.engineVersion, impl_version) self.assertEqual(set(f.extensions), set([’py’])) self.assertEqual(f.languageName, "python")
  19. Integrating Jython with Java Jim Baker, Shashank Bharadwaj And more

    such tests # variants self.assertEqual( f.getParameter(ScriptEngine.ENGINE), "jython") self.assertEqual( f.getParameter(ScriptEngine.ENGINE_VERSION), impl_version) self.assertEqual( f.getParameter(ScriptEngine.LANGUAGE_VERSION), language_version) self.assertEqual( f.getOutputStatement("abc"), "print u’abc’") self.assertEqual( f.getProgram("x = 42", "y = ’abc’"), "x = 42\ny = ’abc’\n")
  20. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Or perhaps

    not But not without issues, as we found in our own usage: # XXX Earlier version of this test also tested # put, get, eval on the engine, however this # introduced action at a distance where aspects # of the sys state changed # (notably sys.stdin.newlines), which then # impacted test_univnewlines later in the regrtest. # # For now, there may be limits in how much # we can test Jython from itself, no matter # how attractive from an ouroboros perspective that # may be :). Certainly worth revisiting in 2.6. (Or later as the case may be.)
  21. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Polyglot programming,

    from Groovy engine = mgr.getEngineByName("jython") engine.eval(’’’ def factorial(n): if i == 0: return 1 else: return n * factorial(n - 1) result = factorial(4) ’’’) println ’jython: ’ + engine.result Slightly modified example from http://groovy.codehaus.org/JSR- 223+access+to+other+JVM+languages
  22. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Callbacks Callbacks

    are commonly used in Java Generally requires a fair amount of boilerplate code, even with anoymous classes And generally you need to pass in some state
  23. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Using closures

    instead Java methods that take objects of single method interfaces Useful for working with Callable or Runnable interfaces Can pass in a lexical closure, that is a function that closes over variables in its lexical scope
  24. Integrating Jython with Java Jim Baker, Shashank Bharadwaj How does

    that actually work? We need to support Python -> Java -> Python For lexically closed variables, you are actually using a closure For globally scoped variables, Python can look up the right state referenced by a thread local, transparent to the using Python code
  25. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Using Python

    code from Java: Object factories Java code can use the Jython runtime to construct Python objects Object factories use Jython-specific APIs to efficiently build Python objects
  26. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Example: Use

    the runtime Use the Python runtime to get a PyObject for the type of interest public class SomeClassFactory { private PyObject someClass; public SomeClassFactory() { PythonInterpreter interpreter = new PythonInte interpreter.exec("from something import SomeCl someClass = interpreter.get("SomeClass"); }
  27. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Construct away!

    public SomeClassType create (String x, String y) { PyObject someObject = someClass.__call__( new PyString(x), new PyString(y)) return (SomeClassType)someObject.__tojava__( someClassType.class); } }
  28. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Gradual Typing

    A new type system proposed by Jeremy Siek & Walid Taha; extended by others Seamless integration between statically typed parts and dynamically typed parts of code in the language Guarantees that statically typed part (after type checking) will not raise type error at runtime.
  29. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Gradual Typing

    for Python gradual module for Python 3.x Provides type checking based on user defined type annotations Uses function annotations and decoraters
  30. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Gradual Typing

    for Python - Example from gradual import typed @typed class YourAwesomeApi(object): def answerToLife(self, a: int, b: int) -> int: return a + b def use_api(x): print(x.answerToLife(10, 32)) print(x.answerToLife(’hello’, ’there’)) # TypeError: Expected int, got: # ’hello’ of type str use_api(YourAwesomeApi())
  31. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Gradual Typing

    for Python: Work in Progress Prototype - in experimental mode! pip install gradual to get it Repo at: http://bitbucket.org/gradual/py-gradual Working on accurate representation of objects Working on Blame Tracking to produce accurate error messages
  32. Integrating Jython with Java Jim Baker, Shashank Bharadwaj Future: Integrate

    Gradual Typing into Jython Use Gradual Typing to enable typed-Jython code Enable better Java integration type information can be used to remove wrappers in Java code reduce the proxies involved Enable optimizations Drive type information to generate idiomatic Java code (where applicable)