PyConZA 2014: "Python at the Observatory - Old telescopes, new instruments" by Carel van Gend & Briehan Lombaard

Astronomy in South Africa has a long and illustrious history. In addition to the premier, 10-m Southern Africa Large Telescope (SALT) , there are a number of small- and medium-sized telescopes hosted at the South African Astronomical Observatory that still produce a great deal of valuable data. We'd like to show how we're using new instruments and the nimbleness of Python to bring seventy-year-old telescopes into the 21st century.

Our long-term goal is to have the telescopes and associated instruments be remotely operable, easy to use, and robust enough that valuable data is reliably captured and stored. In addition, we want to make it as easy as possible to update and improve the software which controls the instruments.

We have developed a distributed framework to control the instruments. Drivers for individual components may be written in whichever language is most suitable (we've used C, C++ and Python where appropriate). These drivers communicate over a TCP/IP socket with a Python controller process, and on top of that is a Python/Flask driven web interface.

We plan to adapt the framework to new instruments (such as a wide-field camera for the 1.9- and 1-m telescopes), under-development instruments (such as an upgrade to the Cassegrain spectrograph on the 1.9-m telescope), and retrofit to older instruments as feasible.

As a demonstration of the new framework, we will present our work of migrating the control software for the Sutherland High-speed Optical Camera (SHOC) from being a disparate collection of proprietary, Windows-based software, to a unified, open, web accessible system. SHOC is a high-speed, accurately-timed, imaging instrument that can be mounted on the 1.9-, 1-, and 0.75-m telescopes in Sutherland, control of which includes that for a camera, a global positioning system (GPS), and a filter wheel.

Transcript

1. Python at the Observatory old telescopes, new instruments Carel van Gend & Briehan Lombaard
2. A very brief history of the observatory... Navigation (and avoiding shipwrecks) required accurate knowledge of star positions, and also accurate timekeeping. In 1820, the British Admiralty established an observatory at the Cape, for the purposes of determining the positions of the southern stars.
4. Telescopes in the Karoo: then... 1972: Royal Observatory and Republic Observatory merged, forming the South African Astronomical Observatory. Headquarters in Cape Town and observing station in Sutherland.
5. 2014: 21 telescopes on the plateau. ...and now
6. Southern African Large Telescope The flagship telescope is SALT. One of the world’s largest optical telescopes. Run by a large team of astronomers, operators, technicians and software engineers.
7. What about the other telescopes? There are several so-called small telescopes. Good optics are (still) good optics. Sutherland is (still) a good site. The small telescopes should not be neglected!
9. Modernising old telescopes How do we keep them useful? Do lots of mechanical maintenance. Build up-to-date new instruments (and write software for them.) Improve data analysis and reduction pipeline.
10. What instruments should be and do • Must work correctly (and be verifiable). • Must be robust and resilient against failure. • Be easy to use, and support multiple modes of access. • They should be easy to maintain. • They should be future-proof.
11. What else? Why should the observer even need to travel all the way to Sutherland? • The instruments should be remotely accessible. • Maybe even robotically operable (run by a script).
12. The SAAO software framework • Supports many different types of hardware • Supports multiple languages • Supports distributed components • Allows new components to be added
13. Framework design • Instrument components separated into user interface, control and hardware driver layers. • Each layer’s implementation may vary independently of the others. • Components communicate via TCP/IP.
14. Framework details • Each instrument driver acts as a server. • A Python controller for each instrument acts as a client. • Client/server interactions mediated using Apache Thrift. • A “controller controller” instantiates each of these, and provides extra business logic.
16. Apache Thrift • Interface (services) specified by an IDL. • Thrift compiler generates stub code for client and server (and all boilerplate code). • Huge number of languages supported. • We use Python where we can, C++ where we must!
17. Thrift IDL example service FilterwheelService { void do_init(1: string wheel); void do_move(1: string wheel, 2: i32 position); map<string,map<string, i32>> get_status(); }
18. Python server code class Iface: def do_init(self, wheel): pass def do_move(self, wheel, position): pass def get_status(self): pass
19. Python client code class FilterwheelController: def __init__(self, tcp_host='localhost', tcp_port=9090): try: self.transport = TTransport.TBufferedTransport(TSocket.TSocket(tcp_host, tcp_port)) protocol = TBinaryProtocol.TBinaryProtocol(self.transport) self.client = Client(protocol) self.transport.open() except Thrift.TException, tx: print '%s' % (tx.message) exit(1) def get_status(self): response = self.client.get_status() return response def do_init(self, wheel): response = self.client.do_init(wheel) def do_move(self, wheel, pos): response = self.client.do_move(wheel, pos)
20. The User Interface • The user interface may be changed without changing the underlying controller. • We have built: ◦ CLIs (used for development, diagnostics, testing) ◦ PyQt GUI ◦ Flask-based web interface (allows remote operability) ◦ A scriptable interface (allows offline, robotic control)
21. New and updated instruments • Spectrograph • Wide-field camera • SHOC (high speed camera)
22. The SHOC instrument • High speed, accurately timed camera • Camera able to take exposures every few milliseconds • GPS unit provides timing pulses with microsecond accuracy • Usually used in conjunction with a filter wheel
23. Picture of SHOC
24. Old SHOC: shortcomings • Existing software was: ◦ a homegrown LabView app to control filter wheels ◦ a proprietary Windows app to control the GPS ◦ a proprietary Windows app to control the camera • Three separate user interfaces • No easy way to navigate between them • NB: there is no way to add filter and timing info into the FITS headers (except by hand)
25. The new SHOC package: • A single interface, with multiple tabs • Runs on Linux! • Filter wheel driver: Python (using pyserial). • GPS driver: same as above. • Camera driver: C++ (‘cos that’s what the hardware manufacturer provides for the API). • FITS headers now include filter and timing info. • In the future: will include RA, DEC, wind speed, humidity, seeing, etc etc.
26. Example: occultation of a minor planet
27. Occultation data Full occultation of Charon, followed by grazing occultation of Pluto’s atmosphere.
28. Where else is Python used? Many small packages to solve specific problems. • Under development - a tool for characterising CCDs by generating photon transfer curves. • Data reduction pipeline • The new observatory-wide time-service
29. SHOC Web Interface
30. Overview • Goals and Requirements • Architecture • Dependencies • Design • Demo
31. Goals and Requirements • Remotely accessible (from a browser) • Fast and responsive • Reliable • Not too resource intensive
32. Architecture
33. Dependencies • Python • Flask • Gunicorn • Backbone.js • Bootstrap
34. What is Flask? • Micro framework • Combines Werkzeug and Jinja • Routes URLs to code
35. from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return 'Hello World!' if __name__ == '__main__': app.run() Hello World
36. Design v1 • JSONRPC for communication between web browser and web server • Seemed like a natural fit at first • But becomes messy quickly and harder to maintain over time
37. Design v2 • A REST-like interface for the web browser to communicate with the web server • Backbone.js to logically separate functionality • Simplifies client code at the cost of adding some complexity on the server-side
38. RPC vs REST POST /api HTTP/1.1 { "method": "Filterwheel.move", "params": { "wheel": "A", "position": 8 }, "jsonrpc": "2.0", "id": 1 } PATCH /instruments/wheels/A HTTP/1.1 { "position": 8 }
39. class Filterwheel(object): def __init__(self, wheel_id): self.wheel_id = wheel_id @property def position(self): return shoc.filterwheel.position @position.setter def position(self, position): shoc.filterwheel.do_move(self.wheel_id, position) Mapping Data to Methods
40. Python api = Blueprint('filterwheel.api', __name__) @api.route('/wheels/<wheel_id>', methods=['PATCH']) def patch(wheel_id): filterwheel = Filterwheel(wheel_id) for key, value in request.json.iteritems(): setattr(filterwheel, key, value) app.register_blueprint(api, url_prefix='/instruments')
41. Javascript filterwheel.collection.get('A').save({ position: 8 }); PATCH /instruments/wheels/A HTTP/1.1 { "position": 8 }
42. Demo
52. Thanks for listening. Questions? www.saao.ac.za shoc.saao.ac.za www.salt.ac.za