The Graphyte Project: design, tools and practices in building a collaborative scientific computation and education system

Offical talk recording and description:
https://sea.ucar.edu/event/graphyte-project-design-tools-and-practices-building-collaborative-scientific-computation-and-

Alternatively, you can view it on Youtube:
http://www.youtube.com/watch?v=eG7Pdf94JY8

Transcript

1. The Graphyte Project The Graphyte Project design, tools and practices design, tools and practices Speaker: Yeukhon Wong Speaker: Yeukhon Wong Michael Grossberg, Irina Gladkova, Michael Grossberg, Irina Gladkova, Jeremy Neiman, Hannah Aizenman Jeremy Neiman, Hannah Aizenman GLASS Lab, CUNY City GLASS Lab, CUNY City April 1 April 1st st, 2013 , 2013 SEA Software Engineering Conference 2013 SEA Software Engineering Conference 2013
3.  Computation  Data  Code  Data  Code  Computation Traditional Workflow
4.  Computation  Code  Data  Code Ideal Workflow
5. API API Layer Layer Core Web Service Core Web Service (GCS) (GCS) Graphyte Architecture Graphyte Architecture
6. API API Layer Layer Data Store Data Store Core Web Service Core Web Service (GCS) (GCS) Graphyte Architecture Graphyte Architecture
7. API API Layer Layer Data Store Data Store Version Control Host Version Control Host Core Web Service Core Web Service (GCS) (GCS) Graphyte Architecture Graphyte Architecture
8. API API Layer Layer Data Store Data Store Version Control Host Version Control Host Web UI Web UI Core Web Service Core Web Service (GCS) (GCS) Graphyte Architecture Graphyte Architecture
9. Web UI: Result HIstory Web UI: Result HIstory
10. Web UI: Web UI: Result Result Details Details
12. Aurum • A RESTful web service built using Graphyte API and infrastructure. • Use automated tests to evaluate learning progress.
13. Aurum on Blackboard Aurum on Blackboard
15. Student Student Views: Views: automated automated assessment assessment
17. Lessons
18. Good: Modular Architecture
19. Bad: Monolithic Implementation
20. Fix: Enforce modularity from beginning
21. Good: Web service-centric
22. Bad: Inconsistent API
23. Fix: RESTful Patterns
24. Good: Automated tests
25. Bad: Tests take >1.5hrs
26. Fix: Write cheap tests Unit tests Unit tests (ultra fast, 100% isolation) (ultra fast, 100% isolation) Integration tests Integration tests (tolerable, b/w modules) (tolerable, b/w modules) Functional tests Functional tests (slow, end-to-end) (slow, end-to-end)
27. Even for prototyping, use these good practices.
28. Packaging Servers Packaging Servers
29. Good: Had instructions to configure machines.
30. Bad: No automation, outdated Wiki
31. Fix: Use automated deployment tool
32. Many tools, which to use? Many tools, which to use?
33. What works for us What works for us http://ansible.cc/ Ansible Configure servers with SSH.
35. 1. More tests, more coverage.
36. 2. Write user and developer documentation.
37. 3. Bring back continuous integration build and test.
38. 4. We want VM with roles. (database servers, web servers, cache server)
39. 5. Open-source Graphyte!
40. Contributors Contributors: Jeremy Neiman, Hannah Aizenman, Luis Bello, Yuriy Steijko, Paul K. Alabi, Joesan Gabaldon, Michael Grossberg, Irina Gladkova
41. Questions? Questions? yeukhon@acm.org yeukhon@acm.org glasslab.org glasslab.org