Rails meets Protocol Buffers - for Schema First Development

Rails Developers Meetup 2019
SESSION: B-8 (ja) @aeroastro
https://railsdm.github.io/#session-aeroastro

When separated frontend team and backend team cooperate on the same project, they sometimes have difficulties in correctly and efficiently sharing the API information, the interface between the teams. This is the case not only in web application development but also in game backend development.

In this session, the effectiveness of Schema First Development will be presented. Also, as an example of its application in game development, we will take a look at Rails with Protocol Buffers.

-------

フロントエンドとバックエンドのチームが分かれて開発する場合、チーム間のインターフェースであるAPIの情報共有は、正確性や効率性の点で難しい場合があります。そして、一般的なWebアプリケーションの開発だけではなく、ゲームのバックエンドの開発においても同様の課題が存在します。

このセッションでは、このような課題に対してのスキーマファースト開発の有用性、および、ゲーム開発への適用の一例として、Rails と Protocol Buffers を組み合わせた開発手法を紹介します。

Transcript

2. • ◦ ◦ • ◦ • ◦ ▪ ◦ ▪ ◦ ▪ ◦ …
3. • ◦ • ◦ ◦ ◦ • ◦ ▪ ▪ …
4. • • • • • • • •
5. • • • • • • • •
6. • • ◦ ◦ *
7. • ◦ ◦ • ◦ ◦ ◦ ◦ • ◦ • ◦ ◦ ◦ def create(title:) issue = Issue.create!(title: title) render_result( issue: issue_to_resource(issue)) end
8. • • • • • • • •
9. Development Timeline Backend Frontend
10. Development Timeline Backend Frontend
11. Development Timeline Backend Frontend
12. Development Timeline Backend Frontend
13. • ◦ ◦ …… • ◦ ◦ … • ◦ ◦ ◦
14. • • •
15. • A representation of a plan or theory in the form of an outline or model. ◦ by English Oxford Living Dictionaries In API, • • •
16. • ◦ Schema for RESTful Web Services • ◦ • ◦ Query language for your API
17. 1. ◦ 2. ◦ 3. ◦ 4. ◦ 5. ◦ ◦ ◦ ◦
18. Development Timeline Backend Frontend
19. • • • • • • • •
20. In game development context, what we chose in the 2010s is...
21. In game development context, what we chose in the 2010s is... Protocol Buffers with In-House Schema and Ruby on Rails Many kinds of schema are employed. This is 1 example.
22. • Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data message Issue { optional string title = 1; optional sint64 created_at = 2; }
23. • Created in the project to generate Client SDK automatically. • Support ◦ I/F to Server API ◦ I/F to Client Code ◦ Internal Glue Code • Not all, but a significant portion of SDK url: /v1/teams/:team_id/issues http_method: POST method_name: createIssue params: - name: team_id protobuf_type: string protobuf_rule: optional param_type: path - name: title protobuf_type: string protobuf_rule: optional param_type: body
24. • • • • • • • •
25. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic Server Client
26. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
27. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
28. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
29. Imperfect Schema Communication Cost Implementation Cost
30. The final objective of Schema First Development is not to make schema perfect but to make products better.
31. Body Schema by Protocol Buffers Protobuf I/F (Body) Protobuf I/F (Body) • ◦ • ◦ ◦ • ◦ ◦ Product is the top priority.
32. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
33. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
34. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
35. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
36. Body Schema by Protocol Buffers RESTful API (Path/Query) Protobuf I/F (Body) Developer Friendly Client SDK I/F In-House Schema for SDK Auto-Gen Server Logic Protobuf I/F (Body) RESTful Client (Path/Query) SDK Internal Glue Code Client Logic
37. Let’s dive deeper into the Rails and Protocol Buffers.
38. • • • • • • • •
39. • • syntax = 'proto2'; message Request { optional string title = 1; } message Response { optional string title = 1; optional sint64 created_at = 2; } { "$schema": "http://json-schema.org/draft-04/schema" , "type": "object", "properties" : { "request": { "type": "object", "properties" : { "title": { "type": "string" } } }, "response" : { "type": "object", "properties" : { "title": { "type": "string" }, "created_at" : { "type": "integer" } } } } } • Concise Syntax ◦ Simpler and less ambiguous • Easy to Learn ◦ You can familiarize yourself in less than 1 hour
40. • • syntax = 'proto2'; message Response { optional string title = 1; optional sint64 created_at = 2; } Response.new( id: 'Ruby on Rails', created_at: 1553006492 ).to_s # => "\n\rRuby on Rails\x10\xB8\xFE\x87\xC9\v" • Small Memory Footprint ◦ Field number instead of key string ( a_very_very_long_key does not affect performance) • Natural Representation of Binary and Number Data ◦ No base64 encoding required ◦ Natural for Game Development • Freedom for Code Refactoring ◦ No naming issue in wire format (only field number) ◦ Free to insert / ignore new fields. • High Serialization / Deserialization Performance ◦ Protocol Buffers LITE is Faster than Rapid JSON in C++* *
41. • • • Natural Extension of Conventional JSON API ◦ A matter of accept and content-type ◦ You can use conventional knowledge • Not Human-Friendly ◦ You cannot easily read/write the body ◦ Some countermeasures are required.
42. • • • • • • • •
43. • ◦ • ◦ • ◦ • ◦
44. • ◦ • ◦ ◦ • ◦ • ◦ Mime::Type.register 'application/x-protobuf', :protobuf ActionDispatch::Request.parameter_parsers = {} before_action do case request.content_mime_type when :protobuf proto_data = request.raw_post when :json raw_hash = JSON.parse(request.raw_post) proto_data = proto_class.build(raw_hash) end proto_class.parse(proto_data).to_hash end
45. • ◦ • ◦ • ◦ ActionController::Renderers.add :protobuf do |object, options| options[:proto_class].build(object).to_s end ActionController::Renderers.add :proto_json do |object, options| options[:proto_class].build(object).to_json end respond_to do |format| format.protobuf do render( protobuf: response_params, protobuf_class: proto_class ) end format.json do render( proto_json: response_params, protobuf_class: proto_class ) end end
46. • ◦ • ◦ • ◦ ◦ ◦ filter_response_resource_fields :issue def create(title:) issue = Issue.create!(title: title) render_result( issue: issue_to_resource(issue) ) end *
47. • ◦ • ◦ • ◦ ◦ ◦ Simple and Performant Architecture We are still on the Rails * filter_response_resource_fields :issue def create(title:) issue = Issue.create!(title: title) render_result( issue: issue_to_resource(issue) ) end
48. • • • • • • • •
49. • ◦ ◦ ◦ • ◦ ◦ ◦ ◦ • ◦ • ◦ ◦ ◦ Overall, Schema First Development is really great.
50. • ◦ ◦ ◦ • ◦ ◦ ◦ ◦ • ◦ • ◦ ◦ ◦ Overall, Schema First Development is really great. However, this is just the beginning towards the Ideal Game Development
51. • ◦ ◦ • ◦ ◦ • ◦ ◦ • ◦ ◦ ◦
52. • • • • • • • •
53. • • • • •
55. One more thing...
56. We’re hiring! • Professionals Who Love ◦ Game Development ◦ Scaling Architecture ◦ Performance Tuning ◦ Reliability Engineering ◦ Development Kaizen
58. Let’s make products better with Schema First Development Rails meets Protocol Buffers for Schema First Development