Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Chapter 8. Partial updates and retrievals

Chapter 8. Partial updates and retrievals

API Design Patterns

Part 3 Fundamentals

Avatar for Rick Hwang

Rick Hwang

April 18, 2023
Tweet

More Decks by Rick Hwang

Other Decks in Programming

Transcript

  1. Part 3 Fundamentals 8. Partial Updates and Retrievals Rick Hwang

    2022/04/20 1 • Why we might want to update or retrieve only specific pieces of a resource ◦ 鄉民都只對特殊資訊感興趣 • How best to communicate the fields of interest to the API service ◦ 鄉民都只對自己有利的感興趣 • How to handle individual items in complex fields, such as maps and interfaces • Whether to support addressing individual items in a repeated field (e.g., arrays) • Defining default values and handling implicit field masks • How to deal with invalid field specifications
  2. 2 • Why we might want to update or retrieve

    only specific pieces of a resource ◦ 鄉民都只對特殊資訊感興趣 • How best to communicate the fields of interest to the API service ◦ 鄉民都只對自己有利的感興趣 • How to handle individual items in complex fields, such as maps and interfaces • Whether to support addressing individual items in a repeated field (e.g., arrays) • Defining default values and handling implicit field masks • How to deal with invalid field specifications
  3. 8.1.1 Partial retrieval ◦ 取的整個 Resource 是必要的? ◦ 如果 Resource

    的資料結構有數百的 fields or subfields? ◦ 需要針對特殊的情境,限制取得的 fields,例如: ◦ 很小的 IoT 設備 - 硬體資源限制 ◦ PII - 資安政策 (Member Service) ◦ 不是所有 Client 都需要取得全部內容 8.1 Motivation 8.1.2 Partial update • 能否針對個別的 fields 修改? • 確保資料更新後的一致性 • 標準方法不夠用的時候怎麼辦? 3
  4. 5 Protocol Data Structure Web Services HTTP/1.1 SOAP / XML

    REST HTTP/1.1 JSON gRPC HTTP/2 Protobuf 補 充
  5. 7 1. Resource 新增 description field,也提供 新的 client 2. 舊的

    client 持續在使用 => 想像是 HTTP Client or SDK 那種 3. 沒有考慮 back compatible 就會出現 data loss 1 2 3
  6. 1. 循序性的寫入,需要一致性檢查 (consistency checks)、或者 locking 2. versioning and compatiablity, see

    chatper 24 a. support new features b. fix bugs c. to add new fields to resources while still considering the new version backward compatible 核心問題 8
  7. 1. Enabling partial retrievals => GET 2. Enabling partial updates

    => PATCH 8.2 Overview - Goals 9 a field mask is just a collection of strings, but these strings represent a list of fields that we’re interested in on a given resource.
  8. 11

  9. 12

  10. 8.3 Implementation How we transport the field mask without causing

    any significant disruption to the standard requests that defined in chapter 7. 13
  11. 8.2.1 Transport two constraints: 1. GET: no body to the

    request permitted (and many HTTP servers will strip it out if one is provided) 2. PATCH: resource-oriented design dictates that the body of a PATCH request must be the resource representation being updated two potential places: 1. headers 2. query strings: how the repeated query string parameters are interpreted will depend on the HTTP server in use 14
  12. 16 HTTP Methods Query String Body HTTP Header GET v

    v (?) v PATCH v v v Node.JS (express) HTTP Methods Query String Body HTTP Header GET v ?? v PATCH v ASP.NET (Kestrel)
  13. have to augment the request messages for both the standard

    update method and standard get method. 18
  14. Does this idea of partial updates and retrievals extend inside

    nested structures? Or does it only apply at the very top level, effectively treating resources as completely flat structures? 8.3.2 Maps and nested interface 19
  15. 1. Separate parts of a field specification must use a

    dot character (.) as a separator. 2. All fields of a nested message may be referred to using an asterisk character (*). 3. Map keys should always be strings. 4. All parts of a field specification that can’t be represented as an unquoted string literal must be quoted using backtick characters (`). 5. A literal backtick character may be escaped by using two backtick characters (``) Rules for field mask to specify nested fields. 21
  16. 23

  17. 8.3.3 Repeated fields • prefix of "administrators.*." as a way

    of saying “For each administrator • retrieve only the name of an administrator, use a field mask value of "administrators.*.name" 24
  18. 8.3.4 Default values the goal of a default value is

    to do “the right thing” for the user. GET: standard get method, the default is almost always the complete list of fields available on a resource. 標準 GET 會取得所有的 Fields There is an exception to this guideline. In cases where a resource has fields which, for whatever reason, would cause a fundamentally worse user experience for API consumers, these fields should be removed from the default of the field mask being left unset. 如果有一些 Fields 會導致 API 使用者的使用體驗不好,這些 fields 應該從 field mask 的預設中 被移除 使用者有興趣的時候,自行透過 fieldMask 指定 Fields 取得。 25
  19. GET /api/chatrooms?fieldMask=participants { "1": { "id": 1, "title": "My Chat!",

    "description": "喇低賽", "participants": [] ⇐ 裡面有 10k 個鄉民,要處理很久 }, "2": { "id": 2, "title": "Your Chat!", "description": "低賽喇", "participants": [] ⇐ 裡面有 100k 個鄉民,要處理很久 } } Design: w/o participants in default GET /api/chatrooms <Default> { "1": { "id": 1, "title": "My Chat!", "description": "喇低賽", }, "2": { "id": 2, "title": "Your Chat!", "description": "低賽喇", } } 26 w/o participants
  20. 預設 = 製造了淺規則? • 文件要明確說明,預設 fieldMask 的行為 • 即使有文件,還是很容易誤解。 •

    不想讓使用者猜測哪些 fields 是預設的? • 使用 asterick (“*”) 表示返回所有的 fields • 取得資源使用預設 fieldMask 是用,但是更 新資源,Default FieldMask 就沒啥用 GET /api/chatrooms?fieldMask=* { "1": { "id": 1, "title": "My Chat!", "description": "喇低賽", "participants": [] ⇐ 裡面有 10k 個鄉民,要處理很久 } } GET /api/chatrooms?fieldMask=id,title { "1": { "id": 1, "title": "My Chat!", } } 27 個 人 觀 點 DesignA: 特定欄位的正向表列? Design B: 全部欄位的正向表列?
  21. 預設 = Policy (政策 / 慣例 / 文化 / 潛規則)

    • 衝突來源 • 怪異 • 高雄式左轉 • 習慣遲到 • 公司的淺規則 • 默契 • 共識 • 守法 • 提前到、準時 • 91APP 的政策 28 個 人 觀 點
  22. 預設 • 專業領域 (Domain Know How) • 企業文化 • 鄉民文化

    • 潛在共識 • 生態系 ◦ .NET method name starts with uppercase ◦ Java method name starts with lowercase 29 個 人 觀 點
  23. The goal of a default value is to do “the

    right thing” for the user. 啥叫做正確的事情? ← 這句話不夠具體 • 給不同使用者,最必要的資訊。 ◦ Chatroom 的參與者,需要怎樣的聊天室資訊? ◦ Chatroom 的管理者,需要怎樣的聊天室資訊? • 91APP 的專業判斷 => Domain Expert 必須思考的 => PO + Team ◦ 對於 會員 (消費者) 而言,訂單 (Order) 要呈現什麼資訊? ◦ 對於 91APP 的客戶 而言,訂單 (Order) 要呈現什麼資訊? ◦ 對於 91APP 營管 而言,訂單 (Order) 要呈現什麼資訊? ◦ 對於 91APP 合作夥伴 而言,訂單 (Order) 要呈現什麼資訊? 30 個 人 觀 點
  24. What does an unset field mask mean on a standard

    update method (using an HTTP PATCH method)? 8.3.5 Implicit field masks 31
  25. 32

  26. What does an unset field mask mean on a standard

    update method (using an HTTP PATCH method)? Most commonly, HTTP PATCH indicates an intent to update the resource with only the data provided in the body of the request. 8.3.5 Implicit field masks 33
  27. • dynamic data structure => object 型別 ◦ ch5 提到的是

    static data structure • settings.test 有三種值: ◦ “STRING_VALUE” ◦ null ◦ 不存在 • 如果要刪除 settings.test 要怎麼做? ◦ 不是 null ◦ “undefined” 不是 JSON 資料型別 註:“undefined” 是語言層級的型別。 8.3.6 Updating dynamic data structures 35
  28. 36

  29. 8.3.6 Updating dynamic data structures • settings.test 有三種值: ◦ “STRING_VALUE”

    ◦ null ◦ 不存在 => undefined 的意思 • 如果要刪除 settings.test 要怎麼做? ◦ 不是 null • “undefined” 不是 JSON 資料型別,是 JS 語言層級的型別。 ◦ python 可能是 None 37
  30. • the goal is still quite limited: a. to minimize

    unnecessary data transfer and b. allow fine-grained modification of API resources. • consider field masks and partial retrievals in particular as SQL-like querying tools to fetch specific data 8.4 Trade-offs 41
  31. 8.4.1 Universal support 44 • NOT at all a requirement

    that every API must support partial retrieval. • more an issue of concurrency than resource size and complexity
  32. 8.4.2 Alternative implementations • JSON Patch (RFC6902), (npm) • JSON

    Merge Patch (RFC7396), (npm) • 哪裡有類似的概念:K8s 實作 ◦ kubectl apply -f xxx.yaml 46
  33. 1. Partial retrieval is particularly important in cases where resources

    are large or clients consuming resource data have limited hardware. => POS 2. Partial updates are critical for fine-grained updates without worrying about conflicts. 3. Field masks, which support ways to address fields, nested fields in interfaces, and map keys, should be used to indicate the fields that should be retrieved or updated. 4. Field masks should NOT provide a mechanism to address items in array fields by their position or index in that field. 5. By default, field masks should assume a value of everything for partial retrievals and an implicit field mask (8.3.5) for partial updates. 6. If fields are invalid, they should be treated as though they do exist but have a value of undefined. Summary 47