Slide 1

Slide 1 text

STANDARDIZING OUR DRIVERS THROUGH SPECS: A LOOK AT THE CRUD API OPENING TUESDAY, JUNE 2ND IN SELECT CONFERENCE ROOMS. Jeremy Mikola jmikola

Slide 2

Slide 2 text

What’s the deal with specs?

Slide 3

Slide 3 text

A Cursory Introduction SHAMELESSLY BORROWED FROM…

Slide 4

Slide 4 text

MongoDB has a lot of drivers…

Slide 5

Slide 5 text

Drivers Today APIs evolved over many years Idiomatic for their language Inconsistent with each other Subtle behavioral differences Our support team loves this!

Slide 6

Slide 6 text

Specifications Authentication SDAM, server selection Wire protocol, write commands User-facing APIs CRUD, enumeration, $ o u t

Slide 7

Slide 7 text

End Goals Consistency across drivers Behavior and API More intuitive documentation Increased developer productivity Guidance for third-party drivers

Slide 8

Slide 8 text

End Goals (Continued) Mitigate Parkinson’s law of triviality

Slide 9

Slide 9 text

Available on GitHub Internal WIPs → Released Specs See: mongodb/specifications Mailing lists are still in vogue: and mongodb-drivers mongodb-dev

Slide 10

Slide 10 text

CRUD Specification

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

What are we addressing? Collection-level read/write methods. Variations in naming and semantics. Inconsistent functionality and APIs.

Slide 13

Slide 13 text

Default Behaviors (Exhibit A) / / U p d a t e s o n e d o c u m e n t d b . t h i n g s . u p d a t e ( { " t y p e " : " h u m a n " } , { " $ s e t " : { " o r g a n i c " : t r u e } } ) ; / / R e m o v e s a l l m a t c h i n g d o c u m e n t s d b . t h i n g s . r e m o v e ( { " t y p e " : " r o b o t " } ) ;

Slide 14

Slide 14 text

Option Names (Exhibit B) / / U p d a t e s a l l d o c u m e n t s d b . t h i n g s . u p d a t e ( { " t y p e " : " h u m a n " } , { " $ s e t " : { " o r g a n i c " : t r u e } } { " m u l t i " : t r u e } ) ; / / R e m o v e s o n e m a t c h i n g d o c u m e n t d b . t h i n g s . r e m o v e ( { " t y p e " : " r o b o t " } , { " j u s t O n e " : t r u e } ) ;

Slide 15

Slide 15 text

Method Signatures (Exhibit C) / / A c c o r d i n g t o t h e d o c u m e n t a t i o n f u n c t i o n ( q u e r y , u p d a t e , o p t i o n s ) { / / . . . } / / A c t u a l i m p l e m e n t a t i o n f u n c t i o n ( q u e r y , u p d a t e , u p s e r t , m u l t i ) { i f ( t y p e o f ( u p s e r t ) = = = " o b j e c t " ) { / / U n p a c k o p t i o n s . . . } / / . . . } Legacy baggage with u p d a t e ( )

Slide 16

Slide 16 text

CRUD Create Read Update Delete i n s e r t ( ) f i n d ( ) u p d a t e ( ) r e m o v e ( )

Slide 17

Slide 17 text

But Wait… There’s More!

Slide 18

Slide 18 text

CRUDCRADBOoMFaM Create Read Update Delete Count Replace Aggregate Distinct Bulk, One or Many Find and Modify

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Our entirely reasonable and level-headed approach… BEAR WITH ME.

Slide 21

Slide 21 text

Terminology Collection: class or interface representing a collection. Spec defines methods to be implemented on this object. Iterable: some iterable object or structure. Cursor or cursor-like abstraction for reads. Vector or array for write method arguments.

Slide 22

Slide 22 text

Operations Methods to be implemented on the Collection object. These have required and optional parameters.

Slide 23

Slide 23 text

Deviations Spec is flexible with naming and option handling. This permits idiomaticity. (real word) ☜

Slide 24

Slide 24 text

Naming Deviations “Root” words are non-negotiable. b a t c h S i z e , b a t c h _ s i z e ʕ•ᴥ•ʔ b a t c h C o u n t щ(゚Д゚щ) m a x T i m e M S , m a x T i m e ͡ ° ͜ ʖ ͡ ° m a x i m u m T i m e ? (╯°□°)╯︵ ┻━┻ F i n d O p t i o n s , F i n d A r g s ﺡ(゚ヮ゚)ノ Q u e r y P a r a m s ? (ノಠ益ಠ)ノ彡┻━┻ o r d e r e d , i s O r d e r e d ᕕ( ᐛ )ᕗ

Slide 25

Slide 25 text

Option Handling Required options are positional arguments. For optional options, you have some options: Named parameters Dictionary or hash objects Option classes (e.g. U p d a t e O p t i o n s ) May be consolidated, shared Fluent builders for f i n d ( ) , a g g r e g a t e ( ) Document when order of operations applies!

Slide 26

Slide 26 text

What we’re not doing Method overloading. Encapsulating params in “Model” classes. Codifying inconsistent APIs for BC.

Slide 27

Slide 27 text

Diving into the API Chapter 1: Reads

Slide 28

Slide 28 text

Querying f i n d ( f i l t e r : D o c u m e n t , o p t i o n s : F i n d O p t i o n s ) : I t e r a b l e < D o c u m e n t > ; f i l t e r is criteria (i.e. $ q u e r y meta operator). Support other operators through options. Iterable is obviously a cursor here.

Slide 29

Slide 29 text

FindOptions a l l o w P a r t i a l R e s u l t s : B o o l e a n b a t c h S i z e : I n t 3 2 c o m m e n t : S t r i n g c u r s o r T y p e : C u r s o r T y p e l i m i t : I n t 3 2 m a x T i m e M S : I n t 6 4 m o d i f i e r s : D o c u m e n t n o C u r s o r T i m e o u t : B o o l e a n o p l o g R e p l a y : B o o l e a n p r o j e c t i o n : D o c u m e n t s k i p : I n t 3 2 s o r t : D o c u m e n t

Slide 30

Slide 30 text

Abstracting Internal Details C u r s o r T y p e enum may be normal, tailable, or tailable and awaitable. Today’s wire protocol flags are tomorrow’s command options. Users shouldn’t care and it’s not worth future API breaks.

Slide 31

Slide 31 text

Other Read Methods a g g r e g a t e ( p i p e l i n e : D o c u m e n t [ ] , o p t i o n s : A g g r e g a t e O p t i o n s ) : I t e r a b l e < D o c u m e n t > ; c o u n t ( f i l t e r : D o c u m e n t , o p t i o n s : C o u n t O p t i o n s ) : I n t 6 4 ; d i s t i n c t ( f i e l d N a m e : s t r i n g , f i l t e r : D o c u m e n t , o p t i o n s : D i s t i n c t O p t i o n s ) : I t e r a b l e < a n y > ;

Slide 32

Slide 32 text

AggregateOptions a l l o w D i s k U s e : B o o l e a n b a t c h S i z e : I n t 3 2 m a x T i m e M S : I n t 6 4 u s e C u r s o r : B o o l e a n u s e C u r s o r default varies by server version. May affect the kind of Iterable returned.

Slide 33

Slide 33 text

Diving into the API Chapter 2: Writes We’re basically 50% done at this point…

Slide 34

Slide 34 text

Write Methods i n s e r t O n e ( d o c u m e n t : D o c u m e n t ) : I n s e r t O n e R e s u l t ; i n s e r t M a n y ( I t e r a b l e < D o c u m e n t > d o c u m e n t s , o p t i o n s : I n s e r t M a n y O p t i o n s ) : I n s e r t M a n y R e s u l t ; d e l e t e O n e ( f i l t e r : D o c u m e n t ) : D e l e t e R e s u l t ; d e l e t e M a n y ( f i l t e r : D o c u m e n t ) : D e l e t e R e s u l t ; One or many behavior is explicit and facilitates self-documenting code. Eliminates inconsistency between m u l t i and j u s t O n e defaults for u p d a t e ( ) and r e m o v e ( ) , respectively.

Slide 35

Slide 35 text

i n s e r t M a n y ( ) i n s e r t M a n y is syntactic sugar for b u l k W r i t e ( ) with inserts. Spec doesn’t address legacy batch O P _ I N S E R T operations.

Slide 36

Slide 36 text

InsertManyOptions o r d e r e d : B o o l e a n

Slide 37

Slide 37 text

Write Methods (Continued) r e p l a c e O n e ( f i l t e r : D o c u m e n t , r e p l a c e m e n t : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; u p d a t e O n e ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; u p d a t e M a n y ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; Same points about explicit, self-documenting code apply. Trivial to validate if replacement or update documents contain operators.

Slide 38

Slide 38 text

UpdateOptions u p s e r t : B o o l e a n

Slide 39

Slide 39 text

Bulk Writes b u l k W r i t e ( r e q u e s t s : W r i t e M o d e l [ ] , o p t i o n s : B u l k W r i t e O p t i o n s ) : B u l k W r i t e R e s u l t ; Remember i n i t i a l i z e O r d e r e d B u l k O p ( ) , or the really old fluent API from 2013?

Slide 40

Slide 40 text

WriteModel Also, remember when we said we weren’t doing “model” classes that encapsulate all arguments?

Slide 41

Slide 41 text

WriteModel Models include required and optional (if any) arguments from single write methods. b u l k W r i t e ( ) ’s r e q u e s t s argument allows users to specify all of their writes at once and in a single method call. Trivial to make a fluent API atop this, although it’s not in the spec.

Slide 42

Slide 42 text

UpdateManyModel (f.e.) f i l t e r : D o c u m e n t r e q u i r e d u p d a t e : D o c u m e n t r e q u i r e d u p s e r t : B o o l e a n o p t i o n a l

Slide 43

Slide 43 text

BulkWriteOptions o r d e r e d : B o o l e a n Basically the same thing as InsertManyOptions.

Slide 44

Slide 44 text

Result Classes Insert results may report driver-generated IDs Delete results include counts Update results include counts and server-generated IDs from upserts Bulk results aggregate all of the above Results are optional for unacknowledged writes. (e.g. O p t i o n a l < B u l k W r i t e R e s u l t > , i s A c k n o w l e d g e d boolean) Since all fields within insert results are optional, i n s e r t O n e ( ) and i n s e r t M a n y ( ) may be void!

Slide 45

Slide 45 text

Write Errors Spec is flexible on how errors are reported Mainly concerned that info is accessible under consistent fields and names. Doesn’t address merging errors We have a bulk write spec for that

Slide 46

Slide 46 text

Diving into the API Chapter 3: Find and Modify I’ll keep this short…

Slide 47

Slide 47 text

Find and Modify Methods f i n d O n e A n d D e l e t e ( f i l t e r : D o c u m e n t , o p t i o n s : F i n d O n e A n d D e l e t e O p t i o n s ) : D o c u m e n t ; f i n d O n e A n d R e p l a c e ( f i l t e r : D o c u m e n t , r e p l a c e m e n t : D o c u m e n t , o p t i o n s : F i n d O n e A n d R e p l a c e O p t i o n s ) : D o c u m e n t ; f i n d O n e A n d U p d a t e ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : F i n d O n e A n d U p d a t e O p t i o n s ) : D o c u m e n t ; Option classes contain only the relevant command options.

Slide 48

Slide 48 text

Preemptive Q&A

Slide 49

Slide 49 text

Read Preferences? Generally, queries on same collection will use the same read preference. Spec assumes it’s set on client, database, or collection. Per-operation read preferences are permitted; the spec simply doesn’t define it.

Slide 50

Slide 50 text

Write Concerns? Everything we just said about read preferences…

Slide 51

Slide 51 text

f i n d O n e ( ) et al. Drivers are free to keep existing methods and add new ones, too. Please keep options and naming consistent.

Slide 52

Slide 52 text

Thanks!

Slide 53

Slide 53 text

FIN. Questions?

Slide 54

Slide 54 text

Image Credits and http://www.instructables.com/id/Egg-Cream/ Giphy Reaction Gifs https://octodex.github.com/wheres-waldocat/