STANDARDIZING OUR DRIVERS THROUGH SPECS: A LOOK AT THE CRUD API OPENING TUESDAY, JUNE 2ND IN SELECT CONFERENCE ROOMS. Jeremy Mikola jmikola

What’s the deal with specs?

A Cursory Introduction SHAMELESSLY BORROWED FROM…

MongoDB has a lot of drivers…

Drivers Today APIs evolved over many years Idiomatic for their language Inconsistent with each other Subtle behavioral differences Our support team loves this!

Specifications Authentication SDAM, server selection Wire protocol, write commands User-facing APIs CRUD, enumeration, $ o u t

End Goals Consistency across drivers Behavior and API More intuitive documentation Increased developer productivity Guidance for third-party drivers

End Goals (Continued) Mitigate Parkinson’s law of triviality

Available on GitHub Internal WIPs → Released Specs See: mongodb/specifications Mailing lists are still in vogue: and mongodb-drivers mongodb-dev

CRUD Specification

No content

What are we addressing? Collection-level read/write methods. Variations in naming and semantics. Inconsistent functionality and APIs.

Default Behaviors (Exhibit A) / / U p d a t e s o n e d o c u m e n t d b . t h i n g s . u p d a t e ( { " t y p e " : " h u m a n " } , { " $ s e t " : { " o r g a n i c " : t r u e } } ) ; / / R e m o v e s a l l m a t c h i n g d o c u m e n t s d b . t h i n g s . r e m o v e ( { " t y p e " : " r o b o t " } ) ;

Option Names (Exhibit B) / / U p d a t e s a l l d o c u m e n t s d b . t h i n g s . u p d a t e ( { " t y p e " : " h u m a n " } , { " $ s e t " : { " o r g a n i c " : t r u e } } { " m u l t i " : t r u e } ) ; / / R e m o v e s o n e m a t c h i n g d o c u m e n t d b . t h i n g s . r e m o v e ( { " t y p e " : " r o b o t " } , { " j u s t O n e " : t r u e } ) ;

Method Signatures (Exhibit C) / / A c c o r d i n g t o t h e d o c u m e n t a t i o n f u n c t i o n ( q u e r y , u p d a t e , o p t i o n s ) { / / . . . } / / A c t u a l i m p l e m e n t a t i o n f u n c t i o n ( q u e r y , u p d a t e , u p s e r t , m u l t i ) { i f ( t y p e o f ( u p s e r t ) = = = " o b j e c t " ) { / / U n p a c k o p t i o n s . . . } / / . . . } Legacy baggage with u p d a t e ( )

CRUD Create Read Update Delete i n s e r t ( ) f i n d ( ) u p d a t e ( ) r e m o v e ( )

But Wait… There’s More!

CRUDCRADBOoMFaM Create Read Update Delete Count Replace Aggregate Distinct Bulk, One or Many Find and Modify

No content

Our entirely reasonable and level-headed approach… BEAR WITH ME.

Terminology Collection: class or interface representing a collection. Spec defines methods to be implemented on this object. Iterable: some iterable object or structure. Cursor or cursor-like abstraction for reads. Vector or array for write method arguments.

Operations Methods to be implemented on the Collection object. These have required and optional parameters.

Deviations Spec is flexible with naming and option handling. This permits idiomaticity. (real word) ☜

Naming Deviations “Root” words are non-negotiable. b a t c h S i z e , b a t c h _ s i z e ʕ•ᴥ•ʔ b a t c h C o u n t щ（ﾟДﾟщ） m a x T i m e M S , m a x T i m e ͡ ° ͜ ʖ ͡ ° m a x i m u m T i m e ? (╯°□°）╯︵ ┻━┻ F i n d O p t i o n s , F i n d A r g s ﺡ(ﾟヮﾟ)ﾉ Q u e r y P a r a m s ? (ノಠ益ಠ)ノ彡┻━┻ o r d e r e d , i s O r d e r e d ᕕ( ᐛ )ᕗ

Option Handling Required options are positional arguments. For optional options, you have some options: Named parameters Dictionary or hash objects Option classes (e.g. U p d a t e O p t i o n s ) May be consolidated, shared Fluent builders for f i n d ( ) , a g g r e g a t e ( ) Document when order of operations applies!

What we’re not doing Method overloading. Encapsulating params in “Model” classes. Codifying inconsistent APIs for BC.

Diving into the API Chapter 1: Reads

Querying f i n d ( f i l t e r : D o c u m e n t , o p t i o n s : F i n d O p t i o n s ) : I t e r a b l e < D o c u m e n t > ; f i l t e r is criteria (i.e. $ q u e r y meta operator). Support other operators through options. Iterable is obviously a cursor here.

FindOptions a l l o w P a r t i a l R e s u l t s : B o o l e a n b a t c h S i z e : I n t 3 2 c o m m e n t : S t r i n g c u r s o r T y p e : C u r s o r T y p e l i m i t : I n t 3 2 m a x T i m e M S : I n t 6 4 m o d i f i e r s : D o c u m e n t n o C u r s o r T i m e o u t : B o o l e a n o p l o g R e p l a y : B o o l e a n p r o j e c t i o n : D o c u m e n t s k i p : I n t 3 2 s o r t : D o c u m e n t

Abstracting Internal Details C u r s o r T y p e enum may be normal, tailable, or tailable and awaitable. Today’s wire protocol flags are tomorrow’s command options. Users shouldn’t care and it’s not worth future API breaks.

Other Read Methods a g g r e g a t e ( p i p e l i n e : D o c u m e n t [ ] , o p t i o n s : A g g r e g a t e O p t i o n s ) : I t e r a b l e < D o c u m e n t > ; c o u n t ( f i l t e r : D o c u m e n t , o p t i o n s : C o u n t O p t i o n s ) : I n t 6 4 ; d i s t i n c t ( f i e l d N a m e : s t r i n g , f i l t e r : D o c u m e n t , o p t i o n s : D i s t i n c t O p t i o n s ) : I t e r a b l e < a n y > ;

AggregateOptions a l l o w D i s k U s e : B o o l e a n b a t c h S i z e : I n t 3 2 m a x T i m e M S : I n t 6 4 u s e C u r s o r : B o o l e a n u s e C u r s o r default varies by server version. May affect the kind of Iterable returned.

Diving into the API Chapter 2: Writes We’re basically 50% done at this point…

Write Methods i n s e r t O n e ( d o c u m e n t : D o c u m e n t ) : I n s e r t O n e R e s u l t ; i n s e r t M a n y ( I t e r a b l e < D o c u m e n t > d o c u m e n t s , o p t i o n s : I n s e r t M a n y O p t i o n s ) : I n s e r t M a n y R e s u l t ; d e l e t e O n e ( f i l t e r : D o c u m e n t ) : D e l e t e R e s u l t ; d e l e t e M a n y ( f i l t e r : D o c u m e n t ) : D e l e t e R e s u l t ; One or many behavior is explicit and facilitates self-documenting code. Eliminates inconsistency between m u l t i and j u s t O n e defaults for u p d a t e ( ) and r e m o v e ( ) , respectively.

i n s e r t M a n y ( ) i n s e r t M a n y is syntactic sugar for b u l k W r i t e ( ) with inserts. Spec doesn’t address legacy batch O P _ I N S E R T operations.

InsertManyOptions o r d e r e d : B o o l e a n

Write Methods (Continued) r e p l a c e O n e ( f i l t e r : D o c u m e n t , r e p l a c e m e n t : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; u p d a t e O n e ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; u p d a t e M a n y ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : U p d a t e O p t i o n s ) : U p d a t e R e s u l t ; Same points about explicit, self-documenting code apply. Trivial to validate if replacement or update documents contain operators.

UpdateOptions u p s e r t : B o o l e a n

Bulk Writes b u l k W r i t e ( r e q u e s t s : W r i t e M o d e l [ ] , o p t i o n s : B u l k W r i t e O p t i o n s ) : B u l k W r i t e R e s u l t ; Remember i n i t i a l i z e O r d e r e d B u l k O p ( ) , or the really old fluent API from 2013?

WriteModel Also, remember when we said we weren’t doing “model” classes that encapsulate all arguments?

WriteModel Models include required and optional (if any) arguments from single write methods. b u l k W r i t e ( ) ’s r e q u e s t s argument allows users to specify all of their writes at once and in a single method call. Trivial to make a fluent API atop this, although it’s not in the spec.

UpdateManyModel (f.e.) f i l t e r : D o c u m e n t r e q u i r e d u p d a t e : D o c u m e n t r e q u i r e d u p s e r t : B o o l e a n o p t i o n a l

BulkWriteOptions o r d e r e d : B o o l e a n Basically the same thing as InsertManyOptions.

Result Classes Insert results may report driver-generated IDs Delete results include counts Update results include counts and server-generated IDs from upserts Bulk results aggregate all of the above Results are optional for unacknowledged writes. (e.g. O p t i o n a l < B u l k W r i t e R e s u l t > , i s A c k n o w l e d g e d boolean) Since all fields within insert results are optional, i n s e r t O n e ( ) and i n s e r t M a n y ( ) may be void!

Write Errors Spec is flexible on how errors are reported Mainly concerned that info is accessible under consistent fields and names. Doesn’t address merging errors We have a bulk write spec for that

Diving into the API Chapter 3: Find and Modify I’ll keep this short…

Find and Modify Methods f i n d O n e A n d D e l e t e ( f i l t e r : D o c u m e n t , o p t i o n s : F i n d O n e A n d D e l e t e O p t i o n s ) : D o c u m e n t ; f i n d O n e A n d R e p l a c e ( f i l t e r : D o c u m e n t , r e p l a c e m e n t : D o c u m e n t , o p t i o n s : F i n d O n e A n d R e p l a c e O p t i o n s ) : D o c u m e n t ; f i n d O n e A n d U p d a t e ( f i l t e r : D o c u m e n t , u p d a t e : D o c u m e n t , o p t i o n s : F i n d O n e A n d U p d a t e O p t i o n s ) : D o c u m e n t ; Option classes contain only the relevant command options.

Preemptive Q&A

Read Preferences? Generally, queries on same collection will use the same read preference. Spec assumes it’s set on client, database, or collection. Per-operation read preferences are permitted; the spec simply doesn’t define it.

Write Concerns? Everything we just said about read preferences…

f i n d O n e ( ) et al. Drivers are free to keep existing methods and add new ones, too. Please keep options and naming consistent.

Thanks!

FIN. Questions?

Image Credits and http://www.instructables.com/id/Egg-Cream/ Giphy Reaction Gifs https://octodex.github.com/wheres-waldocat/