Share
Embed
[EN] Golang Package Composition for Web Application: The Case of Mercari Kauru
Transcript
54 slides
-
Mercari Tech Conf 2017 Software Engineer Osamu Tonomori Golang Package
Composition for Web Applications: The Case of Mercari Kauru -
Contents 1. About Me 2. About Mercari Kauru 3. About
Package Composition 4. Usefulness of the Interface 5. Future Developments 6. Conclusion -
About Me • 主森 理 (Osamu Tonomori) Screen name: @osamingo
• Souzoh Inc. Mercari Kauru team • Software Engineer Server-side, Gopher ʕ◔ϖ◔ʔ -
I presented on how to use GAE at the TECH
PLAY Conference 2017 • Google “gae 新規 web” to see the slides -
Survey Results ʕ◔ϖ◔ʔ from Golang.Tokyo Participants Choose three topics that
you thought were interesting Package separation 29 responses -
Advanced Testing in Go - GopherCon 2017 • Google “gophercon
testing” to see it -
The three types of package composition • One Package When
the repository itself is an individual package • Flat Packages Makes the purpose of each package clear, and separates accordingly. • Multiple Packages Separated by each design pattern, such as MVC, DDD etc. -
One Package • Simple is Best (๑•̀ㅂ•́)و✧ • Easy to
get coverage • Good for things that can be done with a simple structure like libraries etc. . ├── ctx.go ├── ctx_test.go ├── debug.go ├── debug_test.go ├── error.go ├── error_test.go ├── handler.go ├── handler_func.go ├── handler_func_test.go ├── jsonrpc.go ├── jsonrpc_test.go ├── method.go ├── method_test.go ├── parse.go └── parse_test.go -
Difference between Flat Packages and Multiple Packages Feature A Feature
B Feature C Feature A, Feature B, Feature C Feature A, Feature B, Feature C Feature A, Feature B, Feature C Flat Packages Categorized based on different features Multiple Packages Categorized based on design patterns -
Flat Packages • Similar to the standard Golang package structure
• Even if you traverse the layers, it settles down at the second level • As roles are clear for each package, they are loose coupling • Good for Micro Services and Middleware etc. . ├── ctx │ ├── ctx.go │ └── ctx_test.go ├── debug │ ├── debug.go │ └── debug_test.go ├── error │ ├── error.go │ └── error_test.go ├── handler │ ├── handler.go │ ├── handler_func.go │ └── handler_func_test.go ├── jsonrpc.go ├── jsonrpc_test.go ├── parse │ ├── parse.go │ └── parse_test.go └── method ├── method.go └── method_test.go -
Multiple Packages • Package design that is closely linked to
design patterns • Good for monolithic web services . ├── controller │ ├── debug │ │ ├── debug.go │ │ └── debug_test.go │ └── handler.go ├── jsonrpc.go ├── jsonrpc_test.go ├── model │ ├── ctx │ │ ├── ctx.go │ │ └── ctx_test.go │ ├── error │ │ ├── error.go │ │ └── error_test.go │ ├── marshal │ │ ├── unmarshal.go │ │ └── unmarshal_test.go │ └── method │ ├── method.go │ └── method_test.go └── view └── index.html.tpl -
Mercari Kauru uses Multiple Packages • A monolithic web service
managed with one repository As it uses GAE, it works well with a Project Based GOPATH. • Uses various design patterns that simplify the DDD It is close to the composition of our previous Mercari Atte application and accounts for human resource mobility -
About the Google App Engine • Platform as a Service
(PaaS) offered by GCP • Competitors include Heroku, Engine Yard etc. -
Repository Structure • Uses a gb that works well with
GAE projects • $ gb vendor … Dependent library management • $ gb gae … gae-sdk wrap command group • Environment variables used for the test should be injected with direnv. . ├── Makefile ├── appengine # File that defines GAE│ ├── admin-api │ ├── api │ ├── cron.yaml │ ├── default │ ├── dispatch.yaml │ ├── index.yaml │ ├── management │ └── queue.yaml ├── cmd # Handmade convenient command group ├── docs # uml, sql etc. design docs ├── src │ └── kauru # Application itself │ ├── application │ ├── domain │ ├── infrastructure │ ├── interfaces │ └── library ├── test # Defines variables for test environment │ └── .envrc └── vendor # Dependent library management ├── manifest └── src -
• Made from 4 layers Application Domain Infrastructure Interface(s) •
Existence of the library directory Package group forms a complete unit Lightweight DDD . └── kauru ├── application ├── domain ├── infrastructure ├── interfaces └── library -
• Gathers up domain layer processes and provides them to
the interface layer • Separates packages with same concept as Flat Packages • Used with internal packages and covers up the common logic Application Layer . └── kauru ├── application │ ├── audio │ │ └── audio.go │ ├── book │ │ └── book.go │ ├── exhibit │ ├── install │ ├── internal │ ├── product │ ├── ranking │ ├── search │ ├── stock │ ├── user │ └── visual ├── domain ├── infrastructure ├── interfaces └── library -
Domain Layer • The Domain layer does not exist in
other layers • Provides Entity, Enum, Interface etc. . └── kauru ├── application ├── domain │ ├── bookmaster │ │ └── repository.go │ ├── bookmaster.go │ ├── configuration.go │ ├── domain.go │ ├── error.go │ ├── mercari.go │ ├── pager.go │ ├── product.go │ ├── time.go │ ├── user │ │ └── repository.go │ └── user.go ├── infrastructure ├── interfaces └── library -
Infrastructure Layer • Wrap the GAE library • Has basic
processing capabilities Logging Validation Rule Word Filtering Value Get/Set in Context Persistence . └── kauru ├── application ├── domain ├── infrastructure │ ├── backoff │ │ └── backoff.go │ ├── configuration │ │ └── configuration.go │ ├── crypto │ ├── ctx │ ├── identifier │ ├── log │ ├── persistence │ ├── queue │ ├── storage │ ├── tx │ ├── validation │ └── wordfilter ├── interfaces └── library -
Interface Layer • JSON-RPC’s Handler group Search.FindProduct etc. • Original
Error definition -32044: Not Found etc. • Interceptor group Packs a lot into context Action Logger etc. • Other Startup processing Obtains movement condition . └── kauru ├── application ├── domain ├── infrastructure ├── interfaces │ ├── api │ │ └── search │ │ └── find_product.go │ ├── error.go │ ├── interceptor │ │ └── recovery.go │ ├── interfaces.go │ ├── management │ │ └── cron │ │ └── backup_datastore.go │ ├── stats.go │ └── warmup.go └── library -
Library Directory • Package group that is not dependent on
the application • Many have been made into OSS packages github.com /osamingo - 5 packages /tenntenn - 3 packages . └── kauru ├── application ├── domain ├── infrastructure ├── interfaces └── library ├── device │ └── device.go ├── errutil │ └── temporary.go ├── io └── strrecord -
Points to Be Careful of • Each underlying layer is
composed of flat packages The most important thing is cutting the packages appropriately • Even within the layer, we need to protect the domain layer above all else If the Domain fails, we’ll be in cycle import hell and it will become weak to changes. • If you’re lost, choose the most simple path I like the quote “Keep It Simple Stupid” -
Background • The design pattern used for Multiple Packages often
waits for object oriented movement • With Golang, we can use the interface and fillers to expect object oriented movement. -
So does Golang even have objects? • “Is Golang an
object oriented language?” http://postd.cc/is-go-object-oriented/ This post from @spf13 is really helpful. • What does “object oriented” here refer to? a. It doesn’t refer to the program as code and data, but uses the concept of an “object” to integrate the two b. An object is an abstract data type that has a condition (data) and a behavior (code) -
type ( Duck interface { Waddler Quacker } Waddler interface
{ Waddle() (lat, lng float64) } Quacker interface { Quack() string } ) • Defined method list • An interface without a method is interface{} • All defined types fulfill interface{} How to Define the Interface -
type Osamingo struct { Lat float64 Lng float64 } func
(o *Osamingo) Waddle() (float64, float64) { return o.Lat, o.Lng } func (o *Osamingo) Quack() string { return "Hey" } func main() { var o interface{} = &Osamingo{ Lat: 35.662056, Lng: 139.728451, } switch o.(type) { case Duck: fmt.Println("Hi, duck!") } } How to Implement the Interface • Similar to Duck Typing • If you implement Waddler and Quacker, it will become Duck. -
An Example of the Flow of Registering User Info in
the Database • An example of what we are actually doing with Mercari Kauru. • Will introduce with the following steps a. Entity interface design b. User entity implementation c. UserRepository interface design d. UserRepository implementation e. Cloud Datastore library function wrap -
About Cloud Datastore • A fully-managed NoSQL service offered by
GCP GCP • Used by Pokémon GO etc. • “Google Cloud Datastore Inside-Out” https://www.slideshare.net/enakai/google-cloud-datastore-insideout -
// Domain layer - domain/domain.go type ( EntityBehavior interface {
Identifier datastore.PropertyLoadSaver } Identifier interface { GetID() string SetID(string) } UpdateTimestamper interface { GetUpdatedAt() time.Time SetUpdatedAt(time.Time) } ) • EntityBehavior Condition for storing in the database • Identifier Getter/Setter for the ID datastore. • PropertyLoadSaver Manages load, save from the datastore • CreateTimestamper Manages CreatedAt • UpdateTimestamper Manages UpdatedAt Entity Interface -
User Entity Architecture • Construct User Entity Nothing complicated //
Domain layer - domain/user.go type User struct { ID string `json:"id" datastore:"-" validate:”required”` ConnectedAt time.Time `json:"connected_at" datastore:"connected_at" validate:”required”` } -
User Entity Implementation // Domain layer - domain/user.go func (u
*User) GetID() string { return u.ID } func (u *User) SetID(id string) { u.ID = id } func (u *User) Load(p []datastore.Property) error { return Load(u, p) } func (u *User) Save() ([]datastore.Property, error) { return Save(u) } • Implementing EntityBehavior for the user • For Load and Save, prepare a common function within the entity -
Entity’s Load Function // Domain layer - domain/domain.go func Load(dst
interface{}, p []datastore.Property) error { err := datastore.LoadStruct(dst, p) if err != nil { return err } return LoadTimestamps(dst, p) } func LoadTimestamps(i interface{}, p []datastore.Property) error { ut, isUT := i.(UpdateTimestamper) if !isUT { return nil } for i := range p { if p[i].Name == "updated_at" { ut.SetUpdatedAt(p[i].Value.(time.Time)) } } return nil } • Uses App Engine library’s LoadStruct • If xTimestamper is implemented, it will be loaded -
// Domain layer - domain/domain.go func Save(src interface{}) ([]datastore.Property, error)
{ p, err := datastore.SaveStruct(src) if err != nil { return nil, err } return append(p, SaveTimestamps(src)...), nil } func SaveTimestamps(i interface{}) []datastore.Property { p := make([]datastore.Property, 0, 1) if ut, ok := i.(UpdateTimestamper); ok { ut.SetUpdatedAt(time.Now()) return append(nil, datastore.Property{ Name: "updated_at", Value: ut.GetUpdatedAt(), }) } return nil } • Uses the App Engine library’s savestruct • If xTimestamper is implemented, a time will be set Entity’s Save Function -
Repository’s Interface // Domain layer - domain/user/user.go type repository interface
{ Get(c context.Context, id int64) (*domain.User, error) Save(c context.Context, u *domain.User) error } var Repository repository • Java Interface definition • The Infrastructure layer’s persistence package is in charge of implementation -
Repository Implementation // Infrastructure layer - // infrastructure/persistence/user_repository.go type UserRepository
datastoreRepository func NewUserRepository() *UserRepository { return &UserRepository{ kind: "User", } } func (r *UserRepository) Get(c context.Context, id int64) (*domain.User, error) { u := &domain.User{} u.SetID(id) return u, get(c, r.kind, u) } func (r *UserRepository) Save(c context.Context, u *domain.User) error { return put(c, r.kind, u) } • DI the domain layer from the Infrastructure layer • For get and put, prepare a common function within the repository. -
Datastore Function Wrap // Infrastructure layer - // infrastructure/persistence/persistence.go func
get(c context.Context, kind string, dst domain.EntityBehavior) error { key := datastore.NewKey(c, kind, dst.GetID(), 0, nil) return datastore.Get(c, key, dst) } func put(c context.Context, kind string, src domain.EntityBehavior) error { if err := validation.Check(src); err != nil { return err } key :-= datastore.NewKey(c, kind, src.GetID(), 0, nil) _, err := datastore.Put(c, key, src) return err } • For each parameter, set a kind and EntityBehavior • Use the Get, Put in the app engine library -
Unique Points • Separating use of an interface as an
entity and as a repository For the Entity, expect an action For the Repository, expect DI • Wrap the App Engine library and use it Use the interface as an entity -
Moving Towards Clearer DDD • As the application grows, the
logic expands into the application layer and enlarges internal packages more than required. => In order to prioritize development speed, I tried my best to implement DRY => However, it is true that DDD and DRY are not compatible => Remove the reference from the interface, Application layers to the infrastructure layer. -
Streamlining the Initialization Process • As an application grows, the
initialization processing becomes split up into interface layer and infrastructure layer parts. => Register JSON-RPC method in the interface layer => DI of domain repository in the infrastructure layer => You’re really not supposed to initialize on each layer… (ヽ´ω`) => Start an application and streamline initialization processing within the main function. -
Knowledge Extraction • Of the interfaces introduced, identifier and TimeStamper
are some of the more commonly used => Extracting the Datastore library as a loosely wrapped construct makes it easier to generalize and maintain => Most new services within Souzoh use GAE and Datastore, which helps improve the synergy => Will continue to create libraries for sharing, adding test functionality as well -
Conclusion • It is best to consider the programming language
you are going to use before designing. Dev will be smoother if you consider what each language is best at beforehand • It is hard to be perfect from the start. However, it is possible to make it adaptable to change. It is not good to create a condition where special cases are not accepted right from the start. -
“ ” Clear is better than clever. Rob Pike Go
Proverts, Google, Inc.