Microservices and Testing, talking from the experience
At Tuenti we have been using microservices for a while and we have enjoyed the advantages but also suffered the disadvantages. In this talk we want talk about our experiencia with microservices and testing.
The test who knew too much... Storage FriendsApi UsersApi MomentsApi MVC Memcache Feature 1 Feature 2 Feature N Clients Tuenti social network architecture
MVC Clients Storage FriendsApi UsersApi MomentsApi Memcache Feature 1 Feature 2 Feature N Tuenti social network architecture The test who knew too much...
MVC Clients Storage SubscriptionsApi UsersApi MomentsApi Memcache Feature 2 Service Gateway Scripts Tuenti Movil Feature N Tuenti social network architecture during the “transition” The test who knew too much...
MVC SubscriptionsApi UsersApi MomentsApi Feature 2 Service Gateway Scripts Tuenti Movil Feature N Storage (tests DB) Fixtures Test A IntegrationTestA Tests during the “transition” The test who knew too much...
MVC SubscriptionsApi UsersApi MomentsApi Feature 2 Service Gateway Scripts Tuenti Movil Feature N Storage (tests DB) Fixtures Test A BrowserTestA Selenium/ WebDriver Common Fixtures Tests during the “transition” The test who knew too much...
MVC SubscriptionsApi UsersApi MomentsApi Feature 2 Tuenti Movil Feature N BrowserTestA Selenium/ WebDriver Service1 Service1 Service1 IntegrationTestA Tests during the “transition” The test who knew too much...
MVC SubscriptionsApi UsersApi MomentsApi Feature 2 Tuenti Movil Feature N BrowserTestA Selenium/ WebDriver Service1 Service1 Service1 IntegrationTestA The test who knew too much... Tests during the “transition”
The test who knew too much... Was it worth the trouble? - For Tuenti, yes. “The Big Snowball” Divide & Conquer Everything under control More complex Limited by PHP Different languages fit better for different purposes Huge releases, blocked released Deployment of small pieces is faster Monsters of general purpose Dedicated machines with a specific purpose
When the persistency layer stops being accessible from the testing environment because it is in a service... a. Have an instance of the service running on testing environment. b. Offer a “Stub-Service” with each service to use in testing environment. c. Mock/Stub the code that calls the service. d. Hybrid solution The dog ate my fixtures
a. Have an instance of the service running on testing environment. ○ Pros ■ All my business flows tests could be still valid! ■ My tests are going to detect the service contract changes. ○ Cons ■ Persistency of the data. ○ There should exist a setUp routine for each test (fixtures?). ■ For a certain use case could be too complicated. ■ Still, nothing ensures that the testing instance behaves like the production instance. ■ The testing environment becomes more complicated (logs/traces coming from services) The dog ate my fixtures
b. Offer a “Stub-Service” with each service to use in testing environment. ○ Pros ■ My tests are going to detect the service contract changes. ○ Cons ■ Who configures the Stub-Service responses? ● How? ■ To keep working my my business flows tests, the stub needs to be too smart ■ The Stub-Service gains complexity as the service grows. The dog ate my fixtures
c. Mock/Stub the code that calls the service. ○ Pros ■ Don’t worry anymore about how it works. Just worry about the input and output. ■ Traces are simpler ■ Helps to detect code smells ○ Cons ■ All the business flows tests are not valid anymore. ● Well, they can work with some twisted logic, but that is not the purpose. ■ Question: Do the tests still satisfy the contract? ● Contract testing The dog ate my fixtures
d. Hybrid solution: ○ There is no “silver bullet”. ■ Do all my tests need to cover the whole stack? ■ Am I going to be confident using just mocked data? ■ Where is the limit? ○ A good combination of test types leads us to a high coverage. The dog ate my fixtures
d. Hybrid solution (cont): ● Thumb rule: Low coupling ○ Minimize the contact between the components of my application. ○ And we know how… we’ve been taught! ■ Remember your OO teacher yelling “high cohesion, low coupling!” ■ SOLID ■ Clean Architecture ■ ... The dog ate my fixtures
1. Isolate the services ● Behind a semantic and meaningful layer of apis for my application. ○ How this API interacts with the rest of the world should not be of my interest. If i use it, I trust on it. ○ Keep my business logic services agnostic ● The test of my features should test my features, not the code in a service! ○ “The nest of spiders” paradigm ○ The easier-to-use and more meaningful the APIs and their methods are, the easier it gets the stubbing. Low coupling
1. Isolate the services My application (My happy features with their happy tests) Semantic API layer (Castle black, it just works) Something that the tests of my happy features should not interact with. (The nest of spiders) Low coupling
1. Isolate the service behind a semantic and meaningful layer of apis for my application. My application (My happy features with their happy tests) Semantic API layer (Castle black, it just works) Something that the tests of my happy features should not interact with. (The nest of spiders) Low coupling
1. Isolate the service behind a semantic and meaningful layer of apis for my application. My application (My happy features with their happy tests) Semantic API layer (Castle black, it just works) Something that the tests of my happy features should not interact with. (The spiders nest) Low coupling
1. Isolate the service behind a semantic and meaningful layer of apis for my application. My application (My happy features with their happy tests) Semantic API layer (Castle black, it just works) Something that the tests of my happy features should not interact with. (The spiders nest) Low coupling
2. Assume asynchronous interactions. Deal with it. ● Why? ○ Not all the environments have the same requirements ■ Scripts (background mode). I can wait forever. ● retries/monitoring/controlled timeouts => Synchronous. ■ With the user waiting. ● Beware of the timeouts! => Asynchronous. ● The user does not like to wait. Early return and process in background. Low coupling
2. Assume asynchronous interactions. Deal with it. ● How? ○ Polling, promises, events, notifications, queued jobs… ○ Ideally I should be able to instantiate the service in synchronous or asynchronous mode for execution. ○ If the service is slow, I should wrap the call to a service in a background task. ○ In both cases (asynchronous service response or background task) my API should state explicitly that this method is asynchronous. Low coupling
3. More than testing: logs & monitoring - Broken you said? Impossible! the tests are passing!! - Is testing environment, for God’s sake! Nobody thinks about production? ● Even with a high coverage and specially when we have distributed architecture we need monitoring and logs more than ever. Low coupling
The most difficult part Gateway The service Low coupling: How to test? Contract testing Service Stub Testing service instance Unit But at least, we have isolated the problem Another application, same strategy.
function onPaymentOk($millieuros, $purchaseId) { $purchase = PurchaseApi::getInstance()->getPurchase($purchaseId); InvoiceApi::getInstance()->generateAndSendInvoice($purchase); } function generateAndSendInvoice(PurchaseData $purchaseData) { $name = $purchaseData->getName(); InvoiceService::get()->generateAndSendInvoice($name, ...); } // Can we trust an early return and asynchronous execution or should I look into the Invoice service? Low coupling. Example:
// If the invoice service DOES NOT work in asynchronous mode… /* @return AsynchronousInvoice */ function generateAndSendInvoice(PurchaseData $purchaseData) { InvoiceGenerationQueuedJob::getInstance() ->generateAndSendInvoice($purchaseData); return AsynchronousInvoice::createUnavailable(); } Low coupling. Example:
tuenti
daihiraoka
tacck
jordihofc
chmikata
okazuki
hikiroku
tarappo
taka2noda
yujisoftware
asei
dzeyelid
nnstt1
holman
brettharned
hatefulcrawdad
sstephenson
reverentgeek
jakevdp
nonsquared
kneath
ufuk
thoeni
jmmastey