high maintenance difficult to diagnose benefits better than humans at input generation more tests, more coverage better chance at finding bugs before users do
high maintenance difficult to diagnose benefits better than humans at input generation more tests, more coverage better chance at finding bugs before users do
are relatively easy to test most practical systems involve making a series of stateful api calls, most of which are CRUD problems inertia, learning curve thinking in properties slow, and ineffective high maintenance difficult to diagnose
test failures aren’t necessarily relevant more tests ≠ more coverage problems inertia, learning curve thinking in properties slow, and ineffective high maintenance difficult to diagnose
and act portions of test are still dependent on source problems inertia, learning curve thinking in properties slow, and ineffective high maintenance difficult to diagnose
a point complex, stateful tests aren’t necessarily deterministic, and are difficult to reproduce finding root-cause of failure without the right tools can be very difficult problems inertia, learning curve thinking in properties slow, and ineffective high maintenance difficult to diagnose
([{:as param}] ...)) declare action dependencies what action must occur before? use a DAG dependencies are transitive, and can have order automate the “arrange” phase
[::user/add-card] ::user/get [::user/create] ::cart/add-item [::user/authenticate] ::cart/update-item [::cart/add-item] ::cart/remove-item [::cart/add-item] ::cart/get [::user/authenticate] ::payment/apply-offer [::cart/add-item] ::payment/pay [::user/add-card ::cart/add-item] ::payment/refund [::user/pay] ::payment/cancel [::cart/add-item]}) declare action dependencies what action must occur before? use a DAG dependencies are transitive, and can have order automate the “arrange” phase
::user/add-card [::user/authenticate] ::user/update-card [::user/add-card] ::user/get [::user/create] ::cart/add-item [::user/authenticate] ::cart/update-item [::cart/add-item] ::cart/remove-item [::cart/add-item] ::cart/get [::user/authenticate] ::payment/apply-offer [::cart/add-item] ::payment/pay [::user/add-card ::cart/add-item] ::payment/refund [::user/pay] ::payment/cancel [::cart/add-item]}) what action must occur before? use a DAG dependencies are transitive, and can have order automate the “arrange” phase
::user/get ::cart/add-item ::cart/update-item ::cart/remove-item ::cart/get ::payment/apply-offer ::payment/pay ::payment/refund ::payment/cancel}) (for [action actions] (assert-no-error action)) what action must occur before? use a DAG dependencies are transitive, and can have order automate the “arrange” phase
:blacklist #{::payment/pay ::payment/refund ::payment/cancel ::user/signup}} f(a) · f(a) ≡ f(a) 2 kinds of adjacencies blacklist / whitelist sensitive / distributed systems a b c c d e a b c d c e immediate distant a1.
when there are 100s of flows as on CI d2. {:checkpoints [{:action ::user/authenticate :description "Logged in"} {:action ::cart/add-item :description "Added item to cart"} {:action ::payment/pay :description "Payment attempted"}]}
matrices for flows 4. deterministic seed data 5. model external domain 1. store everything immutably 2. params as latest values in state 3. catalog known errors 4. abstract request engine 1. algebraic properties 2. state machines as properties 3. domain based invariants 1. tests as portable data 2. domain based checkpoints 3. flow timeline, and walk throughs 4. automated bug reports generation assertion execution diagnosis