Hands-on engineering, not just coding...

Transcript

1. Hands-on engineering, not just coding… Maksim Naumovich Lead Developer at Epam Systems How to involve 100+ devs to work on single codebase…
2. 2 Agenda WHAT WE WILL TALK ABOUT • History of one project in Epam • Problem of involving big amount of devs to single codebase • Our modular solution • Speeding up testing and debugging in distributed system Grodno, 2019
3. 3 What are we doing Grodno, 2019
4. 4 What are we doing Grodno, 2019
5. 5 Initial implementation Grodno, 2019 E-MAIL Verizon LinkedIn Twitter Slack Teas LinkedIn Teams Facebook Skype Skype For Bizz Bloomberg TRM Yammer AT&T Verizon More More More More More More More More More More More More
6. 6 And then… Grodno, 2019
7. 7 And that was a problem… Grodno, 2019
8. 8 We could get this Grodno, 2019
9. 9 What we had Grodno, 2019 Micro-service (fetching, transforming, etc.) Embedded tomcat by spring boot Security Data access Messaging Storage
10. 10 Does anybody remember architects lesson? ☺ Grodno, 2019
11. 11 Idea Grodno, 2019 Security Data Fetch / Transformation Messaging Fetch / Transformation interface SDK Engine Plugins Storage
12. 12 Our ask Grodno, 2019 public interface FetchExecutor { void executeFetch(FetchExecution execution, FetchPipelines pipelines); } public interface FetchExecution { FetchState getFetchState(); void saveFetchState(FetchState fetchState); FetchSettings getFetchSettings(); FetchJobDetails getJobDetails(); } public interface FetchPipelines { SaveResourcePipeline saveResourcePipeline(); ResourceTrackingPipeline resourceTrackingPipeline(); SplitExecutionPipeline splitExecutionPipeline(); }
13. 13 Our help Grodno, 2019 public interface HttpFetchExecution extends FetchExecution { … HttpResponse fetchData(HttpFetchSettings settings); // few default methods } public interface RestFetchExecution extends HttpFetchExecution { … <V> V unmarshal(HttpResponse httpResponse, Class<V> type); <T> T fetchAsObject(RestFetchSettings settings, Class<T> type); <T> void interceptOnFetch(String path, Class<T> type, Consumer<T> consumer); // few default methods }
14. 14 Our config Grodno, 2019 @Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME) public @interface Fetch { /** * @return a protocol name */ String protocol(); /** * It's a hit for the settings deserialization. */ Class<? extends FetchSettings> settingsType() default FetchSettings.class; /** * Hint for state object deserialization. */ Class<? extends FetchState> stateType() default FetchState.class; /** * @return Transformer mapping for the plugin. */ String transformer() default ""; }
15. 15 Startup Grodno, 2019
16. 16 How it works Grodno, 2019
17. 17 Let’s save some time! Grodno, 2019 Security Classes Mock Storage File system Data access Messaging In-memory Fully configured local test env in one jar!
18. 18 As simple as… Grodno, 2019 <dependency> <groupId>com.epam</groupId> <artifactId>fetch-sdk-test</artifactId> <version>${fetch-sdk-test.version}</version> <scope>test</scope> </dependency> @BeforeClass public void startTestSdk() { FetchTestSdkApplication.run(); } { "settings": { "externalFeedId": "100_external", "feedId": 100, "protocol": "http.rest.slice", "path" : "/dataPath", "host": "localhost", "method": "GET", "authentication": { "type": "BASIC", "name": "username", "secret": "password" }, "port": 8080 }, "enabled":true, "schedule":"0 0/2 * * * ? *", "type":"fetch", "notifyOnError":true }
19. 19 And our controlled haven ☺ Grodno, 2019
20. 20 Thank You and Have fun! GROCON 2019 Grodno, 2019