Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Методы превращения логов использования системы в тестовые сценарии

Методы превращения логов использования системы в тестовые сценарии

Иосиф Иткин, генеральный директор и соучредитель компании Exactpro

5 ноября 2020
Exactpro Online Meetup

---
Подписывайтесь на Exactpro в социальных сетях:

LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Подписывайтесь на YouTube канал Exactpro http://www.youtube.com/c/ExactproVlog

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

November 05, 2020
Tweet

Transcript

  1. 1 Build Software to Test Software exactpro.com Методы превращения логов

    использования системы в тестовые сценарии Exactpro Online Meetup 5 ноября 2020 Automation Technology Project of the Year: Exactpro and R3 The QA Financial Software and App Awards 2020
  2. 2 Build Software to Test Software exactpro.com Про Exactpro

  3. 3 Build Software to Test Software exactpro.com Requirement / specification

    analysis - non-executable description of the system Test libraries - executable description of the system Log files - traces left in the system after test execution Bug descriptions - discrepancies between test execution results and requirements / expectations News media - articles about the leaked software bugs and their consequences Applying AI to software testing
  4. 4 Build Software to Test Software exactpro.com Applying AI to

    software testing Requirement / specification analysis - non-executable description of the system Test libraries - executable description of the system Log files - traces left in the system after test execution Bug descriptions - discrepancies between test execution results and requirements / expectations News media - articles about the leaked software bugs and their consequences
  5. 5 Build Software to Test Software exactpro.com Посчитать каждый голос

    Кто я?
  6. 6 Build Software to Test Software exactpro.com Debugger – что

    это такое https://game.exactpro.com/
  7. 7 Build Software to Test Software exactpro.com Debugger – что

    это такое
  8. 8 Build Software to Test Software exactpro.com Debugger – а

    что плохого Что плохого в чрезмерном использовании отладчика в работе программиста? Голосуем
  9. 9 Build Software to Test Software exactpro.com Загадка

  10. 10 Build Software to Test Software exactpro.com Observability / Наблюдаемость

    Logs Traces Metrics
  11. 11 Build Software to Test Software exactpro.com Системы электронной торговли

  12. 12 Build Software to Test Software exactpro.com Смотрите на Exactpro

    Vlog https://youtu.be/6fWRdASYx0s
  13. 13 Build Software to Test Software exactpro.com Веб приложения Browser

    send requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Responsive front-end design allows a site to adapt to user’s device. Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet Web applications are always multi-user applications and many of them work under heavy load and are subject to concurrency access 6
  14. 14 Build Software to Test Software exactpro.com EXACTPRO Веб приложения

    Биржевые системы Тестирование толпой для разных систем
  15. 15 Build Software to Test Software exactpro.com Большие данные для

    тестирования CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 3) access logs 4) user data 5) random clicking 6) other applications
  16. 16 Build Software to Test Software exactpro.com Логи доступа

  17. 17 Build Software to Test Software exactpro.com Логи доступа inappropriate

  18. 18 Build Software to Test Software exactpro.com Генератор нагрузки Historical

    Production Data Generating Configuration Files for Load Injector Tool Instruments Activity Message Distributing by Types (Limit/Market/IOC/...) Message Mix (NEW/Amed/Cancel) Ration Message Rates User Activity across Different Gateways Analyzing and Aggregating Data
  19. 19 Build Software to Test Software exactpro.com Daily capacity -

    100+ mln transactions Peak rates - 40k+ transactions per second Average round-trip latency - <100 microseconds Availability - 100% 3000 trx 2.5 cm <1 mm Да какие логи при такой то нагрузке
  20. 20 Build Software to Test Software exactpro.com Да какие логи

    при такой то нагрузке
  21. 21 Build Software to Test Software exactpro.com Логи доступа inappropriate

    https://jmeter.apache.org/usermanual/jmeter_accesslog_sampler_step_by_step.html
  22. 22 Build Software to Test Software exactpro.com Что сделать легко

    в автоматизации тестирования ?
  23. 23 Build Software to Test Software exactpro.com Environment Context Diagram

    PRODUCTION TEST Environment Context Diagram
  24. 24 Build Software to Test Software exactpro.com 1. Preparation Phase

    – Key Steps, Volumetrics & Timeline 130 Million Messages analysed down to 800K tests - avg 4000 per fix connection 1. Preparation Phase – Key Steps, Volumetrics & Timeline
  25. 25 Build Software to Test Software exactpro.com How The Steps

    Relate to Data Held in the Shsha Database How The Steps Relate to Data Held in the Test Database
  26. 26 Build Software to Test Software exactpro.com 2. Execution Phase

    – Batch 1 – Key Steps and Timeline 2. Execution Phase – Batch 1 – Key Steps and Timeline
  27. 27 Build Software to Test Software exactpro.com Проигрывание логов https://exactpro.com/ideas/research-papers/trading-day-logs-replay-limitations-and-test-tools-applicability

  28. 28 Build Software to Test Software exactpro.com Проигрывание логов The

    following is required to ensure precise event replication: • Synchronized inbound data feeds; • Control over events sequencing within a distributed trading system, the ability to re-order events across gateways and internal components; • The ability to replace physical timing with logical timing; • The ability to intervene at original time scale; • Control over non-deterministic nature of the trading system.
  29. 29 Build Software to Test Software exactpro.com Cравнение картинок

  30. 30 Build Software to Test Software exactpro.com Недостатки процесса проигрывания

    логов https://exactpro.com/ideas/white-papers/test-automation-ccps-and-exchanges-operational-day-replay-limitations
  31. 31 Build Software to Test Software exactpro.com Нет/Нет – незначительные

    изменения
  32. 32 Build Software to Test Software exactpro.com Да/Нет – изменения

    в формате данных
  33. 33 Build Software to Test Software exactpro.com Нет/Да – новая

    функциональность
  34. 34 Build Software to Test Software exactpro.com Нет/Да – новая

    функциональность
  35. 35 Build Software to Test Software exactpro.com Да/Да – существенные

    изменения
  36. 36 Build Software to Test Software exactpro.com Два способа тестирования

    D - data-based M - model-based Что лучше? Test Scripts Log Files Database Interfaces System Configuration Network Captures Source Code Models + Descriptions Test Scripts + Descriptions
  37. 37 Build Software to Test Software exactpro.com Два способа тестирования

    Production Model Кто видел в море корабли не на конфетном фантике…
  38. 38 Build Software to Test Software exactpro.com Deliberate Practice https://www.slideshare.net/aavais/deliberate-practice-myth-magic-and-mindfulness

    10,000 Hours
  39. 39 Build Software to Test Software exactpro.com Тестирование это непрерывное

    обучение
  40. 40 Build Software to Test Software exactpro.com Два способа тестирования

    D - data-based M - model-based Process Mining Enum Checker Test Scripts Log Files Database Interfaces System Configuration Network Captures Source Code Models + Descriptions Test Scripts + Descriptions
  41. 41 Build Software to Test Software exactpro.com Process mining https://www.youtube.com/c/ExactproVlog

  42. 42 Build Software to Test Software exactpro.com Process mining

  43. 43 Build Software to Test Software exactpro.com Process mining

  44. 44 Build Software to Test Software exactpro.com Process mining

  45. 45 Build Software to Test Software exactpro.com Process mining

  46. 46 Build Software to Test Software exactpro.com NEXT GENERATION SOFTWARE

    TESTING TOOLKIT ADDRESSING THE NEW LEVELS OF COMPLEXITY IN FINANCIAL SYSTEMS Tackling the ever-increasing complexity of financial platforms, th2 is a next-generation toolkit for automated functional and non-functional testing of distributed transaction processing systems - securities trading systems and exchanges, banking and broker systems, post-trade and payments platforms, and many more. th2 is a Kubernetes-driven microservices solution tailor-made to deliver efficient machine-driven end-to-end test libraries with comprehensive coverage of your system. th2 consolidates the power of the entire Exactpro test tool suite in a single solution. Platform and technology agnostic AI-driven Customised for financial systems Strategic Data Acquisition Unified Data Warehouse Pervasive Automation Opportunities Deliberate Practice of Software Testing Follow us on social media: https://exactpro.com/ Exactpro Systems @exactpro On github.com this November!