Software Testing in AWS IoT with The Power of Python

Serverless Meetup Tokyo #10 の登壇資料です。

Transcript

1. Software Testing in AWS IoT with The Power of Python Serverless Meetup Tokyo #10 2018/08/31 Koji Nakayama 1
2. ᛔ૩奧Օ • Ӿઊ ଛလ • μ϶φϮϊϐϖ໌ୗտᐒ • ςЄϝЄϹφ樄咲᮱ • ςЄϝЄςαϖεЀυϘί • GitHub: knakayama • অͣ΀Θ΄: 䩚ܖίυί 2
3. ψϐτϴЀٖ਻΁ͺ͚ͼ • 扖ͯͩ; • AWS IoTΨڥአͭ͵ίЄκϓμώϰ • AWS IoT΁䌏ͯΡαЀϓνϹЄτϴЀϓφϕ • Python΁ͧ͠Ρͳ΄䋚ᤰֺ • 扖ͫ΀͚ͩ; • ϜЄϖγδί΄扖 3
4. 揾ාل樄ͭΔͯ 4
5. ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ 5
6. ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ 6
7. AWS IoTͽͽͣΡԆ΀ͩ; • IoTϔϝαφ΄ϔЄό㷧ᵞ • 敋΄֖ᗝఘ䁭Ψࣈ㾩΁ϤϺϐϕͯΡ • Ⴥଶ/氱ݻͣ΀Ϳ΄ॠ䶲ఘ䁭ΨίϤϷͽᤒᐏͯΡ • IoTϔϝαφ΄ګக • 䒐ਛڹ΁είπЀ΄櫮რΨͺͧΡ • 㪥敋ڹ΁敋΄ჅଶΨ䄜͞Ρ 7
8. AWS IoTͽͽͣΡԆ΀ͩ; • IoTϔϝαφ΄ϔЄό㷧ᵞ ! Քࢧ΅ͩͷΟ΁ϢζЄθφ • 敋΄֖ᗝఘ䁭Ψࣈ㾩΁ϤϺϐϕͯΡ • Ⴥଶ/氱ݻͣ΀Ϳ΄ॠ䶲ఘ䁭ΨίϤϷͽᤒᐏͯΡ • IoTϔϝαφ΄ګக • 䒐ਛڹ΁είπЀ΄櫮რΨͺͧΡ • 㪥敋ڹ΁敋΄ჅଶΨ䄜͞Ρ 8
9. IoTϔϝαφ΄ϔЄόΨ㷧ᵞͯΡ ίЄκϓμώϰ 9
10. IoTϔϝαφ΄ϔЄό㷧ᵞ 10
11. IoTϔϝαφ - AWS IoT 11
12. AWS IoT - Kinesis Data Streams 12
13. IoT Rule;AWS IoT SQL • IoT Rule΅SQLΨڥአͭͼIoTϔϝαφ͡ΟᭆΟ΢ͼͣ͵ϔЄό Ψ඙̵֢ͭ஍姆΄AWSςЄϠφ΁ͳ΄ϔЄόΨჁͯͩ;͢ͽͣ Ρ SELECT *, topic(3) as deviceId, timestamp() as timestamp FROM '$aws/things/+/shadow/update' WHERE state.reported.temperature > 20 13
14. Kinesis Data Streams - Consumers 14
15. Consumers - Storage 15
16. IoTϔϝαφ΄ϔЄόΨ݇ᆙͯΡ ίЄκϓμώϰ 16
17. IoTϔϝαφ΄ϔЄό݇ᆙ 17
18. ϳЄσ - ϯϝαϸίϤϷ 18
19. ϯϝαϸίϤϷ - 扯戣 19
20. ϯϝαϸίϤϷ - API Gateway 20
21. ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ ! 3. Python΁ΞΡ䋚ᤰֺ 21
22. ֜Ψϓφϕͭ͵͚΄͡ • ϓφϕͭ͵͚ᅩ΅2ͺ • IoTϔϝαφ΄ϔЄό΅φϕϹЄυ΁כਂͽͣΡҘ • API͡Οͳ΄ϔЄόΨ݇ᆙͽͣΡҘ • ᥝͯΡ΁αЀϓνϹЄτϴЀϓφϕΨͭ͵͚ 22
23. αЀϓνϹЄτϴЀϓφϕΨͿͩͽΚΡ͡ • ϯϐμᔮϑЄϸ vs AWS΄䋚厏ह • ϯϐμᔮϑЄϸ(LocalStack/moto/etc...)ͽ䌏䖕ݢᚆ΀䁰ݳ΅च๜ͳ΢ ͽ亹ΔͱͼΡ • ϢΰЄϖϝϐμϸЄϤΨ㱢ض • ϯϐμᔮϑЄϸͽ΄αЀϓνϹЄτϴЀϓφϕΨग़ͥ䨗͚ͼ̵AWS 䋚厏हͽ΄αЀϓνϹЄτϴЀΨ੝΀ͥͯΡ䜐ኼ • IoT Rule΅ͳΘͳΘ๚䌏䖕΀Ξ͜΀΄ͽAWS΄䋚厏हͽ䋚ෞ 23
24. ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ ! 24
25. ϓφϕ΄༷ኼ • MQTTμ϶αίЀϕͽϔЄόΨכਂ • ͩ΢Ψϓφϕڹ΁䋚ෞͭͼͥ͠ • HTTPμ϶αίЀϕͽAPI͡Οͳ΄ϔЄόΨ݇ᆙ • ๗இͭ͵ϔЄό͢ݐ஑ͽͣΡ͡ϓφϕ 25
26. MQTTμ϶αίЀϕͽϔЄόΨכਂ 26
27. Python΁ΞΡMQTTള姆 27
28. Python΁ΞΡMQTTള姆 28
29. HTTPμ϶αίЀϕͽϔЄόΨ݇ᆙ 29
30. HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 30
31. HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 31
32. HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 32
33. ڥአͯΡϯυϲЄϸ • MQTTμ϶αίЀϕ: AWS IoT Device SDK for Python • ϓφϕϢϹЄϭϼЄμ: pytest • ͳ΄՜: • HTTPμ϶αίЀϕ: requests • Cognito;΄SRP: warrant • AWS IoT;΄Ϸϕ϶α㳌ቘ: tenacity 33
34. ϓφϕϔЄό΄伛㯪 @pytest.mark.parametrize( 'publish_message', [ (({ 'state': { 'reported': { 'foo': 'bar', 'baz': 'qux' } } })) ], indirect=True) def test_fetch_device_data(api_endpoint, token, publish_message): ... 34
35. ᥴ抍 • pytest΅fixture;͚͜՛奲ΕͽϥϸϞ樛හΨ conftest.py ΁懿ᬿݢᚆ1 • ϓφϕξЄφ΄୚හ΁fixtureΨჁͭͼ̵ϓφϕڹ΁ͳ΢Ψ䋚ᤈͽͣΡ • ͩͩͽMQTTμ϶αЀϕ͡ΟϔЄό΄Publish;SRP扯戣Ψ亹Δͱͼͥ͠ • @pytest.mark.parametrize ͽϓφϕϔЄόΨϞ϶ϮЄό۸̵ͭ1ͺ΄ϓφϕξЄφͽ姜嬄௔ ΨṛΗΡͩ;͢ͽͣΡ2 • indirect=True ͭͼͥ͠;fixture΁ϓφϕϔЄόΨჁͯͩ;͢ͽͣΡ3 3 https://docs.pytest.org/en/latest/example/parametrize.html#apply-indirect-on-particular-arguments 2 https://docs.pytest.org/en/latest/parametrize.html 1 https://docs.pytest.org/en/latest/plugins.html 35
36. MQTTμ϶αЀϕ͡Ο΄Publish from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient @pytest.fixture(scope='function') @retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, max=10)) def publish_message(request): ... mqtt = AWSIoTMQTTClient('my_thing_01') mqtt.configureEndpoint(..., 8883) mqtt.configureCredentials(...) mqtt.connect() for message in request.param: mqtt.publish(topic, json.dumps(message), 1) time.sleep(5) 36
37. ᥴ抍 • fixture΅ϔπϹЄόͽ愢හ೰ਧͽͣΡ • pytest΅scope;͚͜՛奲ΕͽfixtureΨϓφϕξЄφ樌ͽوํͯΡͩ;͢ݢᚆ4 • ηϣυδμϕ΄ኞ౮΁πφϕ͢͡͡ΡfixtureΨوํͭͼͥ͠;ϞϢζЄϫЀφ͢ṛΔΡ • tenacity΄ retry ϔπϹЄόͽAWS IoTεЀϖϪαЀϕ;΄Ϸϕ϶α㳌ቘΨ懿ᬿ5 • pytest䰤伛΄ request fixtureͽϞ϶ϮЄό۸ͫ΢͵ϓφϕϔЄόΨ݇ᆙͭͼ͚Ρ6 6 https://docs.pytest.org/en/latest/reference.html#request 5 https://github.com/jd/tenacity 4 https://docs.pytest.org/en/latest/fixture.html#scope-sharing-a-fixture-instance-across-tests-in-a-class-module-or- session 37
38. requestsϯυϲЄϸͽAPI΁ίμψφ import requests @pytest.mark.parametrize( 'publish_message', [ ... def test_fetch_device_data(api_endpoint, token, publish_message): response = requests.get(f'{api_endpoint}/devices/my_thing_01', headers={'Authorization': token}) body = response.json() assert response.status_code == 200 assert body['deviceId'] == 'my_thing_01' 38
39. warrant΁ΞΡCognito΄SRP from warrant.aws_srp import AWSSRP @pytest.fixture(scope='session') def token(): config_abs_path = str(pathlib.Path(__file__).parent.joinpath('configs').resolve()) config = json.load(open(f'{config_abs_path}/config.json', 'r')) aws = AWSSRP(username=config['username'], password=config['password'], pool_id=config['pool_id'], client_id=config['client_id'], client=boto3.client('cognito-idp')) tokens = aws.authenticate_user() return tokens['AuthenticationResult']['AccessToken'] 39
40. ᥴ抍 • ϓφϕξЄφ΄୚හ΁ token fixtureΨਧ嬝̵HTTPϥϐύ΁ϕЄ μЀΨ戔ਧͭͼAPIΨGET • ݐ஑ͭ͵奾ຎ͢఺㾩ͭ͵Θ΄͡ assert ͽϓφϕ 40
41. Δ;Η • AWS IoTΨֵ͜;IoTϔϝαφ΄ϔЄό㷧ᵞ/ګக͢ͽͣΡ • ςЄϝЄϹφίϤϷξЄτϴЀ΄αЀϓνϹЄτϴЀϓφϕ΅ ϓφϕ΁䖕ͮͼ䋚ᤈ厏हΨᘍ䚅ͯΏͣ • Python΄ϯυϲЄϸΨֵ͜;఺क़;墋㶨΁αЀϓνϹЄτϴЀ ϓφϕ䨗ͧΡ 41