Tweet
Tweet

Slide 1

Slide 1 text

Software Testing in AWS IoT with The Power of Python Serverless Meetup Tokyo #10 2018/08/31 Koji Nakayama 1

Slide 2

Slide 2 text

ᛔ૩奧Օ • Ӿઊ ଛလ • μ϶φϮϊϐϖ໌ୗտᐒ • ςЄϝЄϹφ樄咲᮱ • ςЄϝЄςαϖεЀυϘί • GitHub: knakayama • অͣ΀Θ΄: 䩚ܖίυί 2

Slide 3

Slide 3 text

ψϐτϴЀٖ਻΁ͺ͚ͼ • 扖ͯͩ; • AWS IoTΨڥአͭ͵ίЄκϓμώϰ • AWS IoT΁䌏ͯΡαЀϓνϹЄτϴЀϓφϕ • Python΁ͧ͠Ρͳ΄䋚ᤰֺ • 扖ͫ΀͚ͩ; • ϜЄϖγδί΄扖 3

Slide 4

Slide 4 text

揾ාل樄ͭΔͯ 4

Slide 5

Slide 5 text

ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ 5

Slide 6

Slide 6 text

ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ 6

Slide 7

Slide 7 text

AWS IoTͽͽͣΡԆ΀ͩ; • IoTϔϝαφ΄ϔЄό㷧ᵞ • 敋΄֖ᗝఘ䁭Ψࣈ㾩΁ϤϺϐϕͯΡ • Ⴥଶ/氱ݻͣ΀Ϳ΄ॠ䶲ఘ䁭ΨίϤϷͽᤒᐏͯΡ • IoTϔϝαφ΄ګக • 䒐ਛڹ΁είπЀ΄櫮რΨͺͧΡ • 㪥敋ڹ΁敋΄ჅଶΨ䄜͞Ρ 7

Slide 8

Slide 8 text

AWS IoTͽͽͣΡԆ΀ͩ; • IoTϔϝαφ΄ϔЄό㷧ᵞ ! Քࢧ΅ͩͷΟ΁ϢζЄθφ • 敋΄֖ᗝఘ䁭Ψࣈ㾩΁ϤϺϐϕͯΡ • Ⴥଶ/氱ݻͣ΀Ϳ΄ॠ䶲ఘ䁭ΨίϤϷͽᤒᐏͯΡ • IoTϔϝαφ΄ګக • 䒐ਛڹ΁είπЀ΄櫮რΨͺͧΡ • 㪥敋ڹ΁敋΄ჅଶΨ䄜͞Ρ 8

Slide 9

Slide 9 text

IoTϔϝαφ΄ϔЄόΨ㷧ᵞͯΡ ίЄκϓμώϰ 9

Slide 10

Slide 10 text

IoTϔϝαφ΄ϔЄό㷧ᵞ 10

Slide 11

Slide 11 text

IoTϔϝαφ - AWS IoT 11

Slide 12

Slide 12 text

AWS IoT - Kinesis Data Streams 12

Slide 13

Slide 13 text

IoT Rule;AWS IoT SQL • IoT Rule΅SQLΨڥአͭͼIoTϔϝαφ͡ΟᭆΟ΢ͼͣ͵ϔЄό Ψ඙̵֢ͭ஍姆΄AWSςЄϠφ΁ͳ΄ϔЄόΨჁͯͩ;͢ͽͣ Ρ SELECT *, topic(3) as deviceId, timestamp() as timestamp FROM '$aws/things/+/shadow/update' WHERE state.reported.temperature > 20 13

Slide 14

Slide 14 text

Kinesis Data Streams - Consumers 14

Slide 15

Slide 15 text

Consumers - Storage 15

Slide 16

Slide 16 text

IoTϔϝαφ΄ϔЄόΨ݇ᆙͯΡ ίЄκϓμώϰ 16

Slide 17

Slide 17 text

IoTϔϝαφ΄ϔЄό݇ᆙ 17

Slide 18

Slide 18 text

ϳЄσ - ϯϝαϸίϤϷ 18

Slide 19

Slide 19 text

ϯϝαϸίϤϷ - 扯戣 19

Slide 20

Slide 20 text

ϯϝαϸίϤϷ - API Gateway 20

Slide 21

Slide 21 text

ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ ! 3. Python΁ΞΡ䋚ᤰֺ 21

Slide 22

Slide 22 text

֜Ψϓφϕͭ͵͚΄͡ • ϓφϕͭ͵͚ᅩ΅2ͺ • IoTϔϝαφ΄ϔЄό΅φϕϹЄυ΁כਂͽͣΡҘ • API͡Οͳ΄ϔЄόΨ݇ᆙͽͣΡҘ • ᥝͯΡ΁αЀϓνϹЄτϴЀϓφϕΨͭ͵͚ 22

Slide 23

Slide 23 text

αЀϓνϹЄτϴЀϓφϕΨͿͩͽΚΡ͡ • ϯϐμᔮϑЄϸ vs AWS΄䋚厏ह • ϯϐμᔮϑЄϸ(LocalStack/moto/etc...)ͽ䌏䖕ݢᚆ΀䁰ݳ΅च๜ͳ΢ ͽ亹ΔͱͼΡ • ϢΰЄϖϝϐμϸЄϤΨ㱢ض • ϯϐμᔮϑЄϸͽ΄αЀϓνϹЄτϴЀϓφϕΨग़ͥ䨗͚ͼ̵AWS 䋚厏हͽ΄αЀϓνϹЄτϴЀΨ੝΀ͥͯΡ䜐ኼ • IoT Rule΅ͳΘͳΘ๚䌏䖕΀Ξ͜΀΄ͽAWS΄䋚厏हͽ䋚ෞ 23

Slide 24

Slide 24 text

ίυδЀύ 1. AWS IoTΨڥአͭ͵ίЄκϓμώϰ 2. ϓφϕොဩΨᘍ͞Ρ 3. Python΁ΞΡ䋚ᤰֺ ! 24

Slide 25

Slide 25 text

ϓφϕ΄༷ኼ • MQTTμ϶αίЀϕͽϔЄόΨכਂ • ͩ΢Ψϓφϕڹ΁䋚ෞͭͼͥ͠ • HTTPμ϶αίЀϕͽAPI͡Οͳ΄ϔЄόΨ݇ᆙ • ๗இͭ͵ϔЄό͢ݐ஑ͽͣΡ͡ϓφϕ 25

Slide 26

Slide 26 text

MQTTμ϶αίЀϕͽϔЄόΨכਂ 26

Slide 27

Slide 27 text

Python΁ΞΡMQTTള姆 27

Slide 28

Slide 28 text

Python΁ΞΡMQTTള姆 28

Slide 29

Slide 29 text

HTTPμ϶αίЀϕͽϔЄόΨ݇ᆙ 29

Slide 30

Slide 30 text

HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 30

Slide 31

Slide 31 text

HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 31

Slide 32

Slide 32 text

HTTPμ϶αίЀϕ΁ΞΡϔЄό݇ᆙ 32

Slide 33

Slide 33 text

ڥአͯΡϯυϲЄϸ • MQTTμ϶αίЀϕ: AWS IoT Device SDK for Python • ϓφϕϢϹЄϭϼЄμ: pytest • ͳ΄՜: • HTTPμ϶αίЀϕ: requests • Cognito;΄SRP: warrant • AWS IoT;΄Ϸϕ϶α㳌ቘ: tenacity 33

Slide 34

Slide 34 text

ϓφϕϔЄό΄伛㯪 @pytest.mark.parametrize( 'publish_message', [ (({ 'state': { 'reported': { 'foo': 'bar', 'baz': 'qux' } } })) ], indirect=True) def test_fetch_device_data(api_endpoint, token, publish_message): ... 34

Slide 35

Slide 35 text

ᥴ抍 • pytest΅fixture;͚͜՛奲ΕͽϥϸϞ樛හΨ conftest.py ΁懿ᬿݢᚆ1 • ϓφϕξЄφ΄୚හ΁fixtureΨჁͭͼ̵ϓφϕڹ΁ͳ΢Ψ䋚ᤈͽͣΡ • ͩͩͽMQTTμ϶αЀϕ͡ΟϔЄό΄Publish;SRP扯戣Ψ亹Δͱͼͥ͠ • @pytest.mark.parametrize ͽϓφϕϔЄόΨϞ϶ϮЄό۸̵ͭ1ͺ΄ϓφϕξЄφͽ姜嬄௔ ΨṛΗΡͩ;͢ͽͣΡ2 • indirect=True ͭͼͥ͠;fixture΁ϓφϕϔЄόΨჁͯͩ;͢ͽͣΡ3 3 https://docs.pytest.org/en/latest/example/parametrize.html#apply-indirect-on-particular-arguments 2 https://docs.pytest.org/en/latest/parametrize.html 1 https://docs.pytest.org/en/latest/plugins.html 35

Slide 36

Slide 36 text

MQTTμ϶αЀϕ͡Ο΄Publish from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient @pytest.fixture(scope='function') @retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, max=10)) def publish_message(request): ... mqtt = AWSIoTMQTTClient('my_thing_01') mqtt.configureEndpoint(..., 8883) mqtt.configureCredentials(...) mqtt.connect() for message in request.param: mqtt.publish(topic, json.dumps(message), 1) time.sleep(5) 36

Slide 37

Slide 37 text

ᥴ抍 • fixture΅ϔπϹЄόͽ愢හ೰ਧͽͣΡ • pytest΅scope;͚͜՛奲ΕͽfixtureΨϓφϕξЄφ樌ͽوํͯΡͩ;͢ݢᚆ4 • ηϣυδμϕ΄ኞ౮΁πφϕ͢͡͡ΡfixtureΨوํͭͼͥ͠;ϞϢζЄϫЀφ͢ṛΔΡ • tenacity΄ retry ϔπϹЄόͽAWS IoTεЀϖϪαЀϕ;΄Ϸϕ϶α㳌ቘΨ懿ᬿ5 • pytest䰤伛΄ request fixtureͽϞ϶ϮЄό۸ͫ΢͵ϓφϕϔЄόΨ݇ᆙͭͼ͚Ρ6 6 https://docs.pytest.org/en/latest/reference.html#request 5 https://github.com/jd/tenacity 4 https://docs.pytest.org/en/latest/fixture.html#scope-sharing-a-fixture-instance-across-tests-in-a-class-module-or- session 37

Slide 38

Slide 38 text

requestsϯυϲЄϸͽAPI΁ίμψφ import requests @pytest.mark.parametrize( 'publish_message', [ ... def test_fetch_device_data(api_endpoint, token, publish_message): response = requests.get(f'{api_endpoint}/devices/my_thing_01', headers={'Authorization': token}) body = response.json() assert response.status_code == 200 assert body['deviceId'] == 'my_thing_01' 38

Slide 39

Slide 39 text

warrant΁ΞΡCognito΄SRP from warrant.aws_srp import AWSSRP @pytest.fixture(scope='session') def token(): config_abs_path = str(pathlib.Path(__file__).parent.joinpath('configs').resolve()) config = json.load(open(f'{config_abs_path}/config.json', 'r')) aws = AWSSRP(username=config['username'], password=config['password'], pool_id=config['pool_id'], client_id=config['client_id'], client=boto3.client('cognito-idp')) tokens = aws.authenticate_user() return tokens['AuthenticationResult']['AccessToken'] 39

Slide 40

Slide 40 text

ᥴ抍 • ϓφϕξЄφ΄୚හ΁ token fixtureΨਧ嬝̵HTTPϥϐύ΁ϕЄ μЀΨ戔ਧͭͼAPIΨGET • ݐ஑ͭ͵奾ຎ͢఺㾩ͭ͵Θ΄͡ assert ͽϓφϕ 40

Slide 41

Slide 41 text

Δ;Η • AWS IoTΨֵ͜;IoTϔϝαφ΄ϔЄό㷧ᵞ/ګக͢ͽͣΡ • ςЄϝЄϹφίϤϷξЄτϴЀ΄αЀϓνϹЄτϴЀϓφϕ΅ ϓφϕ΁䖕ͮͼ䋚ᤈ厏हΨᘍ䚅ͯΏͣ • Python΄ϯυϲЄϸΨֵ͜;఺क़;墋㶨΁αЀϓνϹЄτϴЀ ϓφϕ䨗ͧΡ 41