AWS APIGateway + Python Lambda + NEologdで作るサーバレス日本語形態素解析API

AWS APIGateway + Python Lambda + NEologdで作るサーバレス日本語形態素解析API #pyconjp2017

Transcript

1. AWS APIGATEWAY + LAMBDA + NEOLOGD Ͱ࡞ΔαʔόϨε೔ຊޠܗଶૉղੳAPI
2. ࣗݾ঺հ • ໳࿬ ་ (KADOWAKI Satoru) • BakFoo, Inc
 όΫϑʔגࣜձࣾ CTO • PyConJP2015τʔΫ • Tornado/ElasticSearchͰ࣮ݱ͢ΔେྔπΠʔτͷ
3. όΫϑʔגࣜձࣾ • NHK NMAPS ϦΞϧλΠϜؾ৅ՄࢹԽ
4. όΫϑʔגࣜձࣾ • SocioMeasure http://socio.bakfoo.com
5. όΫϑʔגࣜձࣾ • ιʔγϟϧղੳ - πΠολʔ ࣗવݴޠղੳ-
6. ໨త • MeCab + ৽ޠΛ࣋ͭࣙॻ؀ڥ • Ͱ͖Δ͚ͩؾܰʹ࢖͍͍ͨ • Θ͟Θ͟ઐ༻αʔόͰͳͯ͘΋...(αʔόϨεʣ • MeCabͷηοτΞοϓ͸ҙ֎ʹ໘౗ • OSʹґଘͨ͠จࣈίʔυ໰୊ • ࣙॻ͸Ͳ͏͢΂͖͔໰୊ • ԿΛ࢖͏͔ɼͲ͏͍͏ࣙॻ͕ඞཁ͔ • ൺֱత௿ίετͰօͰ࢖͑Δ؀ڥ • Ϗϧυ͔ΒσϓϩΠ·Ͱͷ֓ཁ
7. ࣮૷؀ڥ֓ཁ • AWS API Gateway + Lambda + Python MeCab API Gateway Lambda S3
8. Ϗϧυ͔ΒσϓϩΠ·ͰͷྲྀΕ 1. Lambda༻Ϗϧυ؀ڥΛ࡞੒ 2. LambdaεΫϦϓτ࡞੒ 3. AWS΁σϓϩΠ
9. 1. Lambda༻Ϗϧυ؀ڥΛ࡞੒
10. Ϗϧυ؀ڥ • AWS LambdaͷͨΊʹEC2 (ඞਢ) • AMI: amzn-ami-hvm-2016.03.3.x86_64-gp2 • Python 3.6.1 + Miniconda • Python2.7ͷྫ • AWS Lambda Ͱ MeCab Λಈ͔͢ • http://dev.classmethod.jp/cloud/improved-aws-lambda-with-mecab/ • NEologdࣙॻͷϏϧυ • Ubuntu 16.04 ͜ͷAMIͰmecabΛίϯύΠϧʂ
11. Ϗϧυ؀ڥͷৄࡉ
12. 1. Ϗϧυ؀ڥߏங • @Lambda༻EC2Πϯελϯε • AMI: amzn-ami-hvm-2016.03.3.x86_64-gp2 • Πϯετʔϧ࡞ۀ $ sudo yum install gcc $ sudo yum install gcc-c++ $ sudo yum install git $ sudo yum install patch $ sudo rpm -ivh \ > http://packages.groonga.org/centos/groonga-release-1.1.0-1.noarch.rpm
13. 1. Ϗϧυ؀ڥߏங • Python؀ڥ $ wget --quiet \ > https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh \ > -O ~/miniconda3.sh $ /bin/bash miniconda3.sh -b -p $HOME/miniconda3 $ echo 'export PATH="$HOME/miniconda3/bin:$PATH"' >> .bashrc $ pip install boto3
14. 1. Ϗϧυ؀ڥߏங • Lambda༻ͷmecabΠϯετʔϧ $ mkdir $HOME/lambda_neologd $ curl -L \ > "https://drive.google.com/uc? export=download&id=0B4y35FiV1wh7cENtOXlicTFaRUE" \ > -o mecab-0.996.tar.gz $ tar zxvf mecab-0.996.tar.gz && cd mecab-0.996 $ ./configure --prefix=$HOME/lambda_neologd/local --enable-utf8-only $ make && make install ๨Εͣʹ
15. 1. Ϗϧυ؀ڥߏங • ಉ༷ʹmecab-ipadic΋Πϯετʔϧ $ curl -L \ > "https://drive.google.com/uc? export=download&id=0B4y35FiV1wh7MWVlSDBCSX ZMTXM" \ > -o mecab-ipadic-2.7.0-20070801.tar.gz $ tar xvzf mecab-ipadic-2.7.0-20070801.tar.gz $ cd mecab-ipadic-2.7.0-20070801/ $ export PATH=$HOME/lambda_neologd/local/bin:$PATH $ export \ > LD_LIBRARY_PATH=$HOME/lambda_neologd/local/lib:$LD_LIBRARY_PATH $ ./configure --prefix=$HOME/lambda_neologd/local \ > --enable-utf8-only --with-charset=utf8 $ make && make install ๨Εͣʹ ؀ڥม਺
16. 1. Ϗϧυ؀ڥߏங • mecabͷಈ࡞֬ೝ $ mecab -F "%m\t%h,%H,%pw,%pc,%pn\n" mecabςετ mecab 41,໊ࢺ,ݻ༗໊ࢺ,Ұൠ,*,*,*,mecab,ϝΧϒ,ϝΧϒ,6518,6208,6208 ςετ 36,໊ࢺ,αม઀ଓ,*,*,*,*,ςετ,ςετ,ςετ,3637,9107,2899 • Python MeCabͷΠϯετʔϧ $ cd $HOME/sandbox/lambda_neologd $ pip install mecab-python3 -t . ๨Εͣʹ
17. Lambda؀ڥ֓ཁ • AWS Lambda 512MB! NEologdࣙॻ͸900MBఔ౓͋Γͦͷ··Ͱ͸ಈ͔ͳ͍ʂ
18. 1. Ϗϧυ؀ڥߏங • NEologdࣙॻͷੜ੒ • ඪ४తͳࣙॻ࡞੒Ͱ͸Lambda࣮૷Ͱ͖ͳ͍ • ࠷খߏ੒ͰࣙॻΛ࡞੒͢Δ $ git clone --depth 1 https://github. com/neologd/mecab-ipadic-neologd.git $ cd mecab-ipadic-neologd $ ./bin/install-mecab-ipadic-neologd -y \ > -p $HOME/neologd -n --eliminate-redundant-entry --eliminate-redundant-entryͰࣙॻαΠζ͸400MBఔ౓
19. 1. Ϗϧυ؀ڥߏங • --eliminate-redundant-entry: Ͳ͏͢Δʁ • NEologdͷࣙॻʹ͸༳Β͕͗ొ࿥͞Ε͍ͯΔ • ྫ: • AMAZON, amazon, Amazon • ͍͢͝ɼ͢͝ʔ͍ɼ͢͝ʔʔʔʔʔ͍ ϊʔϚϥΠζ͞ΕͨࣙॻΛ࡞੒͢Δ • ۩ମతʹ΍Δ͜ͱ • ΞϧϑΝϕοτ൒֯খจࣈԽ • AMAZON, amazon, Amazon →ʮamazonʯ
20. 1. Ϗϧυ؀ڥߏங • ࣙॻͷݩσʔλ • mecab-ipadic-neologd/seed/*.csv.xz • (ྫ) mecab-ipadic-neologd/seed/mecab-user-dict-seed.20170828.csv.xz • ͜ͷCSV.XZͷϑΝΠϧͷதʹ୯ޠ͕ొ࿥͞Ε͍ͯΔ • ͜ΕΒͷϑΝΠϧΛϊʔϚϥΠζͯࣙ͠ॻ࡞੒͢Δ
21. 1. Ϗϧυ؀ڥߏங - ࠷ॳͷࣙॻ࡞੒ޙ $ python seed_normalize.py - ࣙॻΛ࠶࡞੒(ࣙॻߋ৽͠ͳ͍) $ ./bin/install-mecab-ipadic-neologd -y \ > -p $HOME/neologd --eliminate-redundant-entry ᶃ seedΛϊʔϚϥΠζ ᶄࣙॻͷݩσʔλΛߋ৽͠ͳ͍ͰࣙॻΛ࠶࡞੒ ==ʮ"-n" ΦϓγϣϯΛ͚ͭͳ͍ʯ
22. 1. Ϗϧυ؀ڥߏங • seed_normalize.py # coding: utf-8 import lzma import os import unicodedata seeds_dir = 'mecab-ipadic-neologd/seed/' files = os.listdir(seeds_dir) for file in files: if 'xz' in file: print('normalizing mecab seed...', file) f = lzma.open(seeds_dir + file, 'rb') lines = f.readlines() f.close() with lzma.open(seeds_dir + file, "w") as f: for line in lines: norm_text = unicodedata.normalize( 'NFKC', line.decode()).lower() f.write(norm_text.encode('utf-8')) ϊʔϚϥΠζ͢Δ͚ͩ
23. 1. Ϗϧυ؀ڥߏங dicdir = /tmp/neologd • $HOME/lambda_neologd/local/etc/mecabrc • ࣙॻͷύεΛ/tmpʹมߋ • Ϗϧυ؀ڥͰ͸γϯϘϦοΫϦϯΫʹ $ ln -s $HOME/neologd /tmp/neologd Ϗϧυ؀ڥͰ͖͕͋Γʂ
24. 2. LambdaεΫϦϓτ࡞੒
25. 2. LambdaεΫϦϓτ࡞੒ • PythonεΫϦϓτͰ΍Δ͜ͱ 1. ࣙॻΛLambdaΠϯελϯεॳظԽ࣌ʹ S3͔Βμ΢ϯϩʔυ • ࣙॻ͸mecabrcʹ߹Θͤͯ/tmp/neologdʹ 2. τʔΫφΠζͷલॲཧ • ࣙॻੜ੒ͱಉ༷ʹϊʔϚϥΠζΛ࣮ߦ • ςΩετΛαχλΠζ • ςΩετͷछྨʹΑ࣮ͬͯ૷Λม͑Δ 3. ߹໊ࢺɼϑϨʔζநग़ͷ࣮૷Λ௥Ճ
26. 2. LambdaεΫϦϓτ࡞੒ • αχλΠζͱ͸ • ೔ຊޠͷॲཧʹ͓͍ͯ • จষͷதͰͦΕ΄Ͳҙຯͷͳ͍(Ԛ͍)૷০ͷ ͨΊͷจࣈྻͳͲΛҰൠతͳจࣈྻʹ߹Θ ͤΔ • ྫ: • ಉ఺ΰʔʔʔʔϧʂ → ಉ఺ΰʔϧʂ • ௕Իූ߸ʢ−ʣΛ·ͱΊΔ • ۟ಡ఺Λ·ͱΊΔ • จࣈྻͷؒͷεϖʔε͸۟఺ΛຒΊΔ • ϨΠΞ΢τͷͨΊͷ࿈ଓ͢ΔεϖʔεΛ ࡟আʢΠϯσϯτʣ
27. 2. LambdaεΫϦϓτ࡞੒ • ʮNHKϚΠϧνϟϯϐΦϯγοϓʯ
 ˠʮNHKϚΠϧʯͰলུ͞ΕΔέʔε
 ˠ NHKͱϚΠϧʹ෼ׂ͞Εͯ͠·͏
 ɹɹˠʮNHKʯ-ʮϚΠϧʯΛ࿈݁ • ߹໊ࢺͱ͸ • ࣙॻʹ͸ͳ͍࿈ଓ͢Δ໊ࢺΛ૊Έ߹Θͤͯ Ұͭͷ୯ޠʢ৽ޠʁʣͱͯ͠ѻ͏ • ʮετʔϦʔɾϝΠΩϯά͕ૉ੖Β͍͠ʯ
 ɹˠʮετʔϦʔʯ-ʮɾʯ-ʮϝΠΩϯάʯΛ࿈݁
28. 2. LambdaεΫϦϓτ࡞੒ • ࡞੒ͨ͠εΫϦϓτ • lambda_function.py • ϝΠϯεΫϦϓτ • normalize.py • αχλΠζ༻ • termextract.py • ߹໊ࢺ • ϑϨʔζநग़
29. 2. LambdaεΫϦϓτ࡞੒ • lambda_function.py ͷϑϩʔ 1. Πϯϙʔτ෦ 2. S3઀ଓͷॳظԽ (boto3) 3. mecabࣙॻͷμ΢ϯϩʔυ 4. MeCabϞδϡʔϧϩʔυ 5. τʔΫφΠζʢίΞʣॲཧ
30. 2. LambdaεΫϦϓτ࡞੒ • lambda_function.py • 1. Πϯϙʔτ෦ import os import logging import traceback import unicodedata import json import boto3 import ctypes import normalize import termextract libdir = os.path.join(os.getcwd(), 'local', 'lib') libmecab = ctypes.cdll.LoadLibrary(os.path.join(libdir,'libmecab.so')) ctypesΛ࢖༻ͯ͠libmecab.soΛϩʔυ
31. • lambda_function.py • 2. S3ॳظԽ(boto3) # Configuration: AWS AWS_S3_BUCKET = 'mecabdic' BOTOCONF = { 'aws_access_key_id': 'AKI.....', 'aws_secret_access_key': 'pTWl.....', 'region_name': 'ap-northeast-1' } boto3.setup_default_session(**BOTOCONF) session = boto3._get_default_session() session_region = session._session.get_config_variable('region') s3 = boto3.client('s3') 2. LambdaεΫϦϓτ࡞੒
32. • lambda_function.py • 3. mecabࣙॻͷμ΢ϯϩʔυ MECAB_DIC_FILES = [ # mecabࣙॻͷϦετ 'char.bin', 'dicrc', 'left-id.def','matrix.bin','pos-id.def', 'rewrite.def', 'right-id.def', 'sys.dic', 'unk.dic', ] DICDIR = '/tmp/neologd/' # ࣙॻͷอଘઌ # S3͔ΒࣙॻΛμ΢ϯϩʔυ def prepareMecabDic(): if not os.path.exists(DICDIR): os.mkdir(DICDIR) for mdic in MECAB_DIC_FILES: dest_dic = DICDIR + mdic if not os.path.exists(dest_dic) or os.path.getsize(dest_dic) == 0: with open(dest_dic, 'wb') as f: s3.download_file(AWS_S3_BUCKET, mdic, dest_dic) prepareMecabDic() /tmp/neologdʹμ΢ϯϩʔυ 2. LambdaεΫϦϓτ࡞੒ ඞཁͳࣙॻΛϦετʹ
33. • lambda_function.py • 4. MeCabϞδϡʔϧϩʔυ import MeCab # init MeCab MECABRC = os.path.join(os.getcwd(), 'local', 'etc', 'mecabrc') tagger = MeCab.Tagger("-r %s" % MECABRC) import normalize # αχλΠζϞδϡʔϧ import termextract # ߹໊ࢺੜ੒ɼϑϨʔζநग़Ϟδϡʔϧ 2. LambdaεΫϦϓτ࡞੒ mecabrcύεͷࢦఆ αχλΠζɼ߹໊ࢺੜ੒Ϟδϡʔϧ
34. • lambda_function.py • 5. τʔΫφΠζ(ίΞ)෦෼ def lambda_handler(event, context): text = event['queryStringParameters'].get('text', '') is_termext = event['queryStringParameters'].get('termextract', '') is_phrase = event['queryStringParameters'].get('phrase', '') if text: text = unicodedata.normalize('NFKC', text.strip()).lower() text = normalize.cleansingText(text) node = tagger.parse('') node = tagger.parseToNode(text) (ଓ͖͸࣍ͷεϥΠυʹ....) 2. LambdaεΫϦϓτ࡞੒ httpΫΤϦετϦϯάऔಘ ϊʔϚϥΠζ/αχλΠζ/τʔΫφΠζ
35. • lambda_function.py • 5. τʔΫφΠζ(ίΞ)෦෼ node = tagger.parse('') node = tagger.parseToNode(text) ### ଓ͖ tokens = [] if is_termext: # ߹໊ࢺੜ੒ tokens = termextract.tokenize(node) elif is_phrase: # ϑϨʔζநग़ tokens = termextract.phrases(node) else: while node: token = { 'surface': node.surface, 'posid': node.posid, 'feature': node.feature, } tokens.append(token) node = node.next 2. LambdaεΫϦϓτ࡞੒ ߹໊ࢺੜ੒ ϑϨʔζநग़ mecabͦͷ··
36. 2. LambdaεΫϦϓτ࡞੒ • normalize.py αχλΠζॲཧ
37. ؀ڥߏங • normalize.py ςΩετͷαχλΠζॲཧ import re CLEANSING_PATTERNS = [ [r'\r\n|\r|\n|\\n', 'ɻ'], # վߦίʔυΛআڈ [r'\t+|\s+', ' '], # ࿈ଓ͢Δλϒ·ͨ͸࿈ଓ͢Δεϖʔε [u'\u30FC+', u'\u30FC'], # ࿈ଓ͢ΔϋΠϑϯΛ1ͭͷϋΠϑϯʹ͢Δ [r'([^\w])\s([^\w])', r'\1ɻ\2'], # 2όΠτจࣈؒͷεϖʔε͸۟఺Ͱஔ׵ [u'ɾɾ+', 'ɾ'], # ࿈ଓ͢Δத఺Λ1ͭʹ [u'ɻɻ+', 'ɻ'], # ࿈ଓ͢Δ۟఺Λ1ͭʹ ] def cleansingText(text): for src, dst in CLEANSING_PATTERNS: res = re.sub(src, dst, text) return res [ม׵લ, ม׵ޙ] ΛϦετʹ re.subͰஔ׵
38. 2. LambdaεΫϦϓτ࡞੒ • termextract.py 1. ࢖༻͢Δ඼ࢺIDͷఆٛ 2. ߹໊ࢺੜ੒ϧʔϧΛ࡞੒ 2. ϑϨʔζੜ੒ϧʔϧΛ࡞੒
39. 2. LambdaεΫϦϓτ࡞੒ • termextract.py • 1. ࢖༻͢Δ඼ࢺIDͷఆٛ import re APPLY_IDS = [30, 36, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 56, 57] APPLY_JOIN_IDS = [10, 13, 18, 20, 24, 25, 31, 32, 33, 51, 58] NOT_ENDOFWORDS = [ [13, "͔Β"], [18, "͕"], [18, "΋ͷͷ"], [18, "ͱ"], [24, "ͷ"], ] EX_APPLY_IDS = [ [18, "ͯ"], [18, "Ͱ"], [25, "ͨ"], [25, "ͣ"], [25, "ͳ͍"], [25, "·͢"], [25, "Μ"], [25, "·͠"], [25, "·ͤ"], [31, "͍"], [54, "ͦ͏"] ] IGNORE_WORDS =[u"ͦ͏", u"ͨ͠", u"͋ͱ", u"Կ౓΋", u"Կ͔"] ࿈݁ର৅ͷ඼ࢺID ඼ࢺIDͱ෇ଐޠͷηοτΛ૊Έ߹Θͤ
40. • termextract.py • ߹໊ࢺͷੜ੒ϧʔϧ࡞੒ʢྫʣ def wordFilter(token, buf): if len(buf) != 0: if token.posid == 4 \ and re.search(r'[ʔɾ]', token.surface) \ and re.search(r'^[͊-ΜΝ-λμ-ϲʔɾ]+$', buf[-1].surface) \ and buf[-1].posid != 30: return True for pid, w in EX_APPLY_IDS: if pid == token.posid and w == token.surface: return True if token.posid in APPLY_IDS: # ΞϧϑΝϕοτͷཏྻ໊͕ࢺͱ൑அ͞ΕΔͷΛ๷͙ # (ID: 45Ͱݻ༗໊ࢺ,૊৫ͱID:38ͰҰൠ໊ࢺͷ2ύλʔϯ͕͋Δ) if len(token.surface) < 3 and re.search(r'[a-zA-Z:]+', token.surface): return False if len(buf) == 0 and token.surface in IGNORE_WORDS: return False return False (த఺ or ௕Իූ߸) and (࠷ޙͷจࣈྻ͕ͻΒ͕ͳ/ΧλΧφ) and ࠷ޙͷ୯ޠͷ඼ࢺID͕30(໊ࢺ઀ଓ)Ҏ֎Λ࿈݁͢Δ 2. LambdaεΫϦϓτ࡞੒ ৚݅ʹΑͬͯ͸࿈݁͠ͳ͍΋ͷ΋...
41. 2. LambdaεΫϦϓτ࡞੒ • ۩ମతʹ... • ʮετʔϦʔɾϝΠΩϯά͕ૉ੖Β͍͠ʯ ετʔϦʔ APPLY_IDS = [30, 36, 38, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 56, 57] ࿈݁ର৅඼ࢺID ɾ ϝΠΩϯά ඼ࢺID: 38 Ұൠ໊ࢺ ඼ࢺID: 4 ه߸ ඼ࢺID: 41 ݻ༗໊ࢺ ࿈݁ର৅IDʢͻΒ͕ͳɼ͔͔ͨͳɼத఺Λڐ༰ʣ
42. • termextract.py ϑϨʔζநग़ • ߹໊ࢺੜ੒ͱجຊతʹ͸ಉ͡ • ࿈݁ͤ͞Δ඼ࢺIDʹॿࢺͳͲΛ௥Ճ͢Δ 2. LambdaεΫϦϓτ࡞੒
43. 3. AWS΁σϓϩΠ
44. 3. AWS΁ͷσϓϩΠ • σϓϩΠͷϑϩʔ 1. ࡞੒ͨ͠ϓϩάϥϜΛzipʹ·ͱΊΔ 2. AWSϢʔβʔϩʔϧͷઃఆ 3. Lambda → S3 ϧʔτઃఆ 4. ηΩϡϦςΟάϧʔϓ࡞੒ 5. Lambdaͷ࡞੒ 6. API Gatewayͷઃఆ
45. 3. AWS΁ͷσϓϩΠ • 1. lambda_neologdΛzipͰ·ͱΊΔ • ෆཁͳϑΝΠϧΛexcludeʹ (zip: 1.5MBఔ౓ʹ) $ zip lambda_mecabneologd.zip -r * -x@exclude $ vim lambda_neologd/exclude *.dist-info/* *.egg-info *.pyc env/* exclude lambda_mecabneolod.zip local/bin/* local/include/* local/libexec/* local/share/* zipϑΝΠϧͷ࡞੒ exclude(zipʹؚΊͳ͍) ϑΝΠϧͷத਎
46. 3. AWS΁ͷσϓϩΠ • 2. AWS Ϣʔβʔϩʔϧͷઃఆ • IAMઃఆ • S3ಡΈࠐΈݖݶΛઃఆ
47. • 2. AWS Ϣʔβʔϩʔϧͷઃఆ • Lambdaͷ࣮ߦϩʔϧઃఆ lambda࣮ߦʹඞཁͳϩʔϧ 3. AWS΁ͷσϓϩΠ
48. • 3. Lambda → S3΁ͷϧʔτΛ࡞੒ • VPC, SubnetΛ࡞੒ • VPCΤϯυϙΠϯτΛ࡞੒ • ඞਢͰ͸ͳ͍:ݖݶ·ΘΓͰϋϚΒͳ͍ͨΊʹ ↓ → 3. AWS΁ͷσϓϩΠ
49. • 3. Lambda → S3΁ͷϧʔτΛ࡞੒ • ϧʔςΟϯάΛઃఆ 3. AWS΁ͷσϓϩΠ
50. • 4. ηΩϡϦςΟάϧʔϓͷ࡞੒ (IP੍ݶ) 3. AWS΁ͷσϓϩΠ
51. • 5. Lambdaͷ࡞੒ 3. AWS΁ͷσϓϩΠ zipΛΞοϓϩʔυ Python3.6 σϑΥϧτ ϩʔϧࢦఆ
52. • 5. Lambdaͷ࡞੒ • ϝϞϦ͸512MBʙ 3. AWS΁ͷσϓϩΠ λΠϜΞ΢τઃఆ VPCઃఆ ϝϞϦαΠζ
53. • 5. Lambdaͷ࡞੒ 3. AWS΁ͷσϓϩΠ αϒωοτઃఆ(2ͭ) ηΩϡϦςΟάϧʔϓࢦఆ
54. • 6. API Gateway ઃఆ 3. AWS΁ͷσϓϩΠ Lambdaͷࢦఆ
55. • 6. API Gateway ઃఆ • GETϝιουΛ࡞੒ 3. AWS΁ͷσϓϩΠ
56. • 6. API Gateway ઃఆ • API GatewayͷσϓϩΠΛ࣮ߦ 3. AWS΁ͷσϓϩΠ σϓϩΠΛ࣮ߦ ΤϯυϙΠϯτ͕ੜ੒
57. • https://qxhfsso....execute-api.ap-northeast- 1.amazonaws.com/stg/mecab/ tokenize.json?text=NHK૯߹ςϨϏͰ͓΋͠ Ζʔʔʔ͍൪૊΍ͬͯΔ σϞ(API)
58. • text=੺Ӌ୆τϯωϧ͸౦๺৽װઢ্໺Ӻ - େ ٶӺ͓ؒΑͼ࡛ژઢ੺ӋӺ - ๺੺ӋӺؒʹ͋ Δ૯Ԇ௕585ϝʔτϧͷమಓτϯωϧͰ͋Δɻ σϞ(API)
59. • ߹໊ࢺੜ੒Λ༗ޮʹ࣮ͯ͠ߦ • /mecab/tokenize.json?termextract=1&text=੺Ӌ୆... σϞ(API)
60. • ϑϨʔζநग़Λ༗ޮʹ࣮ͯ͠ߦ • /mecab/tokenize.json?phrase=1&text=੺Ӌ୆... σϞ(API)
61. σϞ Ԡ༻ฤͱͯ͜͠Μͳ΋ͷΛ࡞Γ·ͨ͠
62. σϞ • ಺ֳ෎ ݄ྫܦࡁใࠂ • ܠؾʹؔ͢Δ੓෎ͷެࣜݟղΛࣔ͢ใࠂॻ
63. σϞ • ಺ֳ෎ ݄ྫܦࡁใࠂ
64. σϞ • ಺ֳ෎ ݄ྫܦࡁใࠂ
65. σϞ • จষൺֱWEB
66. ·ͱΊ
67. • MeCab + ৽ޠΛ࣋ͭࣙॻ؀ڥͱͯ͠ • AWS Lambda + API GatewayΛ࢖༻ • NEologd͸lambda؀ڥʹ৐ͤΔ͜ͱ͕Ͱ͖Δ • ࣙॻΛ࠷খݶʹ͠ͳ͍ͱ͍͚ͳ͍ • Lambda͢Δ͜ͱͰؾܰʹ࢖͑Δͱ͍͏ϝϦοτ͸͋Δ • Lambdaʹ৐ͤͨͱ͖ͷҰ൪ͷ໰୊͸ॳճىಈ͕஗͍ ·ͱΊ
68. • ࣙॻ͸Ͳ͏͢΂͖͔໰୊ʹ͍ͭͯ • NEologdͷࣙॻ͸େมΑ͘Ͱ͖͍ͯΔ • ͔͠͠σΧ͍ʂʂ • ࣙॻΛ࠷খݶʹ͢ΔσϝϦοτΑΓ΋೔ຊޠͷΏΒ͗ʹ Ԡͯ͡ʮϊʔϚϥΠζʯɼʮαχλΠζʯ͔ͯ͠Βτʔ ΫφΠζ͢Δ͜ͱ͸ࣙॻ͕Ͳ͏Ͱ͋Ε༗ҙٛͰ͋Δ • ໊ࢺ࿈݁Λ࣮૷͢Δ͜ͱͰ߹໊ࢺ͕࡞੒Ͱ͖Δ • Ωʔϫʔυநग़ͳͲͰ͸ͱͯ΋࢖͑Δ • ϑϨʔζʹؔͯ͠͸ඞཁʹԠͯ͡඼ࢺ࿈݁Λ࣮૷ ·ͱΊ ͱ͜ΖͰɼσϞͰݟͤͨจॻൺֱͰ͕͢....
69. ·ͱΊ σϞͷจষൺֱWEBͰ࢖༻͍ͯ͠Δࣙॻ͸ Wikipedia͔Β࡞੒ͨ͠΋ͷʂʂʂʂʂ
70. ·ͱΊ σϞͷจষൺֱWEBͰ࢖༻͍ͯ͠Δࣙॻ͸ Wikipedia͔Β࡞੒ͨ͠΋ͷʂ ͭ·Γ... ϊʔϚϥΠζ΍αχλΠζɼ඼ࢺ࿈͕݁ॏཁ
 ͦͷ࣮૷͕Ͱ͖Ε͹NEologd͡Όͳͯ͘΋ྑ͍ʂʁ Wikipediaࣙॻʢor͸ͯͳࣙॻʣͰे෼Ͱ͸ʂʁ
71. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠