Cloudflare + OpenSearch + Lambdaで始める全文検索開発入門

Developers.IO 2021 Decade で発表した資料です

1.  ৽نࣄۀ౷ׅ෦
   1SPqMZνʔϜ
   ࣋ా३࢙ $MPVEqBSF
   
   0QFO4FBSDI
   
   -BNCEBͰ࢝ΊΔ
   શจݕࡧ։ൃೖ໳
2. ຊηογϣϯͷର৅ऀ  w"NB[PO
   0QFO4FBSDI
   4FSWJDF΍0QFO4FBSDIʹڵຯͷ͋ Δํ
   w"NB[PO
   0QFO4FBSDI
   4FSWJDFΛར༻͍ͯͯ͠ɺ։ൃίετ Λগ͠Ͱ΋཈͍͑ͨํ
   w։ൃதͷϩʔΧϧαʔόʔΛ֎෦ެ։ͯ͠ར༻͍ͨ͠ํ
3. ຊηογϣϯͷΰʔϧ  AWS Lambda -> ϩʔΧϧʹߏங͞Εͨ OpenSearch΁ΞΫηεͰ͖ΔΑ͏ʹ͢Δ
4. ຊηογϣϯͰ࿩͞ͳ͍͜ͱ  wશจݕࡧͷ࢓૊Έ΍ΞϧΰϦζϜ
   w0QFO4FBSDIͷΫΤϦͳͲͷৄࡉͳ಺༰
   w-BNCEB΍αʔόʔϨεʹ͍ͭͯͷجૅతͳ࿩
   w0QFO4FBSDIͱ&MBTUJDTFBSDIʹؔ͢ΔΑ΋΍·࿩
5. ࣗݾ঺հ  ࣋ా
   ३࢙
   
   !BNPU[
   w৽نࣄۀ౷ׅ෦
   1SPqMZνʔϜ
 ιϑτ΢ΣΞΤϯδχΞ
   wௗऔͰϑϧϦϞʔτϫʔΫ
   w4BB4ϓϩμΫτͷاըɾ։ൃɾ ӡ༻
6. ΞδΣϯμ   "NB[PO
   0QFO4FBSDI
   4FSWJDF
   
   -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ
    ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங
    $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ -BNCEBͷ઀ଓ
    ·ͱΊ
7. ΞδΣϯμ   "NB[PO
   0QFO4FBSDI
   4FSWJDF
   
   -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ
    ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங
    $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ -BNCEBͷ઀ଓ
    ·ͱΊ
8. ϓϩϑΟʔϧϏϡʔΞʔαʔϏεʮ1SPqMZʯ 
9. ϓϩϑΟʔϧϏϡʔΞʔαʔϏεʮ1SPqMZʯ 
10. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 
11. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 
12. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 
13. 0QFO4FBSDI
    4FSWJDF
    
    -BNCEB͸खܰͰศར͚ͩͲɾɾ  0QFO4FBSDI
    4FSWJDFΛར༻͢Δ৔߹ʹײͨ͡՝୊
    wݸਓ؀ڥͩͱΠϯελϯεͷίετ͕ؾʹͳΔ
    wUNJDSPλΠϓͰ΋݄׵ࢉ͢Δͱ໿
    💰
    w՝ۚΛఀࢭ͢Δʹ͸Ұ୴υϝΠϯΛ࡟আ͢Δඞཁ͕͋Γɺ࠶ߏங͕໘౗
14. ։ൃίετΛ΋͏গ͠཈͍͑ͨ   4".
    -PDBM
    ΍
    -PDBM4UBDL
    Λ࢖ͬͯϩʔΧϧ؀ڥ Ͱಈ࡞֬ೝΛߦ͏
     ݱঢ়ͷߏ੒͸ͦͷ··ʹɺ0QFO4FBSDI
    4FSWJDF ͚ͩΛϩʔΧϧͷ0QFO4FBSIʹஔ͖׵͑Δ
15. ։ൃίετΛ΋͏গ͠཈͍͑ͨ   4".
    -PDBM
    ΍
    -PDBM4UBDL
    Λ࢖ͬͯϩʔΧϧ؀ڥ Ͱಈ࡞֬ೝΛߦ͏
     ݱঢ়ͷߏ੒͸ͦͷ··ʹɺ0QFO4FBSDI
    4FSWJDF ͚ͩΛϩʔΧϧͷ0QFO4FBSIʹஔ͖׵͑Δ ࠓճ͸ͪ͜ΒΛ࠾༻
16. ։ൃ͢Δ࣌͸ϩʔΧϧ0QFO4FBSDIΛ࢖͏  ΍ͬͯΈΔ 💪
17. ΞδΣϯμ   "NB[PO
    0QFO4FBSDI
    4FSWJDF
    
    -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ
     ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங
     $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ -BNCEBͷ઀ଓ
     ·ͱΊ
18. -BNCEB
    
    0QFO4FBSDI
    4FSWJDFͷγϯϓϧͳྫ 
19. -BNCEB
    
    ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 
20. -BNCEB
    
    ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 
21. %PDLFS
    *NBHFΛར༻ͨ͠0QFO4FBSDIߏங  $ docker pull opensearchproject/opensearch:1.0.1 $ docker pull opensearchproject/opensearch-dashboards:1.0.1 https://opensearch.org/docs/opensearch/install/docker/
22. %PDLFS
    *NBHFΛར༻ͨ͠0QFO4FBSDIߏங  version: '3' services: opensearch: image: opensearchproject/opensearch:1.0.1 container_name: opensearch environment: - cluster.name=docker-cluster - node.name=os-node - cluster.initial_master_nodes=os-node - bootstrap.memory_lock=true - http.host=0.0.0.0 - transport.host=127.0.0.1 - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "plugins.security.disabled=true" ulimits: memlock: soft: -1 hard: -1 volumes: - $PWD/.local/opensearch:/usr/share/opensearch/data ports: - 9200:9200 networks: - sandbox networks: sandbox: docker-compose.yml
23. %PDLFS
    *NBHFΛར༻ͨ͠0QFO4FBSDIߏங  version: '3' services: opensearch: image: opensearchproject/opensearch:1.0.1 container_name: opensearch environment: - cluster.name=docker-cluster - node.name=os-node - cluster.initial_master_nodes=os-node - bootstrap.memory_lock=true - http.host=0.0.0.0 - transport.host=127.0.0.1 - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "plugins.security.disabled=true" ulimits: memlock: soft: -1 hard: -1 volumes: - $PWD/.local/opensearch:/usr/share/opensearch/data ports: - 9200:9200 networks: - sandbox networks: sandbox: docker-compose.yml ࠓճ͸ݕূͷͨΊ
    ηΩϡϦςΟϓϥάΠϯΛແޮԽ
24. %PDLFS
    *NBHFΛར༻ͨ͠0QFO4FBSDIߏங  $ docker-compose up -d .. Creating network “local_sandbox" with the default driver Creating opensearch ... done Creating opensearch-dashboards ... done
25. ΞδΣϯμ   "NB[PO
    0QFO4FBSDI
    4FSWJDF
    
    -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ
     ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங
     $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ -BNCEBͷ઀ଓ
     ·ͱΊ
26. -BNCEB
    
    ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 
27. -BNCEB
    
    ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 
28. $MPVEqBSF
    "SHP
    5VOOFM
    $MJFOU
    w"SHP
    5VOOFMʢ5VOOFMJOH
    %BFNPOʣͷίϚϯυϥΠϯΫϥ ΠΞϯτ
    w$MPVEqBSFͷωοτϫʔΫΛհͯ͠ɺ೚ҙͷϩʔΧϧαʔό ΛϓϩΩγͰ͖Δ $MPVEqBSFEͱ͸ 
29. $MPVEqBSFEͱ͸  $MPVEqBSFEΛར༻͢ΔϝϦοτ
    w։ൃ༻్Ͱ͋Ε͹ɺແঈ͔ͭαΠϯΞοϓෆཁͰ͙͢ʹར༻ Մೳ
    wݻఆͷΧελϜυϝΠϯΛઃఆ͢Δ͜ͱ΋Ͱ͖Δ
    wୠ͠ɺ$MPVEqBSF΁ͷαΠϯΞοϓ͓ΑͼυϝΠϯઃఆ͕ඞཁ
30. $MPVEqBSFEͷΠϯετʔϧʢ.BD04ʣ  $ brew install cloudflare/cloudflare/cloudflared https://developers.cloudflare.com/cloudflare-one/connections/ connect-apps/install-and-setup/installation
31. $ cloudflared tunnel --url localhost:9200 $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel --url localhost:9200 ϩʔΧϧͷ0QFO4FBSDIΛ
    ࢦఆ͢Δ
32. $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel --url localhost:9200 INF Requesting new quick Tunnel on trycloudflare.com... INF +---------------------------------------------------------------+ INF | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): | INF | https://female-attract-average-join.trycloudflare.com | INF +---------------------------------------------------------------+ ..
33. $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel --url localhost:9200 INF Requesting new quick Tunnel on trycloudflare.com... INF +---------------------------------------------------------------+ INF | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): | INF | https://female-attract-average-join.trycloudflare.com | INF +---------------------------------------------------------------+ .. ϥϯμϜʹੜ੒͞Εͨ63-͕ ׂΓ౰ͯΒΕΔ
34. -BNCEB
    
    ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 
35. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js
36. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js 0QFO4FBSDIͷ
    +4༻ΫϥΠΞϯτΛར༻
37. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js ؀ڥม਺ͰΤϯυϙΠϯτΛ ઃఆͰ͖ΔΑ͏ʹ͓ͯ͘͠
38. -BNCEBͷ؀ڥม਺ʹΤϯυϙΠϯτΛઃఆ  $MPVEqBSFEͰ
    ׂΓ౰ͯΒΕͨ63-Λࢦఆ
39. ४උ͕੔ͬͨͷͰ͍͟ɺ࣮ߦʂ 
40. ४උ͕੔ͬͨͷͰ͍͟ɺ࣮ߦʂ  ࢦఆͨ͠ΠϯσοΫεͷ಺༰͕
    ҙਤ௨ΓऔಘͰ͖͍ͯΔ
41. ϛογϣϯɾίϯϓϦʔτ  ແࣄɺLambda -> ϩʔΧϧʹߏங͞Εͨ OpenSearch΁ΞΫηεͰ͖·ͨ͠ 🎉
42. ΞδΣϯμ   "NB[PO
    0QFO4FBSDI
    4FSWJDF
    
    -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ
     ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங
     $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ -BNCEBͷ઀ଓ
     ·ͱΊ
43. ຊηογϣϯͷ·ͱΊ  w"NB[PO
    0QFO4FBSDI
    4FSWJDF
    
    -BNCEBΛར༻ͨ͠શจ ݕࡧγεςϜͷߏஙྫ
    w։ൃ࣌ͷίετ཈੍ͷͨΊʹɺϩʔΧϧ؀ڥʹߏஙͨ͠ 0QFO4FBSDIΛར༻͢Δ
    w$MPVEqBSFEΛར༻͢ΔͱɺϩʔΧϧͷ0QFO4FBSDIͱ -BNCEBΛ؆୯ʹ઀ଓ͢Δ͜ͱ͕Ͱ͖Δ
44. ৄࡉͳखॱ͸ϒϩάʹ΋·ͱΊ͍ͯ·͢