Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloudflare + OpenSearch + Lambdaで始める全文検索開発入門

2bc6f91177ebb2c2681fe44f8522c681?s=47 amotz
October 05, 2021

Cloudflare + OpenSearch + Lambdaで始める全文検索開発入門

Developers.IO 2021 Decade で発表した資料です

2bc6f91177ebb2c2681fe44f8522c681?s=128

amotz

October 05, 2021
Tweet

Transcript

  1.  ৽نࣄۀ౷ׅ෦1SPqMZνʔϜ࣋ా३࢙ $MPVEqBSF 0QFO4FBSDI -BNCEBͰ࢝ΊΔ શจݕࡧ։ൃೖ໳

  2. ຊηογϣϯͷର৅ऀ  w"NB[PO0QFO4FBSDI4FSWJDF΍0QFO4FBSDIʹڵຯͷ͋ Δํ w"NB[PO0QFO4FBSDI4FSWJDFΛར༻͍ͯͯ͠ɺ։ൃίετ Λগ͠Ͱ΋཈͍͑ͨํ w։ൃதͷϩʔΧϧαʔόʔΛ֎෦ެ։ͯ͠ར༻͍ͨ͠ํ

  3. ຊηογϣϯͷΰʔϧ  AWS Lambda -> ϩʔΧϧʹߏங͞Εͨ OpenSearch΁ΞΫηεͰ͖ΔΑ͏ʹ͢Δ

  4. ຊηογϣϯͰ࿩͞ͳ͍͜ͱ  wશจݕࡧͷ࢓૊Έ΍ΞϧΰϦζϜ w0QFO4FBSDIͷΫΤϦͳͲͷৄࡉͳ಺༰ w-BNCEB΍αʔόʔϨεʹ͍ͭͯͷجૅతͳ࿩ w0QFO4FBSDIͱ&MBTUJDTFBSDIʹؔ͢ΔΑ΋΍·࿩

  5. ࣗݾ঺հ  ࣋ా३࢙!BNPU[ w৽نࣄۀ౷ׅ෦1SPqMZνʔϜ
 ιϑτ΢ΣΞΤϯδχΞ wௗऔͰϑϧϦϞʔτϫʔΫ w4BB4ϓϩμΫτͷاըɾ։ൃɾ ӡ༻

  6. ΞδΣϯμ   "NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ  ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங  $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ

    -BNCEBͷ઀ଓ  ·ͱΊ
  7. ΞδΣϯμ   "NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ  ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங  $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ

    -BNCEBͷ઀ଓ  ·ͱΊ
  8. ϓϩϑΟʔϧϏϡʔΞʔαʔϏεʮ1SPqMZʯ 

  9. ϓϩϑΟʔϧϏϡʔΞʔαʔϏεʮ1SPqMZʯ 

  10. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 

  11. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 

  12. 1SPqMZͷόοΫΤϯυΞʔΩςΫνϟʢҰ෦ʣ 

  13. 0QFO4FBSDI4FSWJDF -BNCEB͸खܰͰศར͚ͩͲɾɾ  0QFO4FBSDI4FSWJDFΛར༻͢Δ৔߹ʹײͨ͡՝୊ wݸਓ؀ڥͩͱΠϯελϯεͷίετ͕ؾʹͳΔ wUNJDSPλΠϓͰ΋݄׵ࢉ͢Δͱ໿💰 w՝ۚΛఀࢭ͢Δʹ͸Ұ୴υϝΠϯΛ࡟আ͢Δඞཁ͕͋Γɺ࠶ߏங͕໘౗

  14. ։ൃίετΛ΋͏গ͠཈͍͑ͨ   4".-PDBM΍-PDBM4UBDLΛ࢖ͬͯϩʔΧϧ؀ڥ Ͱಈ࡞֬ೝΛߦ͏  ݱঢ়ͷߏ੒͸ͦͷ··ʹɺ0QFO4FBSDI4FSWJDF ͚ͩΛϩʔΧϧͷ0QFO4FBSIʹஔ͖׵͑Δ

  15. ։ൃίετΛ΋͏গ͠཈͍͑ͨ   4".-PDBM΍-PDBM4UBDLΛ࢖ͬͯϩʔΧϧ؀ڥ Ͱಈ࡞֬ೝΛߦ͏  ݱঢ়ͷߏ੒͸ͦͷ··ʹɺ0QFO4FBSDI4FSWJDF ͚ͩΛϩʔΧϧͷ0QFO4FBSIʹஔ͖׵͑Δ ࠓճ͸ͪ͜ΒΛ࠾༻

  16. ։ൃ͢Δ࣌͸ϩʔΧϧ0QFO4FBSDIΛ࢖͏  ΍ͬͯΈΔ 💪

  17. ΞδΣϯμ   "NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ  ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங  $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ

    -BNCEBͷ઀ଓ  ·ͱΊ
  18. -BNCEB 0QFO4FBSDI4FSWJDFͷγϯϓϧͳྫ 

  19. -BNCEB ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 

  20. -BNCEB ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 

  21. %PDLFS*NBHFΛར༻ͨ͠0QFO4FBSDIߏங  $ docker pull opensearchproject/opensearch:1.0.1 $ docker pull opensearchproject/opensearch-dashboards:1.0.1

    https://opensearch.org/docs/opensearch/install/docker/
  22. %PDLFS*NBHFΛར༻ͨ͠0QFO4FBSDIߏங  version: '3' services: opensearch: image: opensearchproject/opensearch:1.0.1 container_name: opensearch

    environment: - cluster.name=docker-cluster - node.name=os-node - cluster.initial_master_nodes=os-node - bootstrap.memory_lock=true - http.host=0.0.0.0 - transport.host=127.0.0.1 - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "plugins.security.disabled=true" ulimits: memlock: soft: -1 hard: -1 volumes: - $PWD/.local/opensearch:/usr/share/opensearch/data ports: - 9200:9200 networks: - sandbox networks: sandbox: docker-compose.yml
  23. %PDLFS*NBHFΛར༻ͨ͠0QFO4FBSDIߏங  version: '3' services: opensearch: image: opensearchproject/opensearch:1.0.1 container_name: opensearch

    environment: - cluster.name=docker-cluster - node.name=os-node - cluster.initial_master_nodes=os-node - bootstrap.memory_lock=true - http.host=0.0.0.0 - transport.host=127.0.0.1 - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "plugins.security.disabled=true" ulimits: memlock: soft: -1 hard: -1 volumes: - $PWD/.local/opensearch:/usr/share/opensearch/data ports: - 9200:9200 networks: - sandbox networks: sandbox: docker-compose.yml ࠓճ͸ݕূͷͨΊ ηΩϡϦςΟϓϥάΠϯΛແޮԽ
  24. %PDLFS*NBHFΛར༻ͨ͠0QFO4FBSDIߏங  $ docker-compose up -d .. Creating network “local_sandbox"

    with the default driver Creating opensearch ... done Creating opensearch-dashboards ... done
  25. ΞδΣϯμ   "NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ  ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங  $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ

    -BNCEBͷ઀ଓ  ·ͱΊ
  26. -BNCEB ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 

  27. -BNCEB ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 

  28. $MPVEqBSF"SHP5VOOFM$MJFOU w"SHP5VOOFMʢ5VOOFMJOH%BFNPOʣͷίϚϯυϥΠϯΫϥ ΠΞϯτ w$MPVEqBSFͷωοτϫʔΫΛհͯ͠ɺ೚ҙͷϩʔΧϧαʔό ΛϓϩΩγͰ͖Δ $MPVEqBSFEͱ͸ 

  29. $MPVEqBSFEͱ͸  $MPVEqBSFEΛར༻͢ΔϝϦοτ w։ൃ༻్Ͱ͋Ε͹ɺແঈ͔ͭαΠϯΞοϓෆཁͰ͙͢ʹར༻ Մೳ wݻఆͷΧελϜυϝΠϯΛઃఆ͢Δ͜ͱ΋Ͱ͖Δ wୠ͠ɺ$MPVEqBSF΁ͷαΠϯΞοϓ͓ΑͼυϝΠϯઃఆ͕ඞཁ

  30. $MPVEqBSFEͷΠϯετʔϧʢ.BD04ʣ  $ brew install cloudflare/cloudflare/cloudflared https://developers.cloudflare.com/cloudflare-one/connections/ connect-apps/install-and-setup/installation

  31. $ cloudflared tunnel --url localhost:9200 $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel

    --url localhost:9200 ϩʔΧϧͷ0QFO4FBSDIΛ ࢦఆ͢Δ
  32. $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel --url localhost:9200 INF Requesting new

    quick Tunnel on trycloudflare.com... INF +---------------------------------------------------------------+ INF | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): | INF | https://female-attract-average-join.trycloudflare.com | INF +---------------------------------------------------------------+ ..
  33. $MPVEqBSFEΛىಈͯ͠0QFO4FBSDIΛ֎෦ެ։  $ cloudflared tunnel --url localhost:9200 INF Requesting new

    quick Tunnel on trycloudflare.com... INF +---------------------------------------------------------------+ INF | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): | INF | https://female-attract-average-join.trycloudflare.com | INF +---------------------------------------------------------------+ .. ϥϯμϜʹੜ੒͞Εͨ63-͕ ׂΓ౰ͯΒΕΔ
  34. -BNCEB ϩʔΧϧͰಈ࡞͢Δ0QFO4FBSDI 

  35. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT

    = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js
  36. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT

    = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js 0QFO4FBSDIͷ +4༻ΫϥΠΞϯτΛར༻
  37. -BNCEBͷιʔείʔυྫʢ/PEFYʣ  const { Client } = require('@opensearch-project/opensearch'); const OPENSEARCH_ENDPOINT

    = process.env.OPENSEARCH_ENDPOINT || ""; const client = new Client({ node: OPENSEARCH_ENDPOINT }); exports.handler = async (event) => { const result = await client.search({ index: 'devio_sample', pretty: true }); return result; }; /index.js ؀ڥม਺ͰΤϯυϙΠϯτΛ ઃఆͰ͖ΔΑ͏ʹ͓ͯ͘͠
  38. -BNCEBͷ؀ڥม਺ʹΤϯυϙΠϯτΛઃఆ  $MPVEqBSFEͰ ׂΓ౰ͯΒΕͨ63-Λࢦఆ

  39. ४උ͕੔ͬͨͷͰ͍͟ɺ࣮ߦʂ 

  40. ४උ͕੔ͬͨͷͰ͍͟ɺ࣮ߦʂ  ࢦఆͨ͠ΠϯσοΫεͷ಺༰͕ ҙਤ௨ΓऔಘͰ͖͍ͯΔ

  41. ϛογϣϯɾίϯϓϦʔτ  ແࣄɺLambda -> ϩʔΧϧʹߏங͞Εͨ OpenSearch΁ΞΫηεͰ͖·ͨ͠ 🎉

  42. ΞδΣϯμ   "NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛ૊ Έ߹Θͤͨશจݕࡧ  ։ൃͷͨΊͷϩʔΧϧ؀ڥ0QFO4FBSDIߏங  $MPVEqBSFΛར༻ͨ͠ϩʔΧϧ0QFO4FBSDIͱ

    -BNCEBͷ઀ଓ  ·ͱΊ
  43. ຊηογϣϯͷ·ͱΊ  w"NB[PO0QFO4FBSDI4FSWJDF -BNCEBΛར༻ͨ͠શจ ݕࡧγεςϜͷߏஙྫ w։ൃ࣌ͷίετ཈੍ͷͨΊʹɺϩʔΧϧ؀ڥʹߏஙͨ͠ 0QFO4FBSDIΛར༻͢Δ w$MPVEqBSFEΛར༻͢ΔͱɺϩʔΧϧͷ0QFO4FBSDIͱ -BNCEBΛ؆୯ʹ઀ଓ͢Δ͜ͱ͕Ͱ͖Δ

  44. ৄࡉͳखॱ͸ϒϩάʹ΋·ͱΊ͍ͯ·͢ 

  45. None