Function callingによるSlackbotの機能拡張

2023-09-01
ENECHANGE Tech Talk（社内勉強会）

Transcript

1. Function callingによるSlackbotの機能拡張
   2023-09-01
   ENECHANGE Tech Talk（社内勉強会）
   CTO室 岩本隆史
   

   View Slide

2. ChatGPT Botに画像生成機能を追加
   

   View Slide

3. まずcreate_image関数を実装
   def create_image(prompt, size="256x256"):
   """Creates an image given a prompt"""
   openai.api_key = os.environ["FUNCTION_CALL_OPENAI_API_KEY"]
   openai.api_type = "open_ai"
   openai.api_base = "https://api.openai.com/v1"
   openai.api_version = None
   image = openai.Image.create(prompt=prompt, size=size)
   http = urllib3.PoolManager()
   r = http.request("GET", image["data"][0]["url"], preload_content=False)
   key = _generate_object_key("images", "png")
   _upload_to_s3(r, key, "image/png")
   r.release_conn()
   image_info = {
   "prompt": prompt,
   "size": size,
   "image_url": _build_url_for_object(key),
   }
   return json.dumps(image_info)
   

   View Slide

4. そしてfunctions変数を定義
   functions = [
   {
   "name": "create_image",
   "description": "Creates an image given a prompt",
   "parameters": {
   "type": "object",
   "properties": {
   "prompt": {
   "type": "string",
   "description": "A text description in English of the desired image",
   },
   "size": {
   "type": "string",
   "description": "The size of the generated images",
   "enum": ["256x256", "512x512", "1024x1024"],
   },
   },
   "required": ["prompt"],
   },
   },
   ]
   

   View Slide

5. QRコード生成機能も追加
   

   View Slide

6. create_qrcode関数を実装
   def create_qrcode(url):
   """Creates a QR code given a URL"""
   img = qrcode.make(url)
   key = _generate_object_key("images", "png")
   with tempfile.NamedTemporaryFile() as tf:
   img.save(tf.name)
   _upload_to_s3(tf.name, key, "image/png")
   qrcode_info = {
   "url": url,
   "qrcode_image_url": _build_url_for_object(key),
   }
   return json.dumps(qrcode_info)
   

   View Slide

7. functionsを更新
   functions = [
   {
   "name": "create_image",
   ...
   },
   {
   "name": "create_qrcode",
   "description": "Creates a QR code given a URL",
   "parameters": {
   "type": "object",
   "properties": {
   "url": {
   "type": "string",
   "description": "The URL to encode in the QR code",
   },
   },
   "required": ["url"],
   },
   },
   ]
   

   View Slide

8. 音声読み上げ機能も追加
   

   View Slide

9. synthesize_speech関数を実装
   def synthesize_speech(text, gender="female"):
   """Synthesizes speech given text"""
   voice_id = {"female": "Tomoko", "male": "Takumi"}[gender]
   response = boto3.client("polly").synthesize_speech(
   VoiceId=voice_id,
   OutputFormat="mp3",
   Text=text,
   Engine="neural",
   )
   key = _generate_object_key("audios", "mp3")
   _upload_to_s3(response["AudioStream"], key, "audio/mpeg")
   speech_info = {
   "text": text,
   "gender": gender,
   "voice_id": voice_id,
   "audio_url": _build_url_for_object(key),
   }
   return json.dumps(speech_info)
   

   View Slide

10. functionsを更新
    functions = [
    ...
    {
    "name": "synthesize_speech",
    "description": "Synthesizes speech given text",
    "parameters": {
    "type": "object",
    "properties": {
    "text": {
    "type": "string",
    "description": "The text to synthesize",
    },
    "gender": {
    "type": "string",
    "description": "The gender of the voice",
    "enum": ["female", "male"],
    },
    },
    "required": ["text"],
    },
    },
    ]
    

    View Slide

11. 簡単に拡張できる背景
    

    View Slide

12. Function callingが登場 (6/13)
    https://openai.com/blog/function-calling-and-other-api-updates
    

    View Slide

13. 入力を関数呼び出しに変換
    Convert queries such as “Email Anya to see if she wants to get coffee next Friday”
    to a function call like send_email(to: string, body: string) , or “What’s the
    weather like in Boston?” to get_current_weather(location: string, unit:
    'celsius' | 'fahrenheit') .
    https://openai.com/blog/function-calling-and-other-api-updates
    

    View Slide

14. ChatGPT in SlackがFunction callingに対応 (8/21)
    # Experimental: You can try out the Function Calling feature (default: None)
    export OPENAI_FUNCTION_CALL_MODULE_NAME=tests.function_call_example
    https://github.com/seratch/ChatGPT-in-Slack#readme
    

    View Slide

15. 誰でも拡張可能
    

    View Slide

16. enechange-configにPRを作成
    例（QRコード生成分）：https://github.com/enechange/enechange-config/pull/2254
    

    View Slide

17. 関数とfunctions変数を実装
    https://github.com/enechange/enechange-config/blob/develop/chatgpt-in-
    slack/environment/common/function_call.py
    

    View Slide

18. 必要ならパッケージを追加
    ベース：https://github.com/seratch/ChatGPT-in-Slack/blob/main/requirements.txt
    slack-bolt>=1.18.0,<2
    openai>=0.27.9,<0.28
    tiktoken>=0.4,<0.5
    # https://github.com/Yelp/elastalert/issues/2306
    urllib3<2
    --
    追加分：https://github.com/enechange/enechange-config/blob/develop/chatgpt-in-
    slack/environment/common/requirements.txt（このファイルを編集）
    boto3 # Lambda関数も呼び出せる
    qrcode
    

    View Slide

19. ローカル環境構築は意外と簡単
    ChatGPTとSlack上で会話するアプリを10分で構築しよう（前編：ローカル環境）
    https://qiita.com/rubita/items/f143f199e20b0310980a
    

    View Slide

20. 余談：OSS貢献での学び
    

    View Slide

21. ChatGPT in SlackのFunction calling対応に貢献
    @iwamot Thanks for the contribution! Your changes are now merged into the
    main branch: 626e492
    https://github.com/seratch/ChatGPT-in-Slack/pull/52
    

    View Slide

22. issueで打診
    To further enhance this software, I am proposing the addition of Function Calling
    support.
    (snip)
    May I proceed to create a PR for this proposed addition?
    https://github.com/seratch/ChatGPT-in-Slack/issues/50
    

    View Slide

23. 変更を最小限にするのが重要
    The addition of function calling is certainly something I've been looking forward
    to.
    (snip)
    When you submit a pull request, keeping the changes minimal would be very
    helpful.
    https://github.com/seratch/ChatGPT-in-Slack/issues/50
    

    View Slide

24. まとめ
    

    View Slide

25. 機能拡張しよう
    ChatGPT Botの機能拡張PR、お待ちしてます！
    OSSの機能拡張も応援します！
    

    View Slide