Share
Embed
生成AIとPythonで ラズパイカーを動かそう! 〜GradioとLangChainで作るプロトタイプ〜
2026/2/19開催の「みんなのPython勉強会#120」での発表資料です。
個人で気軽に試せる趣味プロジェクトの例として、生成AIを使った、音声や画像で操作できるRaspberry Pi Pico搭載の小さな車を紹介。実際のコードを交えて、仕組みや利用しているライブラリ(LangChain、Gradio、Microdot など)を説明しました。
イベントページ:
https://startpython.connpass.com/event/382895/
GitHub:
https://github.com/komo-fr/llm-pico-car
X(Twitter):
https://x.com/komo_fr
komo_fr
February 19, 2026
Transcript
77 slides
-
9!LPNP@GS w 5PNPLP'VSVLJ w 9ʢ5XJUUFSʣ!LPNP@GS w ॴଐגࣜձࣾϏʔϓϥυ w ීஈ1ZUIPOΛͬͨσʔλαΠΤϯεͷٕज़ࢧԉɺ γεςϜ։ൃͳͲʹैࣄ
w ࠷ۙͷϒʔϜ--.Λͬͨখ͞ͳझຯϓϩάϥϜΛ࡞ͬͨΓ͢Δ͜ͱ "CPVUNF -
9!LPNP@GS w ͭͭͷٕज़ͱͬͯ؆୯ʂʢνϡʔτϦΞϧϨϕϧʂʣ w (SBEJPػցֶशͷσϞΞϓϦͳͲΛ؆୯ʹ࡞ΕΔ8FCΞϓϦͷϑϨʔϜ ϫʔΫɻԻೖྗը૾ೖྗ؆୯ʹ࡞ΕΔ w -BOH$IBJO--.ΞϓϦ։ൃͷϑϨʔϜϫʔΫ w .JDSPEPU3BTQCFSSZ1J1JDP্Ͱಈܰ͘ྔͷ8FCϑϨʔϜϫʔΫ
👉ੜ"*ͱ*P5ΛΈ߹ΘͤΔΞΠσΞͷώϯτʹͳΕ͏Ε͍͠Ͱ͢ ࠓͷϙΠϯτ -
9!LPNP@GS w ੜ"*Λ͑ɺࣗવݴޠˠߏԽ͞Εͨࢦࣔσʔλʹ؆୯ʹมͰ͖ͦ͏ w ϓϩϯϓτͷΠϝʔδ ʮࣗવݴޠʹΑΔࢦࣔΛɺࣙॻͷϦετܗࣜͰίϚϯυʹม͠ͳ͍͞ʯ ·͙ͬ͢ߦͬͯ ӈʹۂ͕ͬͯʂ [ {
"action": { "type": "forward", "distance_cm": 20,}}, { "action": { "type": "turn", "direction": "right", "angle": 90,}}, ...] ࣗવݴޠ ߏԽ͞ΕͨࢦࣔσʔλʢࣙॻͷϦετʣ લʹDNਐΉ ӈʹճస ੜ"* ΞΠσΞ ԿΛ࡞ͬͨͷ͔ -
9!LPNP@GS w ,JUSPOJL3BTQCFSSZ1J1JDP༻ࣗϩϘοτϓϥοτϑΥʔϜʢόΪʔλΠϓʣ w IUUQTLJUSPOJLDPVLQSPEVDUTBVUPOPNPVTSPCPUJDTQMBUGPSNGPSQJDP ࠓճະ༻͚ͲҎԼ͍ͭͯΔ w ڑηϯα w αʔϘίωΫλ
w ϥΠϯτϨʔε༻ηϯα ंྠ͖Ϟʔλʢݸʣ -&% ϒβʔ -&% ࠓճͬͨػମʢ1JDP$BSʣ Ͳ͏࡞ͬͨͷ͔ 3BTQCFSSZ1J1JDP -
9!LPNP@GS w ిࢠ࡞ͷܦݧ͕͋·Γͳ͍ͷͰɺࢢൢͷΩοτΛ͍͍ͨ w ϦνϜΠΦϯిཧ͕େมͦ͏ͳͷͰɺסిͰಈ͔͍ͨ͠ w 3BTQCFSSZ1J❌סి͚ͩͰಈ͔͢ͷݫ͍͠ w 3BTPCFSSZ1J1JDP✅סి͚ͩͰಈ͘ʂ 👉3BTQCFSZ1J1JDP
סిͰಈ͘ຊΩοτʹͨ͠ બఆͷཧ༝ Ͳ͏࡞ͬͨͷ͔ 3BTQCFSSZ1J1JDP8) XJUI8J'JBOEIFBEFSQJOT -
9!LPNP@GS 8FCΞϓϦ͔Β ԻΛೖྗ ίϚϯυΛड৴ ࣮ߦ ԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม Իσʔλ
ߏԽ͞Εͨ ࢦࣔσʔλ 4UFQ 4UFQ 4UFQ શମͷߏ Ͳ͏࡞ͬͨͷ͔ εϚϗͳͲͷ 8FCϒϥβ 1JDP$POUSPMMFS4FSWFS ʢ.BDʣ 1JDP$BS4FSWFS ʢ3BTQCFSSZ1J1*DP$BSʣ -
9!LPNP@GS εϚϗͳͲͷ 8FCϒϥβ 1JDP$POUSPMMFS4FSWFS ʢ.BDʣ 1JDP$BS4FSWFS ʢ3BTQCFSSZ1J1*DP$BSʣ 8FCΞϓϦ͔Β ԻΛೖྗ ίϚϯυΛड৴
࣮ߦ ԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม Իσʔλ ߏԽ͞Εͨ ࢦࣔσʔλ 4UFQ 4UFQ 4UFQ 4UFQ8FCΞϓϦ͔ΒԻΛೖྗ -
9!LPNP@GS Իೖྗίϯϙʔωϯτ ૹ৴ϘλϯϦηοτϘλϯ จࣈى͜͠ͷ݁ՌΛදࣔ͢Δ ςΩετϘοΫε ݱࡏͷεςʔλεΛදࣔ͢Δ ςΩετϘοΫε ߏԽ͞Εͨࢦࣔσʔλ ʢࣙॻͷϦετʣΛ දࣔ͢ΔΤϦΞ
ਐḿϥϯϓ ࠓճ࡞ͬͨ6* 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS Իೖྗίϯϙʔωϯτ ૹ৴ϘλϯϦηοτϘλϯ จࣈى͜͠ͷ݁ՌΛදࣔ͢Δ ςΩετϘοΫε ݱࡏͷεςʔλεΛදࣔ͢Δ ςΩετϘοΫε ߏԽ͞Εͨࢦࣔσʔλ ʢࣙॻͷϦετʣΛ දࣔ͢ΔΤϦΞ
ਐḿϥϯϓ ࠓճ࡞ͬͨ6* 4UFQ8FCΞϓϦ͔ΒԻೖྗ ೖྗΤϦΞ -
9!LPNP@GS Իೖྗίϯϙʔωϯτ ૹ৴ϘλϯϦηοτϘλϯ จࣈى͜͠ͷ݁ՌΛදࣔ͢Δ ςΩετϘοΫε ݱࡏͷεςʔλεΛදࣔ͢Δ ςΩετϘοΫε ߏԽ͞Εͨࢦࣔσʔλ ʢࣙॻͷϦετʣΛ දࣔ͢ΔΤϦΞ
ਐḿϥϯϓ ࠓճ࡞ͬͨ6* 4UFQ8FCΞϓϦ͔ΒԻೖྗ ग़ྗΤϦΞ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Audio Instructions"): # ίϯϙʔωϯτΛஔ audio_input = gr.Audio( sources=["microphone"], type="filepath", format="wav") with gr.Row(): audio_send_button = gr.Button("Submit", variant="primary") audio_reset_button = gr.Button("Reset", variant="secondary") ... # ΠϕϯτΛઃఆ audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps,],) ... # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) HS"VEJP HS#VUUPO HS#VUUPO HS5FYUCPY HS5FYUCPY HS+40/ TUSVDUVSFEBDUJPOT ίʔυ(SBEJP#MPDLT HS)5.- 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ audio_input = gr.Audio( sources=["microphone"], type="filepath", format="wav") with gr.Row(): audio_send_button = gr.Button("Submit", variant="primary") audio_reset_button = gr.Button("Reset", variant="secondary") audio_transcript_output = gr.Textbox(label="Transcription Result") audio_status_output = gr.Textbox(label="Processing Status") audio_commands_output = gr.JSON(label=“Generated Commandas") # ΠϕϯτΛઃఆ ... # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυը໘ϨΠΞτ HS"VEJP HS#VUUPO HS#VUUPO HS5FYUCPY HS5FYUCPY HS+40/ TUSVDUVSFEBDUJPOT HS)5.- 4UFQ8FCΞϓϦ͔ΒԻೖྗ Իೖྗ༻ ίϯϙʔωϯτ Ϙλϯ ݁Ռग़ྗ༻ͷ ίϯϙʔωϯτ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ audio_input = gr.Audio( sources=["microphone"], type="filepath", format="wav") with gr.Row(): audio_send_button = gr.Button("Submit", variant="primary") audio_reset_button = gr.Button("Reset", variant="secondary") audio_transcript_output = gr.Textbox(label="Transcription Result") audio_status_output = gr.Textbox(label="Processing Status") audio_commands_output = gr.JSON(label=“Generated Commandas") # ΠϕϯτΛઃఆ ... # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυը໘ϨΠΞτ HS"VEJP HS#VUUPO HS#VUUPO HS5FYUCPY HS5FYUCPY HS+40/ TUSVDUVSFEBDUJPOT HS)5.- 4UFQ8FCΞϓϦ͔ΒԻೖྗ Իೖྗ༻ ίϯϙʔωϯτ Ϙλϯ ݁Ռग़ྗ༻ͷ ίϯϙʔωϯτ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ audio_input = gr.Audio( sources=["microphone"], type="filepath", format="wav") with gr.Row(): audio_send_button = gr.Button("Submit", variant="primary") audio_reset_button = gr.Button("Reset", variant="secondary") audio_transcript_output = gr.Textbox(label="Transcription Result") audio_status_output = gr.Textbox(label="Processing Status") audio_commands_output = gr.JSON(label=“Generated Commandas") # ΠϕϯτΛઃఆ ... # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυը໘ϨΠΞτ HS"VEJP HS#VUUPO HS#VUUPO HS5FYUCPY HS5FYUCPY HS+40/ TUSVDUVSFEBDUJPOT HS)5.- 4UFQ8FCΞϓϦ͔ΒԻೖྗ Իೖྗ༻ ίϯϙʔωϯτ Ϙλϯ ݁Ռग़ྗ༻ͷ ίϯϙʔωϯτ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ … # ΠϕϯτΛઃఆ audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps,],) audio_reset_button.click(...) audio_input.start_recording(...) audio_input.stop_recording(...) # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυΠϕϯτϋϯυϥ $MJDL 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ ... # ΠϕϯτΛઃఆ audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps,],) audio_reset_button.click(...) audio_input.start_recording(...) audio_input.stop_recording(...) # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυΠϕϯτϋϯυϥ ೖྗ ग़ྗ ΫϦοΫ 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ ... # Πϕϯτͷઃఆ audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps,],) audio_reset_button.click(...) audio_input.start_recording(...) audio_input.stop_recording(...) # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυΠϕϯτϋϯυϥ .click(handler, inputs=..., outputs=...) audio_transcript_output ΫϦοΫ࣌ʹ࣮ߦ͞ΕΔؔ w Իࢦࣔͷจࣈى͜͠ w ੜ"*ʹΑΔίϚϯυมॲཧ w 1JDPΧʔʹίϚϯυૹ৴ w ʜ audio_status_output audio_commands_output audio_input 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS import gradio as gr ... with gr.Blocks() as demo:
... with gr.Tab("Voice Input"): # ίϯϙʔωϯτΛஔ ... # ΠϕϯτΛઃఆ audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps,],) audio_reset_button.click(...) audio_input.start_recording(...) audio_input.stop_recording(...) # αʔόΛىಈ demo.launch(server_name="0.0.0.0", share=True) ίʔυαʔόͷىಈ 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS audio_send_button.click(handler_audio_input, inputs=audio_input, outputs=[audio_transcript_output, audio_commands_output, audio_status_output, progress_lamps]) def handler_audio_input(audio_file): success,
text = transcribe(audio_file) if success: yield text, None, "จࣈىྃ͜͠", render_lamps(["s", "u", "u"]) commands = text_to_command(text) commands = convert_commands(commands) yield text, commands, "ίϚϯυੜྃ", render_lamps(["s", "s", "u"]) response = send_commands(commands) status = response.get("status") yield text, commands, status, render_lamps(["s"] * 3) else: yield text, None, "จࣈى͜͠ʹࣦഊ͠·ͨ͠ɻऴྃ͠·͢", render_lamps(["e", "u", "u"]) w (SBEJPͰʮஈ֊తʹ݁ՌΛฦ͢ʯ6*؆୯ʹ࡞ΕΔ w ΠϕϯτϋϯυϥͷதͰyieldΛͬͯ݁ՌΛฦ͢ ᶈͯ͢ͷ݁ՌΛฦ͢ Πϕϯτϋϯυϥ ग़ྗઌͷίϯϙʔωϯτ ᶃจࣈى͜͠Λ࣮ߦ ᶅੜ"*ΛͬͯɺจࣈྻΛߏԽ͞ΕͨίϚϯυʹม͢Δ ᶇ1JDPΧʔʹίϚϯυΛૹ৴ ZJMFEΛͬͯஈ֊తʹ݁ՌΛฦ͢ ᶆੜ"*͕มͨ͠ίϚϯυΛฦ͢ ᶄจࣈى͜͠ͷ݁ՌΛฦ͢ 4UFQ8FCΞϓϦ͔ΒԻೖྗ -
9!LPNP@GS 1JDP$BS4FSWFS ʢ3BTQCFSSZ1J1*DP$BSʣ εϚϗͳͲͷ 8FCϒϥβ 8FCΞϓϦ͔Β ԻΛೖྗ ίϚϯυΛड৴ ࣮ߦ
ԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม 4UFQ 4UFQ 4UFQ ߏԽ͞Εͨ ࢦࣔσʔλ Իσʔλ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม 1JDP$POUSPMMFS4FSWFS ʢ.BDʣ -
9!LPNP@GS w Իͷ··ੜ"*ʹೖྗ͢Δͱ"1*ྉۚͷίετ͕͔͔Γͦ͏ 👉ϩʔΧϧͰGBTUFSXIJTQFSΛͬͯจࣈى͜͠Λ͢Δ from faster_whisper import WhisperModel model_size =
"tiny" model = WhisperModel(model_size, device="cpu", compute_type="float32") …ུ… segments, _ = model.transcribe(tmp_audio_path, language="ja") text = "".join([segment.text for segment in segments]) ᶃϞσϧͷಡΈࠐΈ ᶄϞσϧͷ࣮ߦʢจࣈى͜͠ʣ ԻˠςΩετʢจࣈى͜͠ʣ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS w Իͷ··ੜ"*ʹೖྗ͢Δͱ"1*ྉۚͷίετ͕͔͔Γͦ͏ 👉ϩʔΧϧͰGBTUFSXIJTQFSΛͬͯจࣈى͜͠Λ͢Δ from faster_whisper import WhisperModel model_size =
"tiny" model = WhisperModel(model_size, device="cpu", compute_type="float32") …ུ… segments, _ = model.transcribe(tmp_audio_path, language="ja") text = "".join([segment.text for segment in segments]) ᶃϞσϧͷಡΈࠐΈ ᶄϞσϧͷ࣮ߦʢจࣈى͜͠ʣ ԻˠςΩετʢจࣈى͜͠ʣ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS w -BOH$IBJO w ʹൃද͞Εͨɺ--.ΞϓϦ։ൃͷͨΊͷ1ZUIPOϑϨʔϜϫʔΫ w -BOH$IBJOܦ༝Ͱ0QFO"*ͷ"1*ʢHQUʣΛݺͼग़͠ɺ ࣗવݴޠͷࢦࣔΛɺߏԽ͞ΕͨࢦࣔσʔλʢࣙॻͷϦετʣʹม͢Δ ·͙ͬ͢ߦͬͯ ӈʹۂ͕ͬͯʂ
[{ "action": { "type": "forward", "distance_cm": 20,}}, { "action": { "type": "turn", "direction": "right", "angle": 90,}}, ...] ࣗવݴޠ ߏԽ͞ΕͨࢦࣔσʔλʢࣙॻͷϦετʣ લʹDNਐΉ ӈʹճస ੜ"* ςΩετˠߏԽ͞Εͨࢦࣔσʔλ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # Add RGB color info based on the LED color name (CSS color name) in the generated data instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ᶃϓϩϯϓτςϯϓϨʔτͷߏங ᶄϞσϧͷॳظԽ ᶅϓϩϯϓτͱϞσϧΛܨ͙ ᶆJOWPLFϝιουͰ࣮ߦ ίʔυ-BOH$IBJOͷجຊ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ᶃϓϩϯϓτςϯϓϨʔτͷߏங ᶄϞσϧͷॳظԽ ᶅϓϩϯϓτͱϞσϧΛܨ͙ ᶆJOWPLFϝιουͰ࣮ߦ ίʔυ-BOH$IBJOͷجຊ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ᶃϓϩϯϓτςϯϓϨʔτͷߏங ᶄϞσϧͷॳظԽ ᶅϓϩϯϓτͱϞσϧΛܨ͙ ᶆJOWPLFϝιουͰ࣮ߦ ίʔυ-BOH$IBJOͷجຊ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ᶃϓϩϯϓτςϯϓϨʔτͷߏங ᶄϞσϧͷॳظԽ ᶅϓϩϯϓτͱϞσϧΛܨ͙ ᶆJOWPLFϝιουͰ࣮ߦ ίʔυ-BOH$IBJOͷجຊ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ᶃϓϩϯϓτςϯϓϨʔτͷߏங ᶄϞσϧͷॳظԽ ᶅϓϩϯϓτͱϞσϧΛܨ͙ ᶆJOWPLFϝιουͰ࣮ߦ ίʔυ-BOH$IBJOͷجຊ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ༩͑ΒΕͨຊޠͷࢦࣔจΛɺpicoΧʔ༻ͷJSONܗࣜͷಈ࡞ίϚ ϯυྻʹม͍ͯͩ͘͠͞ɻ ࢦࣔͰͳ͍ݴ༿ΛݴΘΕͨ߹ʢʮ͓Α͏ʂʯʮ͔ʂʯͳͲʣ ɺݴΘΕͨݴ༿ʹର͢ΔײΛද͢ಈ࡞ͷίϚϯυྻʹม͠ ͍ͯͩ͘͞ɻ ίϚϯυྻ͚ͩΛग़ྗ͠ɺͦΕҎ֎ͷઆ໌จͳͲؚΊͳ͍Ͱ͘ ͍ͩ͞ɻ ֤ίϚϯυҎԼͷܗࣜΛ༻͍ͯͩ͘͠͞: … ϓϩϯϓτ 1SPNQU 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS from langchain.chat_models import init_chat_model from langchain_core.prompts import ChatPromptTemplate ...
prompt = ChatPromptTemplate( [ ("system", TEXT_TO_COMMAND_PROMPT), ("human", "ࢦࣔ: {text}"), ] ) llm = init_chat_model(model="gpt-4.1", temperature=0) chain = prompt | llm result = chain.invoke({"text": text}) instructions = result.content # CSS৭໊ΛRGBʹมͯ͠Ճ instructions = _add_led_rgb(instructions) instructions = json.loads(instructions) ༩͑ΒΕͨຊޠͷࢦࣔจΛɺpicoΧʔ༻ͷJSONܗࣜͷಈ࡞ίϚ ϯυྻʹม͍ͯͩ͘͠͞ɻ ࢦࣔͰͳ͍ݴ༿ΛݴΘΕͨ߹ʢʮ͓Α͏ʂʯʮ͔ʂʯͳͲʣ ɺݴΘΕͨݴ༿ʹର͢ΔײΛද͢ಈ࡞ͷίϚϯυྻʹม͠ ͍ͯͩ͘͞ɻ ίϚϯυྻ͚ͩΛग़ྗ͠ɺͦΕҎ֎ͷઆ໌จͳͲؚΊͳ͍Ͱ͘ ͍ͩ͞ɻ ֤ίϚϯυҎԼͷܗࣜΛ༻͍ͯͩ͘͠͞: … 1SPNQU ϓϩϯϓτ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS ͋ͳͨՄѪ͍ΘͶʙ w ᐆດͳݴ༿ΛݴΘΕͨͱ͖ɺͦͷݴ༿ʹର͢ΔײΛද͢ಈ࡞ɾ৭ʹ ม͢ΔΑ͏ϓϩϯϓτͰࢦࣔ 👉ᐆດͳݴ༿ΛݴΘΕͨ࣌ɺԿΒ͔ͷʮੜ͖Β͍͠ԠʯΛͯ͘͠ΕΔ ͔͔ʂ Γ͠Α͏ ϐϯΫͰલਐͯ͠ࠨӈʹ;Γ;Γ ʢتͼʁʣ
੨৭Ͱޙୀʢ൵͠Έʁʣ Ͱલਐʢߠఆʁʣ ϓϩϯϓτͷ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS ͋ͳͨՄѪ͍ΘͶʙ w ᐆດͳݴ༿ΛݴΘΕͨͱ͖ɺͦͷݴ༿ʹର͢ΔײΛද͢ಈ࡞ɾ৭ʹ ม͢ΔΑ͏ϓϩϯϓτͰࢦࣔ 👉ᐆດͳݴ༿ΛݴΘΕͨ࣌ɺԿΒ͔ͷʮੜ͖Β͍͠ԠʯΛͯ͘͠ΕΔ ͔͔ʂ Γ͠Α͏ ϐϯΫͰલਐͯ͠ࠨӈʹ;Γ;Γ ʢتͼʁʣ
੨৭Ͱޙୀʢ൵͠Έʁʣ Ͱલਐʢߠఆʁʣ ϓϩϯϓτͷ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม ϐϯΫͰલਐͯ͠ࠨӈʹ;Γ;Γʢتͼʁʣ ϓϩϯϓτͰɺʮ൵͠Έ੨Ͱද͢ʯ ͱ͍ͬͨࢦࣔ༩͍͑ͯͳ͍ ੜ"*͕ࣗͰࣗવͳ৭ΛબͿͷͰ ෯͍ݴ༿ʹॊೈʹԠͰ͖Δ -
9!LPNP@GS [ { "action": {"type": "forward", "distance_cm": 20}, "led": {"color":
"green"}}, { "action": {"type": "turn", "direction": "right", "angle": 90}, "led": {"color": "green"}}, ...] BDUJPOӈճస MFE BDUJPOલਐ DN MFE ͭͷίϚϯυ ͭͷίϚϯυ ੜ͞ΕΔࢦࣔσʔλ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS [ { "action": {"type": "forward", "distance_cm": 20}, "led": {"color":
"green", "rgb": (0, 255, 0) }}, { "action": {"type": "turn", "direction": "right", "angle": 90}, "led": {"color": "green", "rgb": (0, 255, 0)}}, ...] $44৭໊Λ3(#ʹม w ੜ"*$44ͷ৭໊͚ͩΛग़ྗɻޙॲཧͰ1ZUIPOͰ3(#ʹม͢Δ w ৭໊ͷํ͕ײΛදͤͦ͏ ਓ͕ؒݟͯཧղ͍͢͠ͷͰɺ "*ʹ৭໊Λग़ྗͤͨ͞ w $44৭໊͔Β3(#ͷมɺγϯϓϧͰ࣮֬ʹΒͤΔͨΊ1ZUIPOʹͤͨ ੜ͞ΕΔࢦࣔσʔλ 4UFQԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม -
9!LPNP@GS 1JDP$POUSPMMFS4FSWFS ʢ.BDʣ εϚϗͳͲͷ 8FCϒϥβ 8FCΞϓϦ͔Β ԻΛೖྗ ίϚϯυΛड৴ ࣮ߦ
ԻΛจࣈى͜͠ ੜ"*Ͱࢦࣔσʔλʹม 4UFQ 4UFQ 4UFQ ߏԽ͞Εͨ ࢦࣔσʔλ Իσʔλ 4UFQίϚϯυΛड৴ ࣮ߦ 1JDP$BS4FSWFS ʢ3BTQCFSSZ1J1*DP$BSʣ -
9!LPNP@GS w 3BTQCFSSZ1J1JDPͰඪ४ͷ$1ZUIPOಈ࡞͠ͳ͍ͷͰ .JDSP1ZUIPOΛ͏ w .JDSPEPUΛͬͯ8FC"1*αʔόΛ࡞ w 'MBTLʹணΛಘͨϛχϚϧͳ8FCϑϨʔϜϫʔΫ w $1ZUIPOͱ.JDSP1ZUIPOΛαϙʔτ͍ͯ͠ΔͷͰɺ1JDP্Ͱಈ͔ͤΔ
w ࠓճͷ༻్ͷ߹ɺmicrodot.pyʢ,#ʣΛ1JDP্ʹஔ͚ͩ͘Ͱ ͑Δ 1JDP্Ͱ8FC"1*αʔόΛಈ͔͢ 4UFQίϚϯυΛड৴ ࣮ߦ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ 4UFQίϚϯυΛड৴ ࣮ߦ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ .JDSPEPUΞϓϦΛ࡞ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ 4UFQίϚϯυΛड৴ ࣮ߦ ಈ࡞֬ೝ͍͢͠Α͏ ઃఆͰϞοΫͱ࣮ػΛΓସ͑ .JDSPEPUΞϓϦΛ࡞ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ .JDSPEPUΞϓϦΛ࡞ 4UFQίϚϯυΛड৴ ࣮ߦ ಈ࡞֬ೝ͍͢͠Α͏ ઃఆͰϞοΫͱ࣮ػΛΓସ͑ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ .JDSPEPUΞϓϦΛ࡞ 4UFQίϚϯυΛड৴ ࣮ߦ 'MBTLͬΆ͍ߏจ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ .JDSPEPUΞϓϦΛ࡞ 4UFQίϚϯυΛड৴ ࣮ߦ 'MBTLͬΆ͍ߏจ -
9!LPNP@GS from microdot import Microdot from pico_car import Car, RealCar
car = Car() if settings.USE_MOCK else RealCar() app = Microdot() ... @app.route("/command", methods=["POST"]) async def handle_command(request): try: car.sound_freq(1000, 0.1) car.sound_freq(1200, 0.1) command_list = request.json car.run_commands(command_list) return {"status": "ok"} except Exception as e: handle_buggy_error() print_exception(e) return {"status": "error", "message": str(e)}, 500 ... app.run(port=5001) ᶃड৴ԻΛ໐Β͢ ࣗ࡞ͷ1JDPΧʔΫϥεͷΠϯελϯεΛੜ ᶄίϚϯυΛ࣮ߦ Τϥʔൃੜ࣌ͷॲཧʢΤϥʔԻΛ໐Β͢ɺϞʔλఀࢭͳͲʣ αʔόΛىಈ ίʔυ1JDP্ͷ.JDSPEPUαʔό ίϚϯυΛड৴࣮ͯ͠ߦ͢Δ"1*ͷఆٛ .JDSPEPUΞϓϦΛ࡞ 4UFQίϚϯυΛड৴ ࣮ߦ -
9!LPNP@GS w ୯७ʹforϧʔϓͰίϚϯυΛॱʹ࣮ߦ w ίϚϯυʹԠͯ͡Ϟʔλ-&%Λ੍ޚ͢Δ w Ϟʔλʔ-&%ͷ੍ޚʹɺ1JDPΧʔͷΩοτ༻ͷϥΠϒϥϦΛ༻ w IUUQTHJUIVCDPN,JUSPOJL-UE,JUSPOJL1JDP"VUPOPNPVT3PCPUJDT1MBUGPSN.JDSP1ZUIPO ίϚϯυΛॱ൪ʹॲཧ
4UFQίϚϯυΛड৴ ࣮ߦ from PicoAutonomousRobotics import KitronikPicoRobotBuggy buggy = KitronikPicoRobotBuggy() ... buggy.motorOn("l", "f", SPEED) # ࠨंྠ: લਐ buggy.motorOn("r", "f", SPEED) # ӈंྠ: લਐ buggy.motorOn("l", "f", SPEED) # ࠨंྠ: લਐ buggy.motorOn("r", "r", SPEED) # ӈंྠ: ޙਐ લਐ ӈճస Ϟʔλͷ੍ޚྫ -
9!LPNP@GS w ੜ"*Y1JDPɺ·ͩ·ͩԠ༻͢Δͱ໘ന͍ͷ͕Ͱ͖ͦ͏ w 1JDPΧʔͷΞΫγϣϯΛ૿͢ w ྫ-&%ͷޫΓํͷύλʔϯΛ૿͢ɺαʔϘϞʔλͰ৲ඌΛ༳Β͢ w ϓϩϯϓτͰ1JDPΧʔͷੑ֨ΛΧελϚΠζ w
ྫʮ͔͔ʂʯͱݴΘΕͨ࣌ʹ൵͠ΉλΠϓ͔ɺౖΔλΠϓ͔ ʢ-&%͕੨PSʣ ΞΠσΞ ݁Ռ -
9!LPNP@GS w (SBEJPͰ࡞ͬͨ6*͔Βɺखඳ͖Ͱϧʔτը૾Λೖྗ w 4ελʔτɺ(ΰʔϧ w ϓϩϯϓτͱҰॹʹը૾Λੜ"*ʹ͢ Ωϟϯόε্ʹϧʔτΛඳ͍ͯ ೖྗ ʢgr.ImageEditorʣ
༩͑ΒΕͨը૾ʹϧʔτ͕ඳ͔Ε͍ͯ·͢ɻ S͕ελʔτͰɺG͕ΰʔϧͰ͢ɻ ͜ͷϧʔτը૾ΛɺpicoΧʔ༻ͷJSONܗࣜͷ ಈ࡞ίϚϯυྻʹม͍ͯͩ͘͠͞ɻ …ུ… ϓϩϯϓτ ϧʔτը૾Λ༩͑ͯ1JDPΧʔΛಈ͔͢ Ԡ༻ը૾ʹΑΔࢦࣔ -
9!LPNP@GS NJDSPCJUΛ͏߹ͷ੍ Ԡ༻ྫNJDSPCJUͱͷ࿈ܞ Իσʔλ ߏԽσʔλ εϚϗͳͲͷ 8FCϒϥβ $POUSPMMFS4FSWFS ʢ.BDʣ $BS4FSWFS
w ͜͜·Ͱઆ໌ͨ͠3BTQCFSSZ1J1JDP༻ͷΈͦͷ··Ͱ͑ͳ͍ w NJDSPCJU୯ମʹ8J'J௨৴ͷػೳ͕ͳ͍ w ͦͷͨΊɺNJDSPCJU্ʹ8FC"1*αʔόΛཱͯΔͷݱ࣮తͰͳ͍ NJDSPCJU -
9!LPNP@GS NJDSPCJU༻ͷߏ Ԡ༻ྫNJDSPCJUͱͷ࿈ܞ Իσʔλ ߏԽ σʔλ εϚϗͳͲͷ 8FCϒϥβ $POUSPMMFS4FSWFS w
.BDͱNJDSPCJU༗ઢͩͱγϦΞϧ௨৴Ͱ؆୯ʹσʔλަͰ͖Δ w NJDSPCJUಉ࢜ແઢͰ؆୯ʹ௨৴Ͱ͖Δ w 👉ͦͷͨΊɺ.BDʹதܧ༻ͷNJDSPCJUΛ64#Ͱܨ͗ ͦͷதܧ༻NJDSPCJUͱϩϘοτ্ͷNJDSPCJUΛ௨৴͢Δ $BS4FSWFS .BD NJDSPCJUᶃ NJDSPCJUᶄ จࣈྻσʔλ จࣈྻσʔλ 64# .BD γϦΞϧ௨৴ ແઢ -
9!LPNP@GS w ੜ"*Λར༻ͯ͠ɺԻը૾Ͱಈ͓ͪ͘Όͷंͷ࡞ΓํΛհ w --.Ͱɺࠓ·Ͱ͔ͬͨ͠ॲཧΛ؆୯ʹ࣮ݱͰ͖ΔΑ͏ʹͳͬͨ w (SBEJPͩͱɺԻը૾ೖྗΛ؆୯ʹѻ͑Δ w -BOH$IBJOͰɺ؆୯ʹੜ"*ΛΞϓϦʹΈࠐΊΔ w
ੜ"*Y1JDPɺ·ͩ·ͩԠ༻͢Δͱ໘ന͍ͷ͕Ͱ͖ͦ͏ ·ͱΊ -
9!LPNP@GS 5IBOLZPVGPSMJTUFOJOH ࢿྉͱϦϙδτϦΛެ։͍ͯ͠·͢ (JU)VC3FQPTJUPSZ 4MJEFT 4MJEFT IUUQTTQFBLFSEFDLDPNLPNPGSTUBQZMMNQJDPDBS (JU)VCSFQPTJUPSZ IUUQTHJUIVCDPNLPNPGSMMNQJDPDBS %FNP
IUUQTXXXZPVUVCFDPNXBUDI WHM&C&;TTX3" 9 5XJUUFS !LPNP@GS