Slide 1
Slide 1 text
RBS meets LLMs
Leaner Technologies, Inc.
黒曜
(@kokuyouwind)
~ Type inference using LLM ~
Slide 2
Slide 2 text
Talk: Japanese
Slides: English (+ Japanese)
Slide 3
Slide 3 text
$ whoami
黒曜 / @kokuyouwind
Work at: Leaner Technologies, Inc.
Ruby Sponsor
Day 1: Sponsor Talk (Done)
Day 2: Leaner Drinkup
Slide 4
Slide 4 text
The idea is covered by Matz's Keynote,
but this is the first time you've heard of it!
Matz
のKeynote
で出た話と発想が被っていますが、
初めて聞いたことにしてください！
Slide 5
Slide 5 text
Large Language Model (LLM)
Machine learning models trained on large amounts of text data.
ChatGPT Bard Bing Chat
⼤量のテキストデータを使ってトレーニングされた機械学習モデル
⼤規模⾔語モデル
Slide 6
Slide 6 text
Example: ChatGPT
Slide 7
Slide 7 text
Can we do something fun with this?
🤔
なにか⾯⽩いことに使えないかな？
Slide 8
Slide 8 text
Type Inference
user = User.new
# `user` is an User type variable
name = user.name
# `name` is a String type variable(?)
1
2
3
4
5
Humans sometimes infer type from word meanings.
⼈間は単語の意味から型を推測することがある
Slide 9
Slide 9 text
With ChatGPT,
can we infer the type
from the meaning of a word?
ChatGPT
を使えば、単語の意味から
型を推測できるのでは？
Slide 10
Slide 10 text
Let's see some cases!
Tool Settings
model: ChatGPT(gpt-3.5-turbo)
Temperature: 0
GUI: BetterChatGPT
Slide 11
Slide 11 text
Case 1: User Builder
user = UserBuilder.new.name('kokuyouwind').build
# We infer types as follows:
# * UserBuilder#name returns an UserBuilder
# * UserBuilder#build returns an User
1
2
3
4
5
Slide 12
Slide 12 text
Case 1: User Builder
Slide 13
Slide 13 text
Case 1: User Builder
Slide 14
Slide 14 text
(There is extra output, but)
Perfect!!!
👏
Slide 15
Slide 15 text
Case 2: Company Repository
company = CompanyRepository.new.find(1).name
# We infer types as follows:
# * CompanyRepository#find returns a Company
# * Company#name returns a String
# Note.
# This expression has the same syntactic form with:
# user = UserBuilder.new.name('kokuyouwind').build
1
2
3
4
5
6
7
8
9
Slide 16
Slide 16 text
Case 2: Company Repository
Slide 17
Slide 17 text
(There is extra output, but)
Perfect, again!!!
👏
Slide 18
Slide 18 text
Problems
RBS sometimes broken (
たまにRBS
の構⽂がおかしい)
Extra Outputs (
不要な出⼒がある)
Only RBS output needed (RBS
だけ出⼒させたい)
Single RBS (RBS
がひとまとめになっている)
Separate User's RBS to user.rbs, and so on
(User
のRBS
はuser.rbs
に、などクラスごとに分割したい)
Slide 19
Slide 19 text
Improve: FewShot
Demonstrate type inference for UserBuilder,
then ask about the CompanyRepository case.
UserBuilder
の模範解答を⼊⼒してから、CompanyRepository
について推論させる
Demo Input
Main Question
Output
模範解答
メインの問い合わせ
出⼒
Slide 20
Slide 20 text
Improve: FewShot
Demo Input Main Question
模範解答を⼊⼒ メインの問い合わせ
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
Really Perfect!!!
👏
Slide 23
Slide 23 text
Please talk to me at parties, etc.
ぜひドリンクアップなどで話しかけてください！
I've tried to infer types for metaprogramming,
refine existing RBS, etc.,
but I don't have time to talks. So,
メタプログラミングコードに対する型の推測や既存RBS
の詳細化なども試しましたが
今回は話す時間が⾜りないので、
Slide 24
Slide 24 text
Future Works
CommandLine Tool (
コマンドラインツール化)
Fine tuning for type inference (
型推測⽤のファインチューニング)
Agent-style autonomous drive (
エージェントスタイルの⾃律駆動)
Refer to the necessary files (
必要なファイルの参照)
Correction with type check results (
型チェックを元にした修正)
Run with LLMs locally (
ローカルで動くLLM
を使った動作)
Alpaca.cpp, ChatRWKV, or else