Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
FastAPI を活用した オープンデータAPI の作成
Search
Yuuki Shimizu
September 11, 2021
Programming
0
860
FastAPI を活用した オープンデータAPI の作成
2021.9.11
Python Charity Talks in Japan 2021.09
Yuuki Shimizu
September 11, 2021
Tweet
Share
More Decks by Yuuki Shimizu
See All by Yuuki Shimizu
Gemini API を試した話と山梨でオススメのものを語るLT
yuukis
0
41
山梨のIT勉強会コミュニティを盛り上げるためにアレ作ってみた!
yuukis
0
29
FastAPIで始める やまなし温泉めぐり
yuukis
0
380
Slack の困りごとを何とかしてみた①
yuukis
0
81
Pythonでslackの困りごとを何とかしてみた①
yuukis
0
250
Pythonでお絵かき①
yuukis
0
310
Pythonを使ってみんなでワイワイしたい
yuukis
0
170
Pythonで始めるお手軽ダイエット(2)
yuukis
0
340
PythonでフィットネスデータをIFTTTに繋いでみた
yuukis
1
260
Other Decks in Programming
See All in Programming
コード生成を伴うLLMエージェント - 2024.07.18 Tokyo AI
smiyawaki0820
11
4.1k
【Go言語】ジェネリクス
tomo1227
0
170
生成AIをkintoneに連携してみた
hideg
0
230
feature環境をGitHub ActionsとCloudFormationでいい感じに管理する
nealle
2
310
Prompt FlowによるLLMアプリケーション開発
yuto2000
1
1k
CSC307 Lecture 11
javiergs
PRO
0
240
小さな開発会社を作った理由
polidog
0
1.9k
ぼっちを避けて楽しむためのアノテコノテ / Various Tips and Tricks to Avoid Loneliness and Have Fun
nrslib
3
1.7k
Clean Architecture by TypeScript & NestJS
ryounasso
0
150
CSC307 Lecture 10
javiergs
PRO
0
310
AWS初心者ってどうやってAWSを学ぶ?〜アプリエンジニアがやってよかったアーキテクチャ学習方法〜
yamanashi_ren01
0
190
Exploring the Gradually Lost Technical Skills in the Cloud Native Era
hwchiu
2
3.9k
Featured
See All Featured
Pencils Down: Stop Designing & Start Developing
hursman
118
11k
5 minutes of I Can Smell Your CMS
philhawksworth
200
19k
BBQ
matthewcrist
82
9k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
155
14k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
149
45k
Navigating Team Friction
lara
181
13k
Imperfection Machines: The Place of Print at Facebook
scottboms
262
13k
Testing 201, or: Great Expectations
jmmastey
33
6.9k
RailsConf 2023
tenderlove
16
720
Building Effective Engineering Teams - LeadDev
addyosmani
47
2.2k
The Brand Is Dead. Long Live the Brand.
mthomps
52
36k
What's in a price? How to price your products and services
michaelherold
239
11k
Transcript
2021. 9. 11 ͠Έͣ Ώ͏͖ 1 ʲPython Charity Talks in
Japan 2021.09ʳ FastAPI Λ׆༻ͨ͠ ΦʔϓϯσʔλAPI ͷ࡞
͓·͑୭Αʁ ͠Έͣ Ώ͏͖ • ϞόΠϧΞϓϦ ΤϯδχΞ ◦ Android/iOS • ࢁསݝߕࢢ
ग़ • Python Shingen.py Ͱ৮ΕΔ͘Β͍ 2
sli.do Ͱ࣭Λड͚͚͍ͯ·͢ʂ 3
͡Ίʹ • ݱࡏɺࢁསݝͷΦʔϓϯσʔλαΠτͰ 12,000݅Λ͑Δσʔλ͕ެ։͞Ε͍ͯΔ • ϞόΠϧΞϓϦͰར༻͍ͨ͠߹ɺWeb API Ͱఏڙ͞Ε͍ͯΔͱखܰʹࢼ͢͜ͱ͕Ͱ͖ͯ خ͍͠ •
Shingen.py ͷษڧձͰ FastAPI Λࢼ͢ػձ͕ ͋ΓɺΦʔϓϯσʔλར༻ʹ׆༻Ͱ͖ͳ͍͔ ݕ౼ͨ͠ 4
FastAPI ͱ • OpenAPI ʹج͍ͮͯ࡞ΒΕ͍ͯ Δ Python ϑϨʔϜϫʔΫ • ࡞Γ͢͞Λҙࣝ
• ʮૣ͍ɾ͍ɾ؆୯ʯ ◦ ։ൃ͕ૣ͍ ◦ ͦΕͳΓʹύϑΥʔϚϯε͕ग़Δ ʢ͍ʣ ◦ ؆୯ʹ࡞ΕΔ 5 from typing import Optional from fastapi import FastAPI app = FastAPI() @app.get("/") def read_root(): return {"Hello": "World"} @app.get("/items/{item_id}") def read_item(item_id: int, q: Optional[str] = None): return return {"item_id": item_id, "q": q}
ྫͱͯ͠ɺ ࢁསͷԹઘࢪઃΛฦ٫͢ΔAPI Λ FastAPI Ͱ࡞͢Δ 6 ͪͳΈʹ ࢁས ઘ࣭͕ଟ༷ɺܠ؍ͷྑ͍Թઘ͕ͱͯଟ͍Ͱ͢ʂ
ݝͷΦʔϓϯσʔλΛར༻ 7
8 PDF !!
PDF Ͱେৎʂ PDF Λͦͷ··ϩʔυͯ͠มޙɺ FastAPI Ͱฦ٫͢ΔΑ͏࣮͠·ͨ͠ 9 Python ศརͰ͢Ͷ
chezou/tabula-py • PDF ϑΝΠϧͷදΛ pandas ͷ DataFrame ΦϒδΣΫτʹ ม͢ΔϥΠϒϥϦ ◦
CSVɺTSVɺJSON ϑΝΠϧʹม ͢Δ͜ͱՄೳ • OCR πʔϧͰͳ͍ • Java 8 Ҏ͕߱ඞཁ 10
main.py - ᶃ PDF ಡΈࠐΈ def check_columns(df, previous_df): difference1 =
set(df.keys()) - set(previous_df.keys()) difference2 = set(previous_df.keys()) - set(df.keys()) return (len(difference1) == 0 and len(difference2) == 0) 11 def get_data(pdf_path): previous_df = pd.DataFrame() dfs = tabula.read_pdf(pdf_path, lattice=True, pages = 'all') for df in dfs: # ෳϖʔδͷදΛ݁߹͢Δ if (check_columns(df, previous_df)): df = pd.concat([previous_df, df]) previous_df = df return previous_df PDFΛಡΈࠐΈɺDataFrame Φϒ δΣΫτΛฦ٫͢Δ ෳϖʔδʹ·͕ͨΔදͷ໊߲ Λൺֱ͠ɺಉ͡ද͔Ͳ͏͔Λఆ ͢Δʢ্ͷ͔ؔΒݺΕΔʣ
main.py - ᶄ API ࡞ 12 app = FastAPI() pdf_path
= "h3012011.pdf" @app.get("/") def read_root(): data = get_data(pdf_path) json_data = data.to_json(orient = 'records') return json.loads(json_data) @app.get("/area/{area}") def read_item(area: str): data = get_data(pdf_path) df_mask = data['ࢢொଜ໊'] == area data = data[df_mask] json_data = data.to_json(orient = 'records') return json.loads(json_data) [get] / શ݅ฦ٫͢Δ API [get] /area/{area} ࢦఆ͞ΕͨࢢொଜͷΈฦ٫͢Δ API
Docker Λ༻ 13 VPS Nginx (ϦόʔεϓϩΩγ) opendata.yamanashi.dev /api/onsen Docker コンテナ
FastAPI localhost:xxxxx main.py ࢁསݝ WebαΠτ CSV PDF tiangolo/uvicorn-gunicorn-fastapi :python3.8-alpine3.10 ্هͷΠϝʔδΛϕʔεʹ openjdk11 ΛΠϯετʔϧ ͨ͠ͷΛ༻
DEMO 14 https://opendata.yamanashi.dev/api/onsen
ࢁསݝΦʔϓϯσʔλAPIϓϩδΣΫτ • ϓϩδΣΫτ αΠτ ◦ ݝͷΦʔϓϯσʔλαΠτͰެ։ ͞Ε͍ͯΔσʔλΛ API Ͱఏڙ •
GitHub ◦ ιʔείʔυΛެ։ ◦ ߋ৽͢Δ͜ͱͰࣗಈσϓϩΠ • DockerHub ◦ FastAPIɺTabula ͕࣮ߦՄೳͳ Docker ΠϝʔδΛఏڙ 15 https://opendata.yamanashi.dev
ຊϓϩδΣΫτ͕ࢦ͢ͱ͜Ζ 16 • ଞͷΦʔϓϯσʔλͷAPIల։Λ༰қʹ͢Δ ◦ ϦϙδτϦΛෳ͠ɺmain.py ΛΧελϚΠζ͢Ε OK • ΦʔϓϯσʔλAPIαʔόͷى্ͪ͛Λ༰қʹ͢Δ
◦ Docker ڥ͕͋ΕϫϯϥΠφʔͰى্ͪ͛Մೳ σʔλར༻ʹ͍ͭͯ ࢁསݝΦʔϓϯσʔλαΠτར༻ن ʹै͏ඞཁ͕͋Γ·͢
͍͞͝ʹ 17
·ͱΊ 18 ʮFastAPI Λ׆༻ͨ͠ ΦʔϓϯσʔλAPI ͷ࡞ʯ • FastAPI Φʔϓϯσʔλ͔Βखܰʹ API
Λ࡞͢Δͷʹ͍͍ͯΔ ◦ Tabula ͱΈ߹ΘͤΔ͜ͱͰɺPDF ϑΝΠϧ FastAPI ʹࡌͤΔ͜ͱ͕ Մೳ • ࢁསݝΦʔϓϯσʔλAPI ϓϩδΣΫτΛى্ͪ͛ ◦ API࡞ɾAPIαʔόى্ͪ͛Λ༰қʹ͠ɺΦʔϓϯσʔλͷར༻ଅਐʹ ܨ͍͛ͨ