Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
try to scraping with linebot
Search
Gin
February 13, 2020
Technology
1
270
try to scraping with linebot
Gin
February 13, 2020
Tweet
Share
More Decks by Gin
See All by Gin
Cで理解するソケットプログラミング
gin2525
0
510
論理と感覚
gin2525
0
240
Other Decks in Technology
See All in Technology
Autonomous Database サービス・アップデート (FY25)
oracle4engineer
PRO
1
730
Amazon DevOps Guru のベースラインを整備して1ヶ月ほど運用してみた #jawsug_asa / Amazon DevOps Guru trial
masahirokawahara
3
210
データベースの引越しを Ora2Pg でスマートにやろう
jri_narita
0
180
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
6.7k
データ戦略部門 紹介資料
sansan33
PRO
1
3.1k
Spring for GraphQLって実際どうなの?〜小規模スタートアップの事例紹介〜
kogayushi
0
160
実践Kafka Streams 〜イベント駆動型アーキテクチャを添えて〜
joker1007
3
830
Oracle Cloud Infrastructure IaaS 新機能アップデート 2025/03 - 2025/05
oracle4engineer
PRO
1
130
生成AIの使い方講座 - 時短に効くTips集 -
mamodev
0
100
JavaのMCPサーバーで体験するAIエージェントの世界
tatsuya1bm
1
210
kintone開発組織のDevOpsへの移り変わりと実践
ueokande
1
280
プロジェクトマネージャーに最後まで残るたった一つの仕事は交渉
ichimichi
1
180
Featured
See All Featured
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
How STYLIGHT went responsive
nonsquared
100
5.6k
BBQ
matthewcrist
89
9.7k
Building Adaptive Systems
keathley
42
2.6k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.3k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Rails Girls Zürich Keynote
gr2m
94
13k
Docker and Python
trallard
44
3.4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
60k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
52
2.8k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
RailsConf 2023
tenderlove
30
1.1k
Transcript
LINEBOT With Scraping twitter : @gin2_5 Created at 2020/02
ࣗݾհ ֶੜʢ̎̍ࡀʣ LINE FUKUOKAʹ ɹɹɹɹɹظΠϯλʔϯ AI, IoT(ϥζύΠ) ,Java ɹɹɹɹɹɹͲΕ࠳தɻ ࢿ֨ɿԠ༻ใٕज़ऀ
࠷ۙɺ ػցֶशͷֶश ΛαϘͬͯ·͢ɻ ࣜʹർΕ·ͨ͠ɻ ॏճؼͰଉΕͰ͢ɻ
Կ͔ɺଉൈ͖Ͱ ؔ࿈͋Δ͜ͱ͍ͨ͠ͳɻ
εΫϨΠϐϯάͬͯ ໘നͦ͏ɻ
ҰԠɺεΫϨΠϐϯάʹ͍ͭͯ “ WebεΫϨΠϐϯάͱɺWebαΠτ͔Β WebϖʔδͷHTMLσʔλΛऩूͯ͠ɺಛ ఆͷσʔλΛநग़ɺܗ͢͜͠ͱͰ͋Δ ” (weblio ༷ΑΓҾ༻)
HTML File Python Something (DB,File, and more) Scrape Output About
scraping
Α͘ɺ͜Μͳ͜ͱ͋Γ·ͤΜ͔ʁ ໘നͦ͏ͳITܥͷΠϕϯτΛݟಀͪ͠Όͬͨɻ ؾ͍ͨΒɺఆһ͍ͬͺ͍ɻ ʢओʹͰ͢ʣ
࡞Γ·ͨ͠ LINEͰΠϕϯτΛ ௨ɾݕࡧͯ͘͠ΕΔౕ
heroku ߏ DB LINE API Flask python cron (ఆظతʹಈ͘) python
Connpassͷ ެࣜαΠτ (HTMLϑΝΠϧ)
݁ߏ؆୯ʹεΫϨΠϐϯάग़དྷͪΌ͍·ͨ͠ $ pip install beautifulsoup4 from bs4 import BeautifulSoup import
requests bash scrape.py Install Import
url = ’https:!//something.com' r = requests.get(url) soup = BeautifulSoup(r.content, "html.parser")
events_name = soup.select('a.url.summary') events_date = soup.select('p.date') events_year = soup.select('p.year') events_img = soup.select('p.event_thumbnail img') scrape.py
ͨͩɺਅ໘ͳɻ Web APIΛ͍ͬͯΔΘ͚Ͱͳ͍ͷͰɺ ϖʔδͷߏΛม͑ΒΕͨΒɺऴΘΓ·͢ɻ ࣗવݴޠॲཧͱ͔བྷΊͯͬͯΈ͍ͨͳ
͓͠·͍ɻ ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ
None
!<- ͜Εɺ શ֯ʮzʴhʯ Ͱೖྗग़དྷ·͢ ͓·͚