Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
try to scraping with linebot
Search
Gin
February 13, 2020
Technology
1
240
try to scraping with linebot
Gin
February 13, 2020
Tweet
Share
More Decks by Gin
See All by Gin
Cで理解するソケットプログラミング
gin2525
0
490
論理と感覚
gin2525
0
210
Other Decks in Technology
See All in Technology
DevFest 2024 Incheon / Songdo - Compose UI 조합 심화
wisemuji
0
160
終了の危機にあった15年続くWebサービスを全力で存続させる - phpcon2024
yositosi
27
23k
Work as an App Engineer
lycorp_recruit_jp
0
370
事業貢献を考えるための技術改善の目標設計と改善実績 / Targeted design of technical improvements to consider business contribution and improvement performance
oomatomo
0
160
How to be an AWS Community Builder | 君もAWS Community Builderになろう!〜2024 冬 CB募集直前対策編?!〜
coosuke
PRO
2
2.9k
サーバーなしでWordPress運用、できますよ。
sogaoh
PRO
0
120
Wantedly での Datadog 活用事例
bgpat
2
700
APIとはなにか
mikanichinose
0
110
スタートアップで取り組んでいるAzureとMicrosoft 365のセキュリティ対策/How to Improve Azure and Microsoft 365 Security at Startup
yuj1osm
0
240
ハイテク休憩
sat
PRO
2
180
オプトインカメラ:UWB測位を応用したオプトイン型のカメラ計測
matthewlujp
0
210
組織に自動テストを書く文化を根付かせる戦略(2024冬版) / Building Automated Test Culture 2024 Winter Edition
twada
PRO
18
5.6k
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.1k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
28
2.1k
The Pragmatic Product Professional
lauravandoore
32
6.3k
The Power of CSS Pseudo Elements
geoffreycrofte
73
5.4k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
365
25k
A designer walks into a library…
pauljervisheath
205
24k
Building Applications with DynamoDB
mza
91
6.1k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
132
33k
Facilitating Awesome Meetings
lara
50
6.1k
Put a Button on it: Removing Barriers to Going Fast.
kastner
59
3.6k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.3k
Transcript
LINEBOT With Scraping twitter : @gin2_5 Created at 2020/02
ࣗݾհ ֶੜʢ̎̍ࡀʣ LINE FUKUOKAʹ ɹɹɹɹɹظΠϯλʔϯ AI, IoT(ϥζύΠ) ,Java ɹɹɹɹɹɹͲΕ࠳தɻ ࢿ֨ɿԠ༻ใٕज़ऀ
࠷ۙɺ ػցֶशͷֶश ΛαϘͬͯ·͢ɻ ࣜʹർΕ·ͨ͠ɻ ॏճؼͰଉΕͰ͢ɻ
Կ͔ɺଉൈ͖Ͱ ؔ࿈͋Δ͜ͱ͍ͨ͠ͳɻ
εΫϨΠϐϯάͬͯ ໘നͦ͏ɻ
ҰԠɺεΫϨΠϐϯάʹ͍ͭͯ “ WebεΫϨΠϐϯάͱɺWebαΠτ͔Β WebϖʔδͷHTMLσʔλΛऩूͯ͠ɺಛ ఆͷσʔλΛநग़ɺܗ͢͜͠ͱͰ͋Δ ” (weblio ༷ΑΓҾ༻)
HTML File Python Something (DB,File, and more) Scrape Output About
scraping
Α͘ɺ͜Μͳ͜ͱ͋Γ·ͤΜ͔ʁ ໘നͦ͏ͳITܥͷΠϕϯτΛݟಀͪ͠Όͬͨɻ ؾ͍ͨΒɺఆһ͍ͬͺ͍ɻ ʢओʹͰ͢ʣ
࡞Γ·ͨ͠ LINEͰΠϕϯτΛ ௨ɾݕࡧͯ͘͠ΕΔౕ
heroku ߏ DB LINE API Flask python cron (ఆظతʹಈ͘) python
Connpassͷ ެࣜαΠτ (HTMLϑΝΠϧ)
݁ߏ؆୯ʹεΫϨΠϐϯάग़དྷͪΌ͍·ͨ͠ $ pip install beautifulsoup4 from bs4 import BeautifulSoup import
requests bash scrape.py Install Import
url = ’https:!//something.com' r = requests.get(url) soup = BeautifulSoup(r.content, "html.parser")
events_name = soup.select('a.url.summary') events_date = soup.select('p.date') events_year = soup.select('p.year') events_img = soup.select('p.event_thumbnail img') scrape.py
ͨͩɺਅ໘ͳɻ Web APIΛ͍ͬͯΔΘ͚Ͱͳ͍ͷͰɺ ϖʔδͷߏΛม͑ΒΕͨΒɺऴΘΓ·͢ɻ ࣗવݴޠॲཧͱ͔བྷΊͯͬͯΈ͍ͨͳ
͓͠·͍ɻ ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ
None
!<- ͜Εɺ શ֯ʮzʴhʯ Ͱೖྗग़དྷ·͢ ͓·͚