Share
Embed
Transcript
30 slides
-
ࢸͬͯγϯϓϧͰ͢. 1. Geocodingͯ͠ٿͷҐஔใΛग़͢. ۩ମతʹٿ໊͔ΒҢܦΛग़͢. 2. 1.ͷσʔλΛݩʹ, ٿؒͷڑΛग़͢ 3. 2.ΛεέδϡʔϧͱJOIN, νʔϜ͝ͱʹूܭ,
CSVग़ྗ. ࠓ1.ͱ2.ΛPythonͰ͍͍ײ͡ʹ͢Δํ๏Λ. ※3.టष͍PandasՔۀͳͷͰࠓճઆ໌͠·ͤΜ -
GeoPy • PythonͰGeocoding͢Δͱ͖ͷఆ൪ϥΠϒϥϦ • ෳͷΠϯλʔωοτਤʢGoogle, Azure, OSM, etc…ʣͷAPIΛಉ͡Α͏ͳίʔυͰPython͔Βѻ͑Δ • ެࣜυΩϡϝϯτ͕ৄ͍͠ͷͰਅࣅ͢Εେମ͍͚Δ
• https://geopy.readthedocs.io/en/stable/# -
GeoPyͰٿ໊͔ΒGeocoding • MLBͷSean Lahman Databaseʹ ٿσʔλ͕͋Δʢͳ͓, Φʔϓϯσʔλʣ • ٿͷ໊લͱࢢ໊Λ࣋ͬͯΔͷͰ, ͔ͦͬΒ
Geocodingͯ͋͛͠ΕOK • શମͷ7ׂ͜ΕͰΠέͨ, Γख࡞ۀ(ry -
ʲงғؾʳGeoPyͰGeocoding import csv import time from geopy.geocoders import Nominatim #
Geocoder(ͲͷαʔϏε͏͔)ࢦఆ from geopy.exc import GeocoderTimedOut from retry import retry # ࠓճOSMϕʔεͷͷΛ͏ geoLocator = Nominatim(user_agent='Baseball Radar24 / 0.1 [email protected]’) # Geocoding͍ͯ͠Δͱ͜Ζ. งғؾΛݟͯRetry @retry((GeocoderTimedOut, ), delay=5, backoff=2, max_delay=4) def get_location(name, alias): loc = geoLocator.geocode(name) if not loc: loc = geoLocator.geocode(alias) return loc # ٿ໊ΛGeocodingͰ͖ΔΑ͏ʹͪΐͬ͜ͱ͚ͩΫϨϯδϯά def park_name(name): return name.replace('I', '').replace('II', '').replace('III', '').replace('IV', '').strip() Nominatimͱ͍͏OSMσʔλͷAPIͰGeocoding ٿ໊geocodersʹؾʹೖΒΕΔΑ͏ʁʹͪΐͬͱ͚ͩΫϨϯδϯά -
ʲงғؾʳGeoPyͰGeocoding # ͔ͬ͜Β࣮ߦ # ٿϦετ values = [] with open('./datasets/baseballdatabank/Parks.csv',
'r') as f: reader = csv.DictReader(f) for r in reader: values.append(r) # GeocodingΛͻͨ͢Β࣮ߦ locations = [] for park in values: loc = get_location(park_name(park['park.name']), park_name(park['park.alias'])) if loc: locations.append( { 'id': park['park.key'], 'name': park['park.name'], 'lat': loc.latitude, 'lng': loc.longitude, 'address': loc.address, 'state': park['state'], 'country': park['country'] } ) else: print('geo not found: ', park['park.name'], park['park.key']) # CSVʹॻ͖ࠐΈ fields = ['id', 'name', 'lat', 'lng', 'address', 'state', 'country'] with open('./datasets/parklist.csv', 'w') as f: writer = csv.DictWriter(f, fieldnames=fields) writer.writeheader() for loc in locations: writer.writerow(loc) CSVΛಡΈࠐΜͰͻͨ͢ΒGeocoding͢ΔʢલͷεϥΠυͷؔͰʣ ͜ͷลׂͱී௨ͷεΫϦϓτͩͬͨΓ͢ΔͷͰײత͔. -
ʲงғؾʳGeoPyͰೋؒڑ # ڑΛग़͢ from geopy.distance import great_circle, geodesic def park2park_distance_datasets(self,
park_datasets: dict) -> list: values = [] for id1, park1 in park_datasets.items(): for id2, park2 in park_datasets.items(): if id1 == id2: continue park1_geo = (park1.get('lat'), park1.get('lng')) park2_geo = (park2.get('lat'), park2.get(‘lng')) # geodesic͕ଌઢ, great_circle͕େԁڑ values.append( { 'id': f"{id1}_{id2}", 'miles_geo': geodesic(park1_geo, park2_geo).mile 'miles_circle': great_circle(park1_geo, park2_geo).mile } ) return values geopy.distanceͷؔΛ͏, ଌઢ, େԁڑϝιου͕͍͔ͭ͘. ҾଌΓ͍ͨڑͷlat/lngೖͬͨtuple -
ݕূํ๏ • ؒͷҠಈڑͱओཁࢦඪͷϚτϦΫεΛग़͢ • উ • ಘࣦࠩ • ༧ଌউʢϐλΰϥεউʣ ※ಘࣦ͔ΒউΛग़͢
• ͳʹ͔ۙͦ͏ͳͷ͕ݟ͔ͭͬͨΒϥοΩʔ • Ռ࣮ͨͯ͠ࡍ…ʂʁ -
Who am I?ʢ͓લ୭Αʣ • Shinichi Nakagawa(@shinyorke) • ͔ͭͯٿΤϯδχΞΛࣄʹͯͨ͠ਓ • ઌ݄·ͰɿʮϓϩʯͷٿΤϯδχΞ
• ࠓ݄͔ΒɿʮੜʯͷٿΤϯδχΞʢʹ෮ؼʣ • Pythonࣗ͘͘शࣨʢ#rettypyʣΦʔΨφΠβʔ • Web, σʔλαΠΤϯε, Opsʹ⽁ΛPythonͰΔਓ -
JX௨৴ࣾʢʹస৬ͯ͠·ͨ͠ʣ • ࠓ݄͔ΒʢגʣJX௨৴ࣾͷSenior Engineerʹ • σʔλج൫ΛθϩϕʔεͰ্ཱͪ͛Δࣄ ʢଞ, Pythonؔ࿈ͷ͋Ε͜Εɾ࠾༻ใͳͲʣ • స৬ͷܦҢɾϙΤϜϒϩάʹͯ
https://shinyorke.hatenablog.com/entry/it-really- could-happen -
#ؒืूத • αʔόʔαΠυɾϑϩϯτΤϯυɾػցֶश ৄ͘͠ https://jobs.jxpress.net/ • ߇͑ΊʹݴͬͯΊͬͪΌPythonͰ͢ʢ͜ͳΈʣ • Serverlessͱ͔Big Dataͱ͔ϝονϟઓͰ͖·͢
• ॻ੶, IDE, ษڧձࢀՃඅձࣾෛ୲, #PyConJP εϙϯαʔଞ • ؾʹͳΔํੋඇ͔͚ͯͶʂ -
ʲAppendixʳͬͨͷҰཡ • σʔλੳ • Jupyter notebook / Jupyter Lab https://jupyter.org/
• Pandas https://pandas.pydata.org/ • Plotly https://plot.ly/python/ • GIS • GeoPyʢGeocodingʣ https://geopy.readthedocs.io/en/stable/ • FoliumʢJupyter notebookਤʣ https://python-visualization.github.io/folium/ • Kepler.glʢVisualizationʣ https://kepler.gl/ • ⚾ ٿɹ˞ͯ͢MLBͰ͢ • Baseball Databank https://github.com/chadwickbureau/baseballdatabank • Retrosheet https://github.com/chadwickbureau/retrosheet • Analyzing Baseball Data with Rʢॻ੶,༸ॻʣ https://www.amazon.co.jp/dp/B07KRNP2BB