Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python & Google Cloud Platform

Python & Google Cloud Platform

Сергей Пронин (CTO @ App in the Air) @ Moscow Python Conf 2017
"В рамках доклада расскажу про шестилетний опыт работы на Python с AppEngine и Google Cloud Platform.
Отличие сервисов, вариантов хранения данных, интеграция между ними; использование Google APIs.
Плюсы / минусы / подводные камни, сравнение с другими облачными платформами.
Другими словами, «Как за 6 лет вырастить стартап с нуля до трех миллионов пользователей, не думая о scaling-е»".
Видео: https://conf.python.ru/python-google-cloud-platform/

Moscow Python Meetup

October 20, 2017
Tweet

More Decks by Moscow Python Meetup

Other Decks in Programming

Transcript

  1. Интро CTO, Co-Founder
 App in the Air Senior Developer
 Empatika

    Python, Swift, JS, Obj-C GCP user from 2010
 Программная инженерия НИУ ВШЭ
  2. Google Cloud Platform • Начиналось с AppEngine Java/Python • Полноценная

    облачная платформа • Fully-Managed — просто заплати • Полноценная поддержка любых конфигураций, начиная с самых простых
  3. AppEngine • PaaS на Google-инфрасткрутуре • Load Balancing, Auto-Scaling, Health

    Checking, Instances management • Front-End / Back-End • Полная интеграция с GCP сервисами • Storage, Memcache, Logs, Crons, Task Queue, Email
  4. Front-End • Дешёвые инстансы • 60 сек на запрос /

    600 сек на Task Queue • Нет background threads • Automatic Scaling • 28-instance часов бесплатно • Pure Python 2.7 на Standard Environment
  5. Front-End Type RAM CPU F1 128 MB 600 MHz F2

    256 MB 1.2 GHz F4 512 MB 2.4 GHz F4_1G 1024 MB 2.4 GHz
  6. Back-End • ∞ запрос / 24 ч. на Task Queue

    • Background threads • Manual / Basic Scaling • 8-instance часов бесплатно • Pure Python 2.7 на Standard Environment
  7. Back-End Type RAM CPU B1 128 MB 600 MHz B2

    256 MB 1.2 GHz B4 512 MB 2.4 GHz B4_1G 1024 MB 2.4 GHz B8 1024 MB 4.8 GHz
  8. Python Standard Env • Pure Python 2.7 libs • Runs

    in Sandbox, 0 instance min • Прямой доступ к GCP API • WSGI-compatible • webapp2, flask, django, jinja2 • lxml, MySQLdb, PIL, ssl, pycrypto, webob, werkzeug, …
  9. class UserAvatarHandler(webapp2.RequestHandler, APIHandler): @user_required def post(self): avatar_file = self.request.POST.get('avatar') avatar

    = self.user.add_avatar(avatar_file) if not avatar: return self.error("bad file", code=400) self.success({"avatar": avatar.to_dict()}) app = webapp2.WSGIApplication([ ('/', WebIndexHandler), ('/oauth/authorize', OAuthAuthorizeHandler), ('/oauth/token', OAuthTokenHandler) #...
 ])
  10. app.yaml application: my-app version: production runtime: python27 threadsafe: true api_version:

    1 handlers: - url: /_ah/mail/.+ script: myflights.app login: admin - url: .* script: main.app 
 libraries: - name: jinja2 version: latest
  11. Python Flexible Env • Custom Python 2/3 env • Custom

    Docker supported • Runs on VM, 1 instance min • GCP API через client libraries • Re-use инфраструктуры Standard AppEngine • Ваш собственный requirement.txt
  12. class Airport(ndb.Model): name = ndb.StringProperty() code = ndb.StringProperty() city =

    ndb.StringProperty() country = ndb.StringProperty() offset = ndb.FloatProperty() phone = ndb.StringProperty() url = ndb.StringProperty() loc = ndb.GeoPtProperty() airport_map = ndb.StructuredProperty(AirportMap) terminal_maps = ndb.StructuredProperty(AirportMap, repeated=True) @classmethod def get_by_code(cls, code): airport = memcache.get('airport-{}'.format(code.encode('utf-8'))) if not airport: airport = cls.query(cls.code == code).get() if airport: airport.set_memcache() return airport
 airport = Airport(name='Domodedovo', code=‘DME') airport.put()
  13. class Airport(ndb.Model): name = ndb.StringProperty() code = ndb.StringProperty() city =

    ndb.StringProperty() country = ndb.StringProperty() offset = ndb.FloatProperty() phone = ndb.StringProperty() url = ndb.StringProperty() loc = ndb.GeoPtProperty() airport_map = ndb.StructuredProperty(AirportMap) terminal_maps = ndb.StructuredProperty(AirportMap, repeated=True) @classmethod def get_by_code(cls, code): airport = memcache.get('airport-{}'.format(code.encode('utf-8'))) if not airport: airport = cls.query(cls.code == code).get() if airport: airport.set_memcache() return airport
 airport = Airport(name='Domodedovo', code=‘DME') airport.put()
  14. class Airport(ndb.Model): name = ndb.StringProperty() code = ndb.StringProperty() city =

    ndb.StringProperty() country = ndb.StringProperty() offset = ndb.FloatProperty() phone = ndb.StringProperty() url = ndb.StringProperty() loc = ndb.GeoPtProperty() airport_map = ndb.StructuredProperty(AirportMap) terminal_maps = ndb.StructuredProperty(AirportMap, repeated=True) @classmethod def get_by_code(cls, code): airport = memcache.get('airport-{}'.format(code.encode('utf-8'))) if not airport: airport = cls.query(cls.code == code).get() if airport: airport.set_memcache() return airport
 airport = Airport(name='Domodedovo', code='DME') airport.put()
  15. – Mattias P Johansson, Spotify “FINALLY I CAN TELL THE

    WORLD THAT BIGQUERY IS THE BEST THING THAT HAS EVER HAPPENED TO ME”
  16. Logging class LoggingHandler(webapp2.RequestHandler): def get(self): logging.debug('This is a debug message')

    logging.info('This is an info message') logging.warning('This is a warning message') logging.error('This is an error message') logging.critical('This is a critical message')
  17. Services • Compute Engine VMs by Google • Cloud Storage

    File storage • Container Engine kubernetes clusters • Spanner New generation SQL storage • Pub/Sub Messaging Center • ML Engine Run ML Models