연구자 및 교육자를 위한 계산 및 분석 플랫폼 설계 - PyCon KR 2015

현대 과학 연구에는 컴퓨터를 이용한 계산 및 분석 작업이 필수적입니다. 그러나 거대 스케일의 계산 및 분석 작업을 수행할 경우 컴퓨팅 리소스의 적절한 관리 및 확장 용이성을 확보하는 것은 많은 리소스를 필요로 합니다. 우리는 컴퓨터 계산 작업 및 분석 작업을 표준화하고 클라우드에서 처리하는 파이썬3 기반의 오픈소스 플랫폼을 설계 및 개발하고 있습니다. 또한 이 플랫폼 위에서 돌아가는 교육 / 연구 플랫폼을 함께 설계하고 있습니다.

새로운 서비스를 변화하는 환경에 맞추어 개발하는 일은 즐거운 경험인 동시에, "무엇을" "어떻게" "왜" 로 이어지는 지뢰밭을 거니는 일이기도 합니다. "무엇을" 만들지 고민하고 토론하며 결정하고, 설계하고, 토론하고, 목표가 바뀌는 과정이 일어납니다. "어떻게" 만드느냐의 지뢰들로는 python 2에서 python 3 로의 전환, 웹 프레임웍인 Django와 프론트엔드 프레임웍들과의 충돌, 아마존 elastic computing cloud와 docker를 사용한 디플로이 시나리오 등 삽질 중에 발생하는 일들이 있습니다. "왜"에 대한 질문들은 무겁지만 피해갈 수 없습니다. "왜 하필 파이썬인가?" "왜 하필 그런 서비스를 만드려 하는가" 등의 질문은, 무엇인가를 만들기로 결심한 사람들에게 주어지는 가장 중요한 질문이자 보상이기도 합니다.

저희는 지난 2개월동안 이 과정을 통해 우리가 배운 것들을 공유하고자 합니다. 구체적으로는 개발 중인 플랫폼 구조, 설계 과정의 경험 및 python 3 기반의 플랫폼 개발시 주의할 점들에 대해 이야기하고, 그와 함께 지뢰밭을 걷게 만든 '원동력' 에 대해 함께 이야기할 수 있는 자리가 되었으면 합니다.

Transcript

1. 펾묺핞 짝 묞퓯핞읊 퓒 몒칾 짝 쭒컫 앹 컲몒 킮헣뮪 / 밎훎믾 (+ 짣홓) 앷픒 펓 lablup.com 핂턺, 삲맧핂, 컿뫃헏.
2. 펾묺핞 짝 묞퓯핞읊 퓒 몒칾 짝 쭒컫 앹 컲몒 킮헣뮪 / 밎훎믾 (+ 짣홓) 앷픒 펓 lablup.com 핂턺, 삲맧핂, 컿뫃헏. 캋힖믾
3. 콚맪
4. 짪핞 ▪ 킮헣뮪 ▪ 쫃핯몒 핂옮 줊읺핞 ▪ 몒줊읺 / 뾚뫊 ▪ 큲쯚 맪짪 ▪ 폲콚큲 쯢옪믆 콚풶펂 / 컪찒큲 ▪ TNF/ Needlworks ▪ 믾 슿슿… ▪ 밎훎믾 ▪  뫃 ▪ 킪큲 컲몒 ▪ NBA 맪짪 ▪ 콚풶펂 믾짦 80 Gbps 않푾 ▪ 큲쯚 맪짪 ▪ 믾 슿슿… http://www.lablup.com펞 많졂 홎 섢 핖킃삖삲.
5. 샎 뫊 펾묺 ▪ 믾쿮픎 21켆믾, 킪큲픎 20켆믾, 몒쁢 19켆믾 ▪ 핆맒핂 뫎펺쁢 쭒퍊픦 짪헒 콛솒많 믆엕힎 팘픎 쭒퍊픦 짪헒 콛솒읊 싾않많힎 좉쁢 캏 ▪ 몒퐎 펓몒, 칺픦 맒믇
6. 묞퓯뫊 펾묺많 21켆믾픦 콛솒펞 재엲졂 펂썮퍊 밚?
7. ▪ 펾묺잞탆짆 슣뮏펞 삺팒 슪옪 컪읂 칺잩씮 팒삖탢 핂얾 혊옪 펂읾 솒많 삖읂몮혆 챊핂켢솒 잖뺂 헪 쓶슲 킪얺싢 좉 뽆짆 삖않. ▪ 뺂 핂앒 퓒퍊 펂폍찒 뻖멶 캖옪 펾묺 슪 빦뿒짆퐎 뫊 뫃 뽆읺읊 쟇많뽆삖 칺앚잖삲 펺 쿦찒삖멶 빮옪 툲젢 쳚 몮혆 싾않짆삖않.
8. 잍밆솒  멆픚쭎. ..컲잖 힒힪 읺 핆멂 팒삖멮힎
9. 잚슲 멑 ▪ 캚슪짣큲 믾짦픦 옪믆앦짛 슪 킲 컪찒큲 ▪ 않푾슪 믾짦 퓮헎 핆핂큲 ▪ 콚퓮핞 / 믆웇 / 쥲삖 / 뽆 믾짦 ▪ 뫃퓮 킪큲 ▪ 컪찒큲읊 칺푷믾 퓒 퓇 / 팿
10. 캊칺 ▪ Three keyboardists but…
11. 믾쿮헏 솒헒슲 ▪ 캚슪짣킿 ▪ 캏 쫂뫎 ▪ 읺콚큲 쫂홂 ▪ resource consolidation ▪ 섾핂 묞 ▪ 펓옪슪 짝 삲풂옪슪
12. 믾쿮 큲
13. 폲쁦픦 훊헪! Python! …픒 훟킺픊옪  멑 엊 쫂핂쁢 펂힎 먾샎 힒핆 캋힖 핂퍊믾…
14. Python! ▪ 핳헞 ▪ 맒멾컿 : 퀺풂 슪 큲핊 퓮힎 ▪ PEP-8 ▪ 많솓컿 : 킺핞많 삲읆 칺앚 슪 핋믾 퀺풎 ▪ 빦픦 믾쁳펞 샎 빦픦 묺 ▪ 짷샎 훎않핂쯚얺읺
15. Python T_T ▪ 삶헞 ▪ 쁞읾 컿쁳 ▪ PyPy / Pyston 옪헫 ▪ 핺: ▪ C옪 핟컿쇪 몒칾 않핂쯚얺읺 ▪ Cython 핂푷 ▪ 삲읆 펆펂옪 핟컿쇪 않핂쯚얺읺 짢핆싷
16. Python픦 뫊/뫃 묞퓯 픟푷 ▪ 칺옎 ▪ 잜픎 샎펞컪 CS101 믾 옪믆앦짛 쿦펓펞 푷 훟 ▪ “핆줆헏핆 펆펂” ▪ 핆읺 펆펂 ▪ 줊옮 앎 핂펞 indent옪 쯢옫 잚슪쁢 펆펂많 쏞 빦폺 훒픎 졾앞킃삖삲잚… ▪ 믾홂 펆펂슲뫊픦 퓮칺컿 ▪ MATLAB 짾풮픊졂 numpy많 펂힎 핃쿧
17. 맪핆헏 몋슲 : 킮헣뮪 ▪ 뫊 펾묺펞컪픦 삲퍟 푷 ▪ MATLAB픎 찒탆삲: matplotlib, ipython notebook ▪ Ultra-complex figures ▪ 킪쥺엖핂켦 얾핒쫂삲 슪 핟컿 킪맒핂 훟푢 쌚 ▪ 맒삶 젎옪켆큲 힎풞 ▪ 섾핂 쭒컫 ▪ 펆펂 핂킫컿 / 컿
18. 맪핆헏 몋슲 : 밎훎믾 ▪ 뫃 펾묺펞컪픦 푷 ▪ NBA  읺 엖핒풚 ▪ 캋힖픒 핞솧믾펞 잲푾 퓮읺 ▪ 슪풶펂 configuration픒 Python 큲잋옪 핟컿 ▪ 킲 큲잋펞컪 뻲풚 콚 I/O퐎 젢핆 옪믆앶픦 stdout픒 솧킪 졶삖잏믾 퓒 asyncio 믾짦픊옪 킲 큲잋 핟컿
19. It's time to go Python 3 ▪ Python 3읊 힎풞쁢 않핂쯚얺읺 흫많 ▪ Django, numpy/scipy, … ▪ Python 3.3 핂 줆핞폂 젢졶읺 칺푷얗 맞콚 ▪ PEP-393: str 맫펞컪 줆핞 샇 짢핂 쿦읊 믆 줆핞폂픦 줆핞 훟 많핳 뽠픎 unicode codepoint 쿹핞옪 멾헣 ▪ 믾홂 narrow build쁢 UTF-16, wide build쁢 UTF-32 몮헣 ▪ 핂헪 핞솧픊옪 줆핞폂 맫쪒 UTF-8/16/32 삲읂멚 칺푷 ▪ ASCII/Latin-1 폏펻잚 삲욶쁢 몋푾 잲푾 퓮읺 ▪ 퓮헒핞 컪폂 쭒컫 / 폏줆·옪믆 찓섾핂 쭒컫
20. It's time to go Python 3 ▪ Python 3.4 핂 쫂삲 맪컮쇪 찒솧믾 옪믆앦짛 졶셆 ▪ PEP-3156: asyncio 않핂쯚얺읺퐎 coroutine ▪ 삶핊 튾엖슪펞컪픦 삲훟 I/O 핂쩲 졶삖잏 ▪ GIL (global interpreter lock) 몋펞컪 삶핊 튾엖슪 퓶컿픒 뽠핂쁢 쿦삶픊옪 퓮푷 ▪ 쭒칾킪큲 맪짪펞 읺 ▪ pip퐎 virtualenv 믾쫆  ▪ 짾몋 핺핂 푷핂
21. 쭒칾 쥲삖핂켦 ▪ RabbitMQ, Kafka, Celery, ... ▪ 헣 픦 킮 잚 힎풞먾빦 푾읺많 튾엲쁢 푷솒펞 찒 뻖줂 쫃핯 ▪ 쪒솒픦 훟몒 옪켆큲(broker)읊 잚슲펂퍊 삲먾빦, ▪ 쫂혾펻픒  redis 컪쩒많 푢삲먾빦…
22. 쭒칾 쥲삖핂켦 – ZeroMQ ▪ 쭒칾읺펞 핞훊 칺푷쇦쁢 몮쿦훎 킮 슲픒 콚 옪 맞탆 헪뫃 ▪ Request-Reply ▪ 퍟짷 젢킪힎 1틷 훊몮짩믾 (칺푷 폖 : HTTP-like server/client) ▪ Push-Pull ▪ 삶짷 N-to-1 젢킪힎 짎펂뻱믾 (칺푷 폖 : logging system, task partitioning) ▪ Publish-Subscribe ▪ 삶짷 1-to-N / N-to-M 젢킪힎 짎펂뻱믾 (칺푷 폖 : broadcast)
23. 쭒칾 쥲삖핂켦 – ZeroMQ ▪ 핳헞 : 맒삶 컲헣뫊 뽠픎 컿쁳 ▪ zero broker, zero latency, zero administration, ... ▪ 핳헞 : 삲퍟 connection 짷킫 힎풞 ▪ TCP / UDP / UNIX domain / local "in-process" ▪ 핞헏핆 zeromq transport header (ZMTP) ▪ payload 팬쭎쭒펞 쭧펂컪 빮팒맞 (64+ 짢핂) ▪ 삶헞 : persistent queue 헪뫃힎 팘픚 ▪ TCP퐎 솧슿 쿦훎픦 reliability잚 헪뫃
24. 쭒칾 쥲삖핂켦 – LogStash ▪ LogStash ▪ input / filter / output 먾쁢 맒삶 묺혾픦 옪밓 킪큲 ▪ ZeroMQ, S3앟 맧핂 칺푷 많쁳 ▪ (힏헟 잚슲펂솒 쇦쁢섾 믆앦솒 핖쁢 먾 튾핞!) Django User Kernel Logstash Server Database ZeroMQ (push-pull) + JSON AWS S3 msgpack
25. On-premise vs. Hosting vs. Cloud ▪ 컮픦 펺힎많 펔픚 ▪ "캊"칺... ▪ 묺믎, 팒잖홂, 잖핂옪콚 ▪ AWS ▪ 컮옎많 잜픚 ▪ 헎쁢 슮 멚 킹펂푢 ▪ 힎잚 잖콚쁢 퓖솒푾펞멚 싾씉멮힎… ▪ 2012 R2  샎 솚엲컪 픎 / 헣쭎 컪윦 / 팯쯚펟큲 헒푷픊옪 틏삖삲.
26. Docker containers ▪ 졶슮 컪찒큲쁢 docker container 퓒펞컪 ▪ 컪찒큲슲픒 큋멚 핺칺푷 ▪ VM쫂삲 찮읆 킪맒 팖펞 deploy/destroy 많쁳 ▪ AWS EC2 Instance: 펺헒 폲앦 멆잊 (쭒 삶퓒) ▪ 핂뻖읊 핂푷 pooling ▪ Amazon ECS? ▪ container 씒푾쁢(run task) latency많 ힵ힎 팘픚 ▪ 멾묻 핂멑솒 핞헏픊옪…
27. Docker containers ▪ 솒헒뫊헪 ▪ 찒푷 : 샎믾 캏픦 많푷 instance 퓮힎 찒푷 콚 ▪ Microsoft픦 docker 힎풞 ▪  Azure 짝 Windows 몋펞컪 칺푷 믾샎
28. Polymer library ▪ HTML5 믾짦픦 뻚 믾짦 않핂쯚얺읺 ▪ HTML imports / Custom elements / Shadow DOM ▪ Web components 믾짦 ▪ “Polyfill” ▪ 핺칺푷 많쁳 뻚 맪짪 많쁳 ▪ DOMelements
29. Polymer library ▪ 팒뽆 펾훊핞슲픎 싢핞핆펞쁢 솒풎핂 팖 쇪삲... ▪ 묺믎 섣 홎 쫂핞! ▪ Bootstrap + theme? ▪ 힎몇삲 ▪ HTML5 킪샎많 폺 섾 푾읺  쩖 캖 켆캏펞컪 핟펓쫂핞!
32. HTML imports! Reusable web components! Shadow DOM! Polyfill!
33. Polymer픦 믊쫆헏 핂큖 ▪ 쯚않푾헎 컿 ▪ 퐪 퓖솒푾 7펞컮 팖쇊푢? 퐪 핃큲옪얺펞컮 팖쇊푢?
34. Electron: Cross-platform Polymer ▪ 멾 : cross-platform webapp container ▪ Electron ▪ Github펞컪 맪짪 Node.js 믾짦픦 퓇 팿 엖핒풛 ▪ Node 팿 쏞쁢 퓇 팿픒 Chromium 얾핒픊옪 wrapping ▪ 퓖솒푾 / 잳 / 읺뿓큲 힎풞 ▪ ATOM 펞싢픦 믾짦 ▪ 팿픎푢? ▪ Apache Cordova
35. 핞솧 읺 ▪ 핂턺픎 홙픎 핞솧 솒묺핓삖삲 ▪ 쫕쫕쫕 ▪ 퓮삩 큲 ▪ Django UnitTest 믾쁳 + Selenium ▪ Selenium webdriver: 쯚않푾헎 솧핟 펞쥺엖핂켦 ▪ 밑 짙 킪 큲 펞 멆몮 핞솧픊옪 큲  줆헪많 캫믾졂 핂큖 샡믎옪 힎헏
36. 핞솧 읺 ▪ 잲쁂펊 핞솧 캫컿 ▪ 줂큶 쿦읊 턶솒 슪 쫃핯솒쁢 폺않많멚 쇦펂 핖삲 ▪ sphinx ▪ reST 믾짦픦 핞솧 줆컪 캫컿 옪믆앶 ▪ 슪펞 삺읾 훊컫픒 믾짦픊옪 콚큲 잲쁂펊 캫컿 ▪ 핞솧 읺쁢 훊헪 ▪ 폲쁦픦 훊헪쁢 팒삖즎옪 펺믾밚힎잚! ▪ 짣홓삦핂  잞핂 펔펂힒 핂퓮
37. 핞솧 읺 ▪ 잲쁂펊 핞솧 캫컿 ▪ 줂큶 쿦읊 턶솒 슪 쫃핯솒쁢 폺않많멚 쇦펂 핖삲 ▪ sphinx ▪ reST 믾짦픦 핞솧 줆컪 캫컿 옪믆앶 ▪ 슪펞 삺읾 훊컫픒 믾짦픊옪 콚큲 잲쁂펊 캫컿 ▪ 핞솧 읺쁢 훊헪 ▪ 폲쁦픦 훊헪쁢 팒삖즎옪 펺믾밚힎잚! ▪ 짣홓삦핂  잞핂 펔펂힒 핂퓮
38. 믆앦컪 핊삶 컲몒읊  쫂팦킃삖삲. 캋힖픎 픦 펞 쁢 먾삖밚푢.
39. 푾컮 핯핊 샎킮 밑 쫕핂앟 큺앧쫕픒  뻲 잖읺 잚슲몮, 핊엗옮핂앟 읺젆 핊 큲잋읊 잚슮 삲픚, 켎엖삖풎 + 밑픊옪 퓮삩 큲 핞솧읊 잚슮 ,
40. 졷 쪎몋 ▪ 펾묺푷 슪 킲 + 뫃퓮 앹쭎 잚슲엲몮 쁢섾 묞퓯푷 앹픒 젊헎 맪짪멚 쇦펖삲. ▪ 펾묺푷 앹 : ▪ 힣 – 뮪졶많 슪읊 폲앦 솚읾삲 ▪ 캚슪짣큲 쿦졓 뫎읺, 쫂팖, 삶핊 슪읊 퓒 scaling, ... ➞ 맪짪킪맒 ++ ▪ 묞퓯푷 앹 : ▪ 힣 – 뮪졶많 핟픎 슪읊 펺얺 맪 솧킪펞 잜핂 솚읾삲 ▪ 캚슪짣큲읊 잗 훋펺솒 쇪삲 맏 캚슪짣큲펞 핞풞픒 헏멚 샇솒 쇪삲
41. 졷 쪎몋 ▪ 핂헞픒 짦폏펺 퓮읺 멑쭎 잚슲핞. ▪ 묞퓯푷 앹쭎! ▪ 컲졓솒 줦 잚슲엲쁢힎 졶읂쁢 칺앚핂 잜팦펂푢 ▪ 펾묺푷 앹픒 틆 칺앚슲쭎 풚퍊
42. CodeON! //codeonweb.com 쩮 큲 훟!
47. …
48. 퓮 뽆 빹앃, 홂 큲뽆푾…
50. “팓잖쁢 싢핊펞 핖삲.”
51. 옮펢슪쭎 짿펢슪옪 ▪ React / Flux / AngularJS / Polymer / ... ▪ 핞짢큲잋+UI 엖핒풚픦 헒묻킪샎 ▪ Polymer 턶쫲쁢섾 0.9펞컪 API 삲 맖팒펜픚.... ▪ 멾묻 펻칺퐎 헒(?)픦 Bootstrap픊옪 쫃뮎… ▪ 삲많 묺믎 I/O 2015펞컪 Polymer 1.0핂 빦퐎컪 삲킪 솚팒맢쁢섾...
52. Polymer: it is too google to be true ▪ 밆힎 팘픎 맪짪 믾맒 솧팖 ▪ 0.5 - 0.8rc2 - 0.9 - … ▪ 맏 쩒헒픦 잖핂믆엖핂켦 줆헪옪 숞 쩖 칺푷픒 믾 ▪ 핞솧 잖핂믆엖핂켦  힎풞 (핦 팖 쇶) ▪ Google I/O - 1.0 :  쩖 잚 섢 쫊밚
53. Polymer: vulcanize ▪ Vulcanize ▪ 않핂쯚얺읺읊 훟쫃 헪먾몮 : 핊홓픦 핊 ▪ IE 푷픊옪 헪샎옪 쇪 핊 팖 쇶 – +헪킃삖삲 ▪ 핊  CSS 헏푷 졶킃핂 삲읒 – +헪킃삖삲 ▪ Crisper픦 javascript map힎풞 슪많 shadow DOM픦 root 핋펂폲쁢 쭎쭒뫊 솚 - +헪킃삖삲
54. Polymer: vulcanize ▪ Vulcanize ▪ 않핂쯚얺읺읊 훟쫃 헪먾몮 : 핊홓픦 핊 ▪ IE 푷픊옪 헪샎옪 쇪 핊 팖 쇶 – 킃삖삲 ▪ 핊  CSS 헏푷 졶킃핂 삲읒 – 킃삖삲 ▪ Crisper픦 javascript map힎풞 슪많 shadow DOM픦 root 핋펂폲쁢 쭎쭒뫊 솚 - 킃삖삲
55. Polymer: 짢삲읊 풚퍊 쁢 polyfill ▪ 핂힎 핺엚섢잏 쿦 ▪ 핃큲옪얺 ▪ 먾픦 졶슮 퓇뻚 믾쁳(html import 짝 shadow DOM)픒 polyfill펞 픦홂퍊  ▪ 칺읺 ▪ 핖섦 뮪멷솒 힎푾쁢 훟
56. Polymer + Django ▪ 줆쩣 솚 ▪ {{ … }} ▪ Polymer – 잍펞컪 퍟짷 섾핂 큲잊 폏펻 ▪ Django – 잍펞컪 쪎쿦 맠  짝 엳 ▪ 폖푆 폏펻 읺읊 힎헣 맖 쿦 핖펂푢 ▪ 솧헏 캫컿 잍픦 몋푾쁢 쭖많쁳 ▪ 숞 솒묺많 캏픦컪 핦 멾픒 퍊…
57. Polymer + Django + Security ▪ CORS (cross-origin resource scripting) ▪ Django – CORS header 헒콯 힎풞 (졶슖) ▪ Polymer – vulcanize 멾뫊줊픎 Javascript퐎 컬펺 CORS 혾멂픒 잚혿힎 좉 ▪ Crisper읊  CORS읊 퓒 Javascript읊 쩥멶뺒 ▪ 쩒헒펓 훟 슪 밶힞 - 힏헟  ▪ XSS 짷펂 ▪ Django - CSRF token 칺푷 ▪ Polymer – 푷쇦힎 팘픎 header읊 iron-ajax읊  빮잂 쿦 펔픚 ▪ 쇦솒옫 힏헟 
58. Polymer ▪ Production ready?! ▪ 쿦폏핳핆훒 팚몮 쿦폏쫃 핓몮 쒾펂슲펖섢삖 칺잗 ▪ “Do not swim in sandocean. Swim in the ocean.” ▪ 짆앦픦 HTML5 옮펢슪 ▪ 핂 짷핂 재픒 슽 ▪ X-tag, react, polymer… ▪ 묻펞컮?
59. Polymer ▪ Production ready?! ▪ 쿦폏핳핆훒 팚몮 쿦폏쫃 핓몮 쒾펂슲펖섢삖 칺잗 ▪ “Do not swim in sandocean. Swim in the ocean.” ▪ 짆앦픦 HTML5 옮펢슪 ▪ 핂 짷핂 재픒 슽 ▪ X-tag, react, polymer… ▪ 묻펞컮?
60. Electron + web ▪ 묻펂... 묻펂많! ▪ 펂싪 많솒 믾쫆 믎봂핂 뭂잊핆 핊엗옮 ▪ 읺짛쇪 믎 퓇 큲 (Noto 믾짦) ▪ 퓇 ▪ CSS: 칺읺퐎 옺픎 픦푆옪 잜핂 삲읓삖삲. ▪  쌚쁢 탛숳핂폎힎잚 핂헪쁢 펒잖많 맧픎 헪 헣솒?
61. 짿펢슪: Django + RDS ▪ UTF-8 읺많 Python펞컪 뫊펾 풞밚? ▪ 풞 (cf., str / bytes / io / codecs) ▪ MySQL핂빦 SQLite읊 튾힎 팘픒 몋푾잚 ▪ MySQL ▪ ForeignKey 뫎엶 졶셆핂 Django퐎 뭏핂 팖 재쁢삲 ▪ MySQL + Django + Korean = EPIC FAIL ▪ (펾묺 쌚 튾섦) Postgres옪 헒삖 큋멚 큋멚... ▪ 믆얾섾 postgres많 (찒푷/핞풞) 졂펞컪 찒탊 멑 맧팒푢.
62. Python 3.4 – asyncio / coroutine 캋힖 ▪ 맒삶 슪읊 핟컿 쌞 폖츦멚 핦 쇪삲. ▪ 쫃핯 슪펞컪 yield from 챊젇픊졂 잫... ▪ AST 훎힎읊 푷 static analysis옪 coroutine픒 yield from픒 칺푷힎 팘몮  몋푾 핞솧멎칺 맪짪 ▪ 믆얺빦, 핂옮헏픊옪 100% static analysis 쭖많쁳 ▪ duck typing + dynamic method generation + method proxy pattern ... 팖쇮 핂퓮쁢 줂쿦 잜삲 ▪ Python 3.5펞컪 await/async 줆쩣핂 캫믾졂 졕졕 슲픒 틺 읺멚 핟컿 많쁳 멑픊옪 믾샎 훟!
63. Python 3.4 – asyncio loop 홓욚 짷쩣 ▪ 줆컪펞 핖쁢 샎옪 핦 힪컪 loop.close()읊 섢삖 펞얺많 홚않않앋.. import asyncio import asyncio_redis loop = asyncio.get_event_loop() conn = loop.run_until_complete( asyncio_redis.Connection.create('localhost', 6379)) conn.close() loop.close() Task was destroyed but it is pending! task: <Task pending coro=<_reader_coroutine() running at /... /lib/python3.4/site-packages/asyncio_redis/protocol.py:919> wait_for=<Future pending cb=[Task._wakeup()]>>
64. Python 3.4 – asyncio loop 홓욚 짷쩣 ▪ asyncio graceful shutdown 핦 엲졂, 1. loop._run_once() 2. loop.run_until_complete(asyncio.sleep(0)) 3. loop.run_until_complete(server.wait_closed()) ▪ 푆쭎 않핂쯚얺읺읊 칺푷쁢 몋푾, wait_closed()퐎 맧픎 졓킪헏핆 핞풞쿦 coroutine핂 헪뫃쇦힎 팘픒 쿦 핖삲. ▪ 핂엂 쌞 홓욚 헒 쿦솧픊옪 event loop읊 힒킪퍊 삲.
65. Python 3.4 – asyncio loop 홓욚 짷쩣 #! /usr/bin/env python3 import asyncio loop = asyncio.get_event_loop() @asyncio.coroutine def my_timer(): i = 0 while True: yield from asyncio.sleep(1) print(i) i += 1 try: asyncio.async(my_timer(), loop=loop) loop.run_forever() except KeyboardInterrupt: pass finally: loop.close() ^CTask was destroyed but it is pending! task: <Task pending coro=<my_timer() done, defined at test.py:4> wait_for=<Future pending cb=[Task._wakeup()]>>
66. Python 3.4 – asyncio loop 홓욚 짷쩣 #! /usr/bin/env python3 import asyncio loop = asyncio.get_event_loop() @asyncio.coroutine def my_timer(): i = 0 while True: yield from asyncio.sleep(1) print(i) i += 1 try: asyncio.async(my_timer(), loop=loop) loop.run_forever() except KeyboardInterrupt: for t in asyncio.Task.all_tasks(): t.cancel() try: loop._run_once() except asyncio.CancelledError: pass finally: loop.close()
67. asyncio_redis ▪ Redis읊 칺푷믾 퓒 asyncio 믾짦 않핂쯚얺읺 ▪ Heisenbug 짪캫! ▪ SCAN 졓옇픒 솚옆섢삖 풞쁢 key많 빦폲믾솒 팖 빦폲믾솒... ▪ 않핂쯚얺읺 뺂쭎펞컪 while 줆 샎킮 if 줆픒 턶컪 짪캫 쩒믆 ➞  쫂뺂컪 멾 ▪ https://github.com/jonathanslenders/asyncio- redis/issues/65 ▪ Connection pool API 핂큖 ▪ 펺얺 맪픦 펾콛쇪 API call슲핂 맧픎 connection픒 핂푷솒옫  쿦 펔펂 stateful API 칺푷 쭖많 (폖: SELECT) ▪ 삲읆 않핂쯚얺읺옪 맖팒믾 멎 훟 (aioredis)
68. 짿펢슪: 쭒칾 쥲삖핂켦 – ZeroMQ ▪ Asynchronous졂 홙삲? block힎 팘팒 찮읂삲?? ▪ socket connect 핂 읺솒 킲헪옪 펾멾펺쭎쁢 팚 쿦 펔삲. (킺힎펂 캏샎짷 컪쩒많 팒폖 팖 썮핖펂솒...) ▪ 핂 send/recv 힒 킪 졓킪헏핆 timeout 멎칺 쿦 ▪ aiozmq읊 핂푷삲졂, blocking call픒 칺푷 쌚 asyncio.wait_for옪 맞탆컪 timeout 멎칺읊 훦퍊 삲. ▪ pyzmq잚 핂푷삲졂, socket.poll(msec) 핂푷컪 timeout 멎칺읊 훦퍊 삲.
69. 짿펢슪: 쭒칾 쥲삖핂켦 – LogStash ▪ 잲쁂펊샎옪 zmq input plugin뫊 s3 output plugin 컲헣몮 솚엲쫲섢삖 zmq recv_string 폲윦 뺂졂컪 줂욶 ▪ 풞핆 1: s3 output plugin 잲쁂펊펞 “write” 뭚픒 훊않몮 쇦펂 핖쁢섾, PutObject잚 훦솒 쇦쁢훒 팚팦섢삖 DeleteObject 뭚솒 훦퍊 쁢 먾폎삲. ▪ 풞핆 2: config 줆쩣 펞얺퐎 삺읺, 졶홓픦 핂퓮옪 output plugin 믾많 킲 몋푾 믾쫆 log level펞컪쁢 칺푷핞펞멚 팒줂얾 몋몮·폲윦읊 쫂펺훊힎 팘쁢삲. ▪ 풞핆 3: output plugin핂 믾쇦힎 팘몮 킲쇪 몋푾 input plugin픦 receive timeout픒 줂킪쁢 쩒믆많 핖삲.
70. 짿펢슪: Docker ▪ 헎핳콚 솚 ▪ 뫃킫픊옪 짾쇦쁢 docker registry픦 "latest"쁢 칺킲 latest많 팒삦 ▪ 졓킪헏픊옪 2.0 쏞쁢 2.0.1 tag읊 턶퍊  ▪ 칺콚 줆헪힎잚 핂얾 멑솒 캋힖옪 킪맒 핯팒젇픚 ▪ 헒잫 ▪ docker 1.8 짪, CoreOS쁢 핞 얾핒(rkt) 솒핓 ▪ 픊옪 팬픊옪솒 솧팖 슪/API 쪎몋핂 캏샇 핖픒 멑픊옪 폖캏
71. 짿펢슪: 쫂팖 ▪ Jupyter / ipython 픦 핳컿 ▪ scalable힎 팘픚 ▪ Jupyter 묺픦 줆헪헞 ▪ Unix 몒헣 믾짦픦 핒킪 몒헣 킪큲 핂푷 ▪ 쫂팖펞 줆헪많 쇮 잚 멑슲픎 삲 솚팒맞 ▪ 믾쫆 컲몒 : 옪 뽆쭏픒 퓇 핆핂큲옪 솚읺쁢 푷솒 ▪ 컪찒큲푷픊옪 맪짪쇦믾 펂엲풎 ▪ 슪 킲 컪찒큲 – Sorna ▪ 믾쫆쭎 헒쭎 삲킪 힪몮 핖킃삖삲.
72. 믆엕멚 잚슲펂힒 CodeON
73. 팬픊옪  핊슲 ▪ Container Resource Consolidation ▪ Kubernetes 믾쿮 멎흫 ▪ 믾홂 콢욶켦핂 풞쁢 컿쁳핂 팖 빦폺 몋푾 paxos빦 raft 맧픎 쭒칾 agreement 팚몮읺흦픒 칺푷쁢 key-value store읊 짿펢슪옪 펺 힏헟 묺 폖헣. ▪ Ingen REPL (read-evaluate-print loop) 몮솒 ▪ stdout/stderr streaming ▪ interactive plot 엳 ▪ image, sound 슿픦multimedia 엳
74. ▪ 팯찒 옪믆 믾짦픦 맪핆 핳 ▪ 퓮헎 팯찒 앦 ▪ User profiler + backtracer + ML with back propagation ▪ 핞솧 큲 캫컿 졶슖픦 몮솒 ▪ d3.js 읊 쿦헣 fluid nonlinear navigation UI ▪ 믾 ▪ 킃푷 핆핂큲 맪컮 ▪ 맪핆쪒 슪 큲 캚슪짣큲 몮솒 ▪  핺묺컿믾 퐒컿 ▪ 킪쥺엖핂 퐒컿
75. 멾옮 ▪ 홙픎 폲콚큲슲픒 벦 졶팒 핦 컪 잚슲졂 푾훊많 빦컪컪 솒퐎훛삖삲. ▪ ...옪 킪핟쁢섾 킲픎 폲콚큲 캋힖얺 + 핞 맪짪 +  헪뫃픊옪… ▪ 핂캏뫊 킲픎 핂엕멚 삲읓삖삲. ▪ 믆얾섾 핂캏픒 킲옪 븚펂폲쁢멚 쏞 잲엳 팒삖멮킃삖밚?