Linux コンテナ最新情報 (2013-06-01)

Linux コンテナ最新情報 (2013-06-01)

2013-06-01 に開催された "第1回コンテナ情報交換会" (http://www.zusaar.com/event/686003) の発表資料です.Speaker Deck に上げるとスライド中のリンクがリンクでなくなるようなのでオリジナルの資料は https://guinan.ten-forward.ws/lxc-20130601/ に上げてあります (or ここで PDF をダウンロードしてください).

2591343b244565d6199f61c4acd148f9?s=128

tenforward

June 01, 2013
Tweet

Transcript

  1. Linux Ï äé^&Žp g̋ c5 r 1 ‚ Ï äéŽpÊ

    ¶ - 2013/06/01
  2. Š̊â∙ ùÚ: g̋c5 þ: ûÕä½ Ì¶&”Øo ! OSS 94ëø ·

    http://www.ten-forward.ws/ twitter: @ten_forward g+: http://gplus.to/tenforward z∼KìÜ”õ Ì - - - - · · Plamo Linux Ÿ äé¿Web úß ÔÏ ä â*¶ lxc man pages ¬/ Jetspeed2 þ꙼ ¬/ ( Øqëø|–—£µ) zÛ\µ”ŠáË v SD •c‚ Ê - - - - 2/68
  3. :Ÿ” Ô Linux ”Ï äé9÷”Øo»ëo ¡́ |v|— —›¤Žpw¶o”w‚ ¡ ^Š”

    Linux ”Ï äé•9¡−øÖ³— Ł–Æ—|¤” oÚ|—¡ w¿ ^&Žp»¿ ́< — Łœ©−Ø” »¶\ƒq ¡ (^_^;) ¶” :Ÿ”̃”ç©»ó Þ ¡ |‚(vœ”âáÏÿ 穳© ”^&Žp•¡−”w© ” Ô ¡ LXC ”³oz¿ Ƥo¶ »m—ß‾›–—£µ Çß ì Ø LXC Ø×»łvœ¢–−“¦¶” 7Èoºt¦ØIov Êo—¡ · · · · · · · 3/68
  4. Agenda Ï ä锩o ¾þèÉ”%È ” LXC Kernel ^&Žp LXC ^&Žp

    ƒ”H · · · Namespace Cgroups ƒ”H - - - · · 4/68
  5. Ï ä锩o 5/68

  6. Ï äé » OS ù ”b a ö ×Õ³Ì ß öa|–H”Ì

    ß ö Ö Ì ß öa|¤ö ×Õ•Z¡− Úß Õv Çß ì ”ófiöl̀ ³³›–b a³ ‚ · · · · 6/68
  7. Ï äé”Ãî Ñfi” OS ( Çß ì ) ”Æwø¢|–o− îß

    èÁ¼”b awóo ¾¶− OS ”ÓÕäŠ / ö Ì Š»øv£¶o nøw o b þÓ ”ö Ø t¶zøz⁄!w · → ẫ - · → Åß ïß ÷áèwµ\o - · · · 7/68
  8. Linux •u¦−Ï äé ! OpenVZ / Virtuozzo( −i) Linux VServer

    Çß ì wÏ äéÖ¦”̀ ³ó¤¶o¤Ł¿ ̂q¶ðáÞ³Qi|¿ Ï ä éb a³ ‚|–o¤ |v|¿Linux 2.6.19 Rßvœªł • !\©ÁŁ¿ øqºz^Š•¶›– i O•¶›–x¤ OpenVZ/Parallels ³»∼Ł |– IBM, Oracle, Google ́¿ ł ¶ !•9 °›–o¤Ã Ôê¼wØo•̨g · · · · · 8/68
  9. LXC Linux Çß ì ”ófì ”Ƴ³›–Ï äé³ ‚¡−âß |– LXC

    (http://lxc.sourceforge.net/) libvirt ( ”LXC Ï äéè ¾ï) (http://libvirt.org/) ∙«œØœ∼ "LXC" oqùÚ³³›–o−w¿ Ł1ô¹¾ »Ił́ ̀ Ø® é•È›–o¤ m systemd ØV¾Ï äéw¢©−œ|o ( ¹z−œ¶o) · Linux Ï äé³ ¢¡− userspace âß ( Ïþ è.) - · · ^Š»? - · 9/68
  10. ¾þèÉ”%È ” LXC ;oå½Õç ò™ß Ó ” LXC +”nø

  11. Ubuntu ‚v” LXC ”ì ö”Øoö áçôÄß Š ‚v”;oØo(» Ubuntu åù

    áðß Ubuntu ðáÍß Ô» LXC ”&̀ wïáËýß ç LXC / libvirt À• t¶zøz ( »¢) precise(12.04LTS) ı ” {̃w o · Ubuntu Øo\©¤¹¿ H”å½Õç ò™ß Ó øzøq•ü}\ ©–o− ∼ - · · oqøß¿ ß• Ubuntu !\©¤̀ ³©l•þß Ô|–− - · ^Š libvirt ” lxc è ¾ï³›–¶o” ... - · apparmor N“¶øx» x¶z¶›–o−́ &̀ ”ß<ß́ precise » 0.7.5 “w 0.8.0 ”̀ wv¶ß‾›–o− - - 11/68
  12. Ubuntu LXC wøz— # a p t - g e

    t i n s t a l l l x c # l x c - c r e a t e - n c t 0 1 - t u b u n t u # l x c - s t a r t - n c t 0 1 - d # l x c - c o n s o l e - n c t 0 1 lxc ðáÍß Ô³¾ Õçß ¡−“¦́cgroup ”þÁ çiÏþ è́¿ 9÷ ”ؔؾ Õçß \©− nøÕË öçØ lxc wøzøq•ü}\©–o− (Upstart ” init ́) AppArmor Øü}hÆ · · · 12/68
  13. CentOS Ô ðáÍß Ô• LXC »¶óepel H¿ øÔ ýÔç •Ø

    lxc ðáÍß Ô» ¶| Wiki ³¢− libvirt ³³›¤z¦wâ∙ → HOWTO: Configure a LXC Linux Container CentOS 6 Ô •»ä ö ß ç (lxc-centos) w?—©¶ógithub/gist ³¿© mß—¡ → lxc-centos kernel w 2.6.32 ¶” v¶ß̂o Úß ÕvœÏ ð¾ ¡© ßms¢øz · · CentOS (RHEL) » libvirt w:¹Ô ? öl>”“¦ »nø|—£µ |¤ - - · · !ŵo - NS Cgroup (namespace ÑõÓÕäŠ) oqØ”wBv (cgroup ” clone_children Œ™ö. µx s) - · ¤“|¿ ^&” 0.9.0 »øxw |o ( cw¡−) !wóo)¶ net_prio, perf_event Rß»þÁ ç|¶o”wïõ - - 13/68
  14. Debian Debian 7.0 » lxc ðáÍß Ômß (0.8.0-rc1) ä ö

    ß çØBv (lxc-debian) „|õ •ðáÍß Ô” lxc-debian ä ö ß ç»pvo ̃¬” {̃w¾ þ¾Þ ( ¶cw¡−) lxc Úß ÕœD” lxc-debian ³q m›\ßnø (6.0 Ï äé ¡w) Çß ì »¶⁄v Memory Cgroup wEÒ ò è\©–o−”•Ÿ •nø¡ − åôÄ ç ïÒ•¶›–o− · · lxc-debian Ï äé¢{¡− ¿ Èoì ³ |–\–z©−5̨\ - · Ï äéå” /etc/inittab ³Úß ÕœD”ä ö ß çO•V|¡− Ï Úß » OK openssh-server ”¤Ø config ÑØ|–−¦∙ Why? - - · · Çß ì nøÅöÓ "enable_cgroup=memory" ³Ws− ( øq•¡− ðáÞw ¤›–−/6.0, 7.0) - 14/68
  15. Debian lxc wøz— Ï äéå” inittab ” getty ôŠ»̃µ¶ ∼•

    cgroupfs þÁ ç · # a p t - g e t i n s t a l l l x c # l x c - c r e a t e - n c t 0 1 - t d e b i a n : ( ł •̂s−) # v i / v a r / l i b / l x c / c t 0 1 / c o n f i g ( ìáç ß Ë94”Ł1” g) # l x c - s t a r t - n c t 0 1 ( d e b i a n Ô ä ö ß çƒ”——“ nø»¶xÌ “µ—ß > < ) 1 : 2 3 4 5 : r e s p a w n : / s b i n / g e t t y 3 8 4 0 0 c o n s o l e c 1 : 1 2 3 4 5 : r e s p a w n : / s b i n / g e t t y 3 8 4 0 0 t t y 1 l i n u x c 2 : 1 2 3 4 5 : r e s p a w n : / s b i n / g e t t y 3 8 4 0 0 t t y 2 l i n u x c 3 : 1 2 3 4 5 : r e s p a w n : / s b i n / g e t t y 3 8 4 0 0 t t y 3 l i n u x c 4 : 1 2 3 4 5 : r e s p a w n : / s b i n / g e t t y 3 8 4 0 0 t t y 4 l i n u x 15/68
  16. Fedora nø|¶o (^_^;) [ r o o t @ l

    o c a l h o s t ~ ] # l x c - s t a r t - n c t 0 1 - d - o l o g - l D E B U G [ r o o t @ l o c a l h o s t ~ ] # l x c - i n f o - n c t 0 1 s t a t e : S T O P P E D l x c - i n f o : ' c t 0 1 ' i s n o t r u n n i n g p i d : - 1 Fedora 18 •» lxc, lxc-libs, lxc-templates ðáÍß ÔwBv́ "update-testing" ýÔç vœ 0.8.0 ³‾©–Æ− Fedora Wiki ” Features/Securecontainers ³¢− virt-sandbox-service Ïþ è ¼ö Íß Ó Ï äé³¢{¡−z¦ẅv©–o− ( ã|–—£µ) · |v|ƒ”—— » 0.7.5 |vØä ö ß ç•» lxc-sshd ”Æ - - · ä ö ß çwïÌ›––¿ x«µ Ï äéâ ß w¢{\©—£µ :p - · 16/68
  17. Plamo —“Øo\©–o− :-p Ÿ©o”å½Õç ò™ß Ó contrib ¶wœ¿lxc ðáÍß ÔwBv|¿

    þŸ•ò&\©–o− ( ‚v 0.9.0) plamo ä ö ß çØBv contrib/Virtualization ı̀”ðáÍß Ô³Þ ‾©© +”• Plamo Ï äé wnøh ́ · · ×wðáÍß ÔŸ äé ¡vœw ¤“|¿ Çß ì ðáÍß Ô”Ÿ äé »¶o” ôł Èo¶̀ w£« −ìØ ^^; Plamo »¾ Õçß ¹¿ Çß ì ]öÀwQ© ¡́ ¶” c•|¶o :p ^&ðáÍß Ô» 0.9.0 “w python3 w Plamo •¶o” python API ºÑ Ïþ è»‾œ¢ - - Plamo 5.1 ô: 3.9.3 Çß ì » Memory Cgroup wÅô• - - - · · » lxc, dnsmasq ðáÍß Ôwm© OK ¡́ - 17/68
  18. Plamo lxc wøz— # i n s t a l

    l p k g / p a t h / t o / c o n t r i b / V i r t u l i z a t i o n / * . t x z # c d / e t c / r c . d / i n i t . d ; c h m o d 7 5 5 l x c - n e t c g r o u p s - m o u n t # / e t c / r c . d / i n i t . d / l x c - n e t # / e t c / r c . d / i n i t . d / c g r o u p s - m o u n t # l x c - c r e a t e - n c t 0 1 - t p l a m o # l x c - s t a r t - n c t 0 1 - d # l x c - c o n s o l e - n c t 0 1 lxcbr0 oqõ áÔ³¢{|¿veth lxcbr0 •¼ÜáÞ¡−øq•¶›–o − (Ubuntu ”ðË ) dnsmasq ³³o DHCP ¼è Õwç ¤−øq•|–m− · · 18/68
  19. Kernel ^&Žp Linux Kernel ”Ï äé9÷̀ ”ò&³ q

  20. Ï äé³ ‚¡−¤Ł”̀ ö ×Õ³Ì ß öa|–H”Ì ß ö Ö

    Ì ß öa|¤ö ×Õ•Z¡− Úß Õv · → Namespace ( ùÚ 7) - · → Cgroups - 20/68
  21. Namespace

  22. Namespace ̋ÓÔzvœ !\©–o−́lxc.sourceforge.net •ø− ¤“|¿user •9|–»¿ Îv• !»\©–o–Çß ì ”

    config Ø USER_NS »|–z−w¿ ̃”9 !\©–o¤̀ w∙”øq¶Ø” ∙q³°© –o¤”vóû́( ¹¡) utsname: 2.6.19 pid: 2.6.24 ipc: 2.6.19 user: 2.6.23 network: 2.6.26 · · · · · 22/68
  23. Namespace ” ¢ clone(2) &|oö ×Õ³¢{ unshare(2) &|oö ×Õ³¢{£¢• Ï

    äÉÕç³v¼¡− setns(2) ö ×Õ³¥B”Namespace •9÷ô¦− · · unshare ”³i× - · 23/68
  24. ̃µ¶ ̃¬•Ø Namespace LinuxSUIDSandbox (chromium) Network Namespace » ip Ïþ

    è (iproute2) +”•¢©—¡ util-linux • nsenter, unshare Ïþ è (nsenter » 2.23 g¶” å½Õç ò™ß Ó •»?—©¶ovØ?) · pid, network namespace - · · 24/68
  25. Namespace ̀ ”NÔ ° setns(2) (kernel 3.0) man 2 setns

    •ø− ô¹¾ å½ÕË öܳt¡øq•¶›–o−́ clone(), unshare() &|o Namespace ³¢{¡−̃ »h “w¿ ƒ” Namespace »&|znø|¤ö ×Õ ƒ”ÄCvœ|v¢s¶ó ƒ̃ ł vœ Namespace •¼Ë×Õ¡−̀ w 3.0 vœ g → setns() Namespace ³̨Ö¡−>ł ( ô¹¾ å½ÕË öÜ) · · 3.0 » net, uts, ipc ” Namespace ”Æ - i n t s e t n s ( i n t f d , i n t n s t y p e ) ; ̃”ô¹¾ å½ÕË öÜ» /proc/[pid]/ns ı̀” Namespace ³̨Ö¡−à @¶ô¹¾ å½ÕË öÜ «¶Æ• glibc » 2.14 ı ³ih Namespace file descriptors (lwn.net) · · · 25/68
  26. Namespace ̀ ”NÔ ° User Namespace (kernel 3.8) LXC ”

    FAQ Ïlxc ”×É™ ä½»?Ð •Z¡−‚̂́ ̃©— »Ï äé” root »ûÕç” root œ∼¦ ³ó›–o¤” ¿ Ï äévœûÕç³£ |¤ß x¤́ Ubuntu » 12.04 vœ AppArmor ûÕç”×É™ ä½³ÎZ|–o¤w¿ C ©•d»ÏUser Namespace ” !Ð ¶›–o¤Ÿ ”xØì · · · 26/68
  27. Namespace ̀ ”NÔ ° User Namespace (kernel 3.8) ̃©— Çß

    ì å ”žß Ò̂ Ì ß ö•9°−ÞÂáË•» uid/gid w³° ©–o¤ Çß ì å”ÞÂáË”¤Ł•®i” uid/gid w&Ł · · t y p e d e f s t r u c t { u i d _ t v a l ; } k u i d _ t ; t y p e d e f s t r u c t { g i d _ t v a l ; } k g i d _ t ; C 27/68
  28. Namespace ̀ ”NÔ ° User Namespace (kernel 3.8) Namespace å”

    uid/gid Çß ì å uid/gid ³þáó Ì¡− · /proc/[pid]/uid_map, /proc/[pid]/gid_map - 0 1 0 0 0 0 0 1 0 0 0 0 N a m e s p a c e å”I D k e r n e l å”I D fi− ×s ¿ ölw uid_map ¡− ¿Namespace å uid=0°10000 — ”žß Òw¿ fi” Namespace (kernel uid/gid) » uid=100000°110000 •þáó Ì\©− Namespace å 10000 ıö” uid ³¢{¡− kernel uid/gid |–» /proc/sys/kernel/overflowuid¿/proc/sys/kernel/overflowgid ”ß ¶− ̃© Namespace ( Ï äé) å”žß ÒwûÕç•Z|–N“¶ìw x¶z ¶− · · · 28/68
  29. Namespace ̀ ”NÔ ° User Namespace (kernel 3.9) 3.8 Ï

    ö ß ç! ”ì ]^|–o¤ User Namespace “w¿kernel uid/gid •ø− !wÅCô¹¾ ÓÕ䊳Ì#•Ü ! ¿ v¶ß”̀ ³ Åô•|¤ config ¶o CONFIG_USER_NS ŠXEÒ• x¶v›¤ 3.9 XFS ıł”ô¹¾ ÓÕäŠ » !whµ“ · → äÕç Oıł³s¶o - · ßms¢>fi » XFS »Ï \ø¶œ°Ð ’jþK›– USER_NS ³EÒ • :-) |v|¿ Åå½Õç ò™ß Ó ”Çß ì » XFS ³Åô•¡−°¦• »ov¶o” ¿ Øq| œz̃”̀ wŸ •fivs−”»ßvØ - - 29/68
  30. Namespace ̀ ”NÔ ° User Namespace (kernel 3.8) Linux 3.8

    ” User Namespace ̀ (1) Linux 3.8 ” User Namespace ̀ (2) Linux 3.8 ” User Namespace ̀ (3) Linux 3.8 ” User Namespace ̀ (4) · · · · 30/68
  31. Namespace ̀ ”NÔ ° setns(2) (kernel 3.8) User Namespace Ñ̆•þß

    Ô\©¤wò !! ( ^ cflv¶v›¤) pid, mount, user Namespace ”ô¹¾ å½ÕË öÜw /proc/[pid]/ns ı̀ • User Namespace ıö•wo!? · · ̃©— pid, mount Namespace • setns x¶v›¤ - Ï äéł vœÏ äéå ”Ïþ è” w x¶v›¤ *¶O•v¶ßóU m›¤ - lxc-attach Ïþ è (OpenVZ ” vzctl exec O¶Ïþ è) wøv¶ v›¤ - - · setns w∙” Namespace •Z|–Ø|\−øq•¶ß¿ Ï äéł vœ Ï äéå ”Ïþ è³ x−øq•¶›¤ - 31/68
  32. Namespace ̀ ”NÔ ° setns(2) (kernel 3.8) 3.7 ıÚ” /proc/[pid]/ns

    ı̀ 3.8 ı ” /proc/[pid]/ns ı̀ - r - - - - - - - - 1 r o o t r o o t 0 M a r 1 1 5 : 4 1 i p c - r - - - - - - - - 1 r o o t r o o t 0 M a r 1 1 5 : 4 1 n e t - r - - - - - - - - 1 r o o t r o o t 0 M a r 1 1 5 : 4 1 u t s l r w x r w x r w x 1 r o o t r o o t 0 3 j 1 Ÿ 1 4 : 5 9 i p c - > i p c : [ 4 0 2 6 5 3 2 3 0 1 ] l r w x r w x r w x 1 r o o t r o o t 0 3 j 1 Ÿ 1 5 : 0 6 m n t - > m n t : [ 4 0 2 6 5 3 2 2 9 9 ] l r w x r w x r w x 1 r o o t r o o t 0 3 j 1 Ÿ 1 5 : 0 6 n e t - > n e t : [ 4 0 2 6 5 3 2 3 0 4 ] l r w x r w x r w x 1 r o o t r o o t 0 3 j 1 Ÿ 1 5 : 0 6 p i d - > p i d : [ 4 0 2 6 5 3 2 3 0 2 ] l r w x r w x r w x 1 r o o t r o o t 0 3 j 1 Ÿ 1 5 : 0 6 u t s - > u t s : [ 4 0 2 6 5 3 2 3 0 0 ] /proc/[pid]/ns ı̀”ô¹¾ wÃ@¶Ó ü áË Ë ¶›¤ œ∼ùÚ 7•:|–o−þ »¿ œ∼ inode ³Ë¡øq•¶›¤ · · stat() +”•œ∼ùÚ 7•:|–o−v∙qvÎ h - 32/68
  33. Namespace ̀ ”NÔ ° ƒ”H NFS ”Ïß èw Network Namespace

    ZD (Linux 3.9) ( ÜÎ ) H•cvc•¶−ò&»? · The conclusion of the 3.9 merge window (lwn.net) "The NFS code has gained network namespace support, allowing the operation of per-container NFS servers." - - · 33/68
  34. Namespace ° Ü !”̀ Ï äé” quota ³ mount namespace

    ”̀ |– ! ( ÜÎ ) · Ï äé” quota Ø'ßS||–z− t container disk quota 2012 5 j”Ÿß “w¿ ƒ”¹∙q¶›¤v»ÜÎ - - - 34/68
  35. Namespace ° Ü !”̀ Syslog Namespace Device Namespace Add namespace

    support for audit (lwn.net) ƒ”HcvÈo¶̀ »? · ûÕç Ï äé”7” syslog ”Ö wóo) Stepping closer to practical containers: "syslog" namespaces (lwn.net) LxcSyslogNs (Ubuntu Wiki) - - - · cgroup ”¼Ë×Õv¼•øß—Þz»ÎZh “w¿Namespace O•» Ñfi Çß ì vœ” uevent »Þ–•#œ©− Device Namespace (ubuntu) - - - · · 35/68
  36. Cgroups

  37. Cgroups Ì ß öa|¤ö ×Õ•Z|– Úß Õv¼³ q̀ö́ 2006 9

    j• Google ”à Ô꼕øß Containers oqðáÞwfiŠ\© − 2.6.24 (2008 ) Control Groups þß Ô (Task Control Groups) 2.6.25 Memory Resource Controller 2.6.26 Device controller (Device whitelist) 2.6.28 Freezer controller 2.6.29 Control Group Classifier (Network) 2.6.33 Block I/O controller 2.6.37 Block I/O controller I/O throttling (linux 2.6.37 ”&̀ "I/O throttling" (2)) · · · · · · · · 37/68
  38. Cgroups RHEL 6.0 Cgroups Ñýß çw‾ß¿ ®iþ꙼ ( Úß Õ*¶È¾è)

    Øm− ” Ñ0ŸÔ‾ß •!! Vʶ Øgö̃”ìÜwm›¤øq¶ hbstudy#19 ” RedHat ”<\µ”ç© ¶µv»v¶ß∼›zß ³ |¤Ð s ̃”9» ßms¢ucV•ã£¤́ gpØ°vߺ¡oØ”wIv›¤” Û |oô] ̃”¹Ø Cgroup »∙µ∙µ̀ g ̇¹w9µ o—¡́ ƒ”Rß³ı · ×w^ • Cgroup ³o∼ß“|¤”w RHEL6 Ú% oq ∼”ô] (2010 2 j9•Vʶ LT |–−) - · · · 38/68
  39. Cgroups ”NÔ ° cgroupfs ”þÁ çý¾ ç ’B−” ß cgroup

    i”ô¹¾ ÓÕ䊻∙̃•þÁ ç|–Ø OK Ĵ” : /cgroup v ?: /dev/cgroup v ¾þèÉ: /sys/fs/cgroup · · · · 2010 9? • /sys/fs/cgroup ³¢{¡−ðáÞw→ [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on - 39/68
  40. Cgroups ”NÔ ° Kernel 3.0 ns_cgroup ”£Œ́"clone_children" • · cgroup:

    remove the ns_cgroup ns_cgroup ł twm›¤Æ¤o ( ¹z−ß—£µ ^^;) 2.6.37 "clone_children" w g\©¿ns_cgroup »Øq¡{Ñs− oq −w g\©–o− lxc ”̂oïß Ô (0.7.5 Rß?) “ ÈK? 0.8.0 (0.9.0 Ø?) Ø̂oÇß ì “ ƒ«œ³³q ( I)) CentOS ¶∙”¹|ڔؔ³³q x»Ó²wÈo - - - - - 40/68
  41. Cgroups ”NÔ ° CFS bandwidth control (Kernel 3.2) p\” cgroups

    •ø− cpu ”v¼»Í =ß̃Î ”v¼“›¤ 3.2 CFS •³s− CPU ]ͳv ¡−̀ wôo¤ Ł1 » 2 fi ̇Sß³ cpu.stat <Àh • Linux 3.2 ” CFS bandwidth control (2) Linux ”CFS ³³›–ö ×Õ”CPU Ñı¾³v¼¡−âß ¢›¤ ( <7 Á õ”Ü\) · · ¼ ܾŠË Õ”ö ×Õ (SCHED_RR) •Z¡−v ̀ »Bv|–o ¤ (CONFIG_RT_GROUP_SCHED) ̃”̀ »Ô ”ÕÍ盛 Ìý Óß (SCHED_OTHER) Ë Õ•Z| – CPU ³³i¡−ô7³v ¡−Ø” - - · cpu.cfs_period_us ( çß –”øô7”Ł1) cpu.cfs_quota_us ( çß –ö ß) cpu.cfs_period_us ô7å cpu.cfs_quota_us “¦ CPU ³³s− - - - · · · 41/68
  42. Cgroups ”NÔ ° CFS bandwidth control (Kernel 3.2) v ß”u

    ( œ∼»¶³|–o−ö ×Õ•œ∼ß) Oz“¦)º¡ # e c h o 5 0 0 0 > / s y s / f s / c g r o u p / c p u / t e s t 1 / c p u . c f s _ q u o t a _ u s # e c h o 5 0 0 0 > / s y s / f s / c g r o u p / c p u / t e s t 2 / c p u . c f s _ q u o t a _ u s # p s a u x P I D U S E R P R N I V I R T R E S S H R S % C P U % M E M T I M E + C O M M A N D 3 1 4 6 k a r m a 2 0 0 1 9 1 0 4 2 2 0 4 1 5 4 0 R 5 0 . 0 0 : 4 2 . 5 2 b a s h 3 1 6 8 k a r m a 2 0 0 1 9 1 0 4 2 2 0 8 1 5 4 0 R 5 0 . 0 0 : 4 2 . 5 0 b a s h # e c h o 1 0 0 0 0 > / s y s / f s / c g r o u p / c p u / t e s t 2 / c p u . c f s _ q u o t a _ u s # p s a u x P I D U S E R P R N I V I R T R E S S H R S % C P U % M E M T I M E + C O M M A N D 3 1 4 6 k a r m a 2 0 0 1 9 1 0 4 2 2 0 4 1 5 4 0 R 1 0 0 . 0 2 : 1 3 . 1 1 b a s h 3 1 6 8 k a r m a 2 0 0 1 9 1 0 4 2 2 0 8 1 5 4 0 R 5 0 . 0 2 : 0 4 . 3 9 b a s h 42/68
  43. Cgroups ”NÔ ° Per-cgroup TCP buffer limits (Kernel 3.3) Memory

    Controller Çß ì ŸŽ •Z¡−v ³ qrÑŒ ̃©— »žß Ò 7”v ”Æ 9÷¡− » 2 fi Vocó« ã|–Ƥؔ”cvøxw]^ Èqøq¶... ∙qoq̀ v”•ºw¶o! Çß ì ô:5̈•¹|m−w¿ gç∙qoq̀ v°vœ¶o ( cØ̈o–¶o” p\ ß•̀ ¡−µ“¬q„¿  (| –¤¦∙È›¤ ^^;)́ m›¤œÐs–z“\ó · · ... »»s¿ Ì7»¹|U  ł©−Ø” |¤ - · memory.kmem.tcp.limit_in_bytes ( v ß”Ł1) memory.kmem.tcp.usage_in_bytes ( ‚v”³i¾) - - · · 43/68
  44. Cgroups ”NÔ ° Per-cgroup TCP buffer limits (Kernel 3.3) gç¿

    Ïß è³ÊªîŸ• ( ¶” ı̀» |ovØ?)́ Memory Controller ”°ôÆ (Resource Counter) ³³›–»o−Ø””¿ ̃” °ôÆå »v »vv›–¶o ( e) p\vœBv¡− sysctl ð Ÿß Ü” net.ipv4.tcp_mem w cgroup ’ •Qi \©− oqØ”“›¤ Ł1|¤ memory.kmem.tcp.limit_in_bytes wÌ ß öÀ” tcp_mem •’ \ ©− · · · · tcp_mem = min pressure max oqŁ1”ô¿limit ”ßw... limit < min ”ô¿"limit limit limit" min <= limit < pressure ”ô "min limit limit" pressure <= limit < max ”ô "min pressure limit" max <= limit ”ô¿ fi” tcp_mem ”ß - - - - - 44/68
  45. Cgroups ”NÔ ° Per-cgroup TCP buffer limits (Kernel 3.3) «¶Æ•¹

    !\©−H”Çß ì ŸŽ ”v ØÈq ! ̃”̀ ³EÒ•|¤Çß ì “w̃”̀ ³³°¶ožß Ò ( q”Ì”å½Õ ç ò™ß Ó ”žß Ò”Iz) ”z ëaw¶oøq•¿ oqz:” ! ”gp Ê°©− Linux 3.3 ”&̀ Per-cgroup TCP buffer limits Linux 3.3 ”&̀ Per-cgroup TCP buffer limits (2) Linux 3.3 ”&̀ Per-cgroup TCP buffer limits (3) Per-cgroup TCP buffer limits (lwn.net) · · · · · · 45/68
  46. Cgroups ”NÔ ° Network priority cgroup (Kernel 3.3) ̃«œ»óž¶̀ :-)

    ö ×ÕÌ ß ö”Åìáç ß Ë¾ Üß ôÂß Õ•Z¡−=ß̃³Ł1¡− 9÷¡− » 2 fi · · · net_prio.prioidx ( Çß ì wå ³i¡−Ì ß ö³Ú¡ß) net_prio.ifpriomap ( ž Üß ôÂß Õ•Z¡−=ß̃) - - $ c a t / s y s / f s / c g r o u p / n e t _ p r i o / n e t _ p r i o . i f p r i o m a p l o 0 e t h 1 0 e t h 0 0 46/68
  47. Cgroups ”NÔ ° Network priority cgroup (Kernel 3.3) =ß̃”Ł1 iperf

    ³ |¤gp net_prio.ifpriomap # e c h o " e t h 0 1 " > / s y s / f s / c g r o u p / n e t _ p r i o / t e s t 1 / n e t _ p r i o . i f p r i o m a p # e c h o " e t h 0 1 0 0 " > / s y s / f s / c g r o u p / n e t _ p r i o / t e s t 2 / n e t _ p r i o . i f p r i o m a p [ 4 ] 0 . 0 - 2 0 . 5 s e c 2 . 1 7 G B y t e s 9 0 8 M b i t s / s e c < = p r i o r i t y 1 0 0 ”z [ 5 ] 0 . 0 - 2 0 . 6 s e c 7 1 . 2 M B y t e s 2 9 . 1 M b i t s / s e c < = p r i o r i t y 1 ”z # c a t / s y s / f s / c g r o u p / n e t _ p r i o / t e s t 1 / n e t _ p r i o . i f p r i o m a p e t h 0 1 # c a t / s y s / f s / c g r o u p / n e t _ p r i o / t e s t 2 / n e t _ p r i o . i f p r i o m a p e t h 0 1 0 0 47/68
  48. Cgroups ”NÔ ° Network priority cgroup (Kernel 3.3) Linux 3.3

    ”&̀ Network priority cgroup Linux 3.3 ”&̀ Network priority cgroup (2) net: add network priority cgroup infrastructure (v4) · · · 48/68
  49. Cgroups ”NÔ ° HugeTLB cgroup (Kernel 3.6) ¡Æ—£µ́ ü̊–¶o” −ß—£µ

    ^^; ̃©Øóž• Resource Counter ³³›–−›̌o mm/hugetlb: add new HugeTLB cgroup · · · 49/68
  50. Cgroups ”NÔ ° Memory Controller ” Kernel Memory Ñýß ç

    (Kernel 3.8) ÕÜáË Õ õ”³i¾”¼ÇÁ ä½ Ì³Ñýß ç Ÿ • Resource Counter ³³›–o− Ÿ •Ł1¡© ]^ ß•øz ozfivÓ²ewm−́ ̃©ØðôÄß þ Õ³£ \¶o¤Ł”áö”Ñ% · · · · 50/68
  51. Cgroups ”NÔ ° Memory Controller ” Kernel Memory Ñýß ç”Ó²e

    ( H” Memory Controller œe) root Ì ß ö•Z¡−v »flv¶o root Ì ß ö” usage »¼ä•¶œ¶ó ÄÌ ß ö ÇÁ ç\©¤ŸŽ ³i¾»ÇÁ ç\©−vØ|©¶o|\©¶ovØ|©¶o ³i¾”ÇÁ ç»v ³Ł1|–vœÁ—− · · · Ì ß ö³¢{¡−“¦ »Á—œ¶o Ñ̃¿ ÇÁ ç\©ÁŁ− ¿ Ì ß öå•ÜÕËw¶z¶›–Ø¿ Ì ß öŠXw¶z¶−— »ÇÁ ç\©− Ñ̃¿ v ³Ł1|–ÇÁ çwÁ—− ¿ v ³•Œ ( echo -1 > memory.kmem.limit_in_bytes ) |–ØÇÁ ç\©− - - - 51/68
  52. Cgroups ”NÔ ° Memory Controller ” Kernel Memory Ñýß ç”Ó²e

    v ³Ł1 x¶oÍß Õ Çß ì ŸŽ ”³i¾»Ì ß ï ”³i¾•Ø7\©− · ÄÌ ß ö³¢{|¤¹ Ì ß ö•ÜÕËwBv¡−þ - - · memory.kmem.usage_in_bytes •7\©¤ß»¿ œô• memory.usage_in_bytes •Ø7\©− - ¤“|¿ œ∼Çß ì ŸŽ ” memory.kmem.tcp.usage_in_bytes ” ß»7\©¶o ( Ê°©−) - 52/68
  53. Cgroups ”NÔ ° ƒ”H xattr Ñýß ç (3.7) Memory Controller

    · security.* trusted.* ”Æßh - · Çïß ¡−fi−wåo” Ÿ•̇¹wgsœ©–o− - memory.numastat g¿stats úß ÔôÄß çÇÁ ç (3.0) Åß ïß ÷á蔣∙ Integrating memory control groups (lwn.net) (3.3) - - 53/68
  54. Cgroups ”:¹ ∙qØ cgroup ÞX Ñ4zw¶v›¤ß|¶o?( I)) memcg: Add memory.pressure_level

    events devcg: introduce proper hierarchy support (3.10?) perf, cgroup: implement hierarchy support for perf_event controller (3.10?) memcg: make memcg's life cycle the same as cgroup soft limit rework · sane_behavior ÅöÓ :Ø (3.10?) Fixing control groups - - · The mempressure control group proposal 3.10 ‾−!? → memory.txt (3.10) - - · · · · 54/68
  55. ƒ”H9÷|ƒq¶Çß ì ̀

  56. CRIU a project to implement checkpoint/restore functionality for Linux in

    userspace. ã|¤”w¹|Ú¶” :»“ŏøxwN°›–o−v ( Øowv¶ßëo¶Û î) http://criu.org/ · CRIU(1) CRIU(2) 9÷¡−Çß ì ̀ · · · checkpoint/restart i” /proc ”Ã ç ” g (3.3) TCP connection repair (3.5) /proc/[pid]/task/[tid]/children à ç g (3.5) /proc/[pid]/pagemap ” checkpoint/restart Ö¦”à ç g (3.5) - - http://criu.org/TCP_connection - - - 56/68
  57. LXC LXC ”9a

  58. LXC ”é¼ Linux Container = Namespace + cgroup + žß

    ÒÕúß Õ”âß (lxc) 2008 9vœ IBM ô Õ” Daniel Lezcano Ôœ•øßØo 0.6.5 Rß Q©O¶̀ »ø¢|–o¤ 0.7.5 ı ¿ ªł • Daniel Lezcano Ô”øxwâz¶− Ø•¿Serge Hallyn, Stéphane Graber œÔ (Canonical) wÚ •|–x¤́ œÔ Ø Ubuntu ”åù áðß mß¿lxc ”Øo» Ubuntu Ÿ¾ 9ª Ubuntu •ß<ß !\©¤̀ ³Ÿ¾ â ß •þß Ô|–oz ∼ Øo w9ª ^Š»Øo•̨g¡−<w)s–o−øq¶cw¡− Github(https://github.com/lxc/lxc) Øo|¿ m−?̃ !w̆—›¤ ̨ß sourceforge ” ýÔç •þß Ô¡−= Øow9ªøq•¶›¤ Ubuntu ”ðáÍß Ô•Qi\©¤ðáÞ³©l•þß Ô · · · · · Ubuntu »øo–vœ Fedora (systemd) øzøq• g¿ ü}|–o¤ - · · · · 12.04LTS ” lxc » 0.7.5 “w¿ ̨̋ 0.8.0 •́|o ( cw¡−) (86 ”ðáÞ) - 58/68
  59. Ubuntu 12.04LTS ” Ubuntu / lxc-0.8.0 &̀ Q©O¶ !whµ“Ûî” 0.7.5

    ”pvo ³ ł ̇¹|¤Ûî + Ubuntu ¶ œ »” Tò Ubuntu Weekly Recipe r226 ‚»LXC V¾b %È”ëi (gihyo.jp) · Apparmor Ñýß ç (Ubuntu lxc ‾©− ƒ©i” profile Ø‾›–cØ s¢•×É™¼( Ï äé→ 5 oqàe) ¶Ï äéwnø) lxc-create, lxc-clone LVM, Btrfs Ñýß ç Ï äé” rootfs ³µzþ w ìÕçö.”Ï äé (Ubuntu 12.04 “ ìÕçi” apparmor profile È ó12.10 ìÕçi profile Ø?—©−) ä ö ß ç” g (Arch Linux, ALT Linux) [Ubuntu] ARM Ï ä锢{ (QEMU ”Ãÿ™ ß Ó ) [Ubuntu] lxc-start-ephemeral g - - - Btrfs ”þ ¿rootfs » subvolume •́clone ”ô» snapshot ³<− LVM ³³qøqË1¡− ¿create » LV ³¢{¿ ô¹¾ ÓÕ䊳 ¢›– rootfs •¡−́clone » LVM ” snapshot - - - - - - · 59/68
  60. lxc 0.9.0 &̀ ¤̆µ Ubuntu 12.10 ” lxc (0.8.0-rc1) ØÑ

    ̀ wïáËýß ç\©–o− liblxc API ÐØ API python, lua ï¾ å½ Ì (python3) seccomp Ñýß ç Ï äénø¿ +Òô”ÅłŒ ”ôáËwh • ìáç ß Ë down ô• ¡−ÕË öç³Ë1h • (up ô»ıÚvœm› ¤) · · · Ñ Ó ̈v©–o¤Ïþ è³ python ̈xž| - · · pre-start, pre-mount, mount, autodev, start, post-stop - · 60/68
  61. lxc 0.9.0 &̀ lxc-start-ephemeral g (python3 Èo) Oracle Linux ä

    ö ß ç Ï äé” /dev •^* ”åï¾Õ³ŠøO•¢{h • lxc-attach ̇¹ lxc-setcap, lxc-setuid £Œ · · · · %ÈNc»ß¶∙ ƒØƒØ lxc-attach »ıÚvœBv»|–o¤w¿ ðáÞ³ –¤ÃI¶Ç ß ì |vøv¶v›¤” ¿ ì ö³s¶v›¤́Ubuntu 13.04 3.8 Çß ì wei\©¤” ³s−øq•́ - - · 61/68
  62. lxc 1.0 ̈Ö¦– uƒœzõ” LTS ( Ø«¬µ Ubuntu) ³Üß Îáç•

    lxc-1.0 ³ s–−µ∼Œ¶o v¶ß lxc-1.0 •Ö¦– s−̃ → [lxc-devel] 0.9 final release, plans for 1.0 and Linux Plumbers 2013 libvirt ” lxc è ¾ï³ liblxc (LXC ” API) ùß Õ• User Namespace Ñýß ç þ ÞÕ áè”Ïþ èÚÍáç API ”}© (stable ¶ API •) Õç ß ÔïáËà è ( ö ̾ O• ł gh ¶øq•) · · · c”Ï äéwnø|–−ô¿ œô•Ï äé³'à¡−øq¶Ïþ è w x¶o (lxc-wait, lxc-monitor) :¿lxc-monitord Ïþ è v∼ ¿ Øo\©–o− → [lxc-devel] [RFC PATCH] allow multiple monitor clients - - · · 62/68
  63. lxc 1.0 ̈Ö¦– lxc-attach ̈x s → [lxc-devel] [PATCH] [RFC]

    Complete rewrite of lxc-attach functionality overlayfs ZD (lxc-start-ephemeral ?) zfs ZD (create, clone) · :» lxc-attach - Äö ×Õ - Cö ×Õ ” 3 Œ™•¶›–o−”³ lxc- attach - Äö ×Õx2 ” 2 Œ™• þ ÞÕ áè¶ö Ì ŠvœØ—Þ• attach x−øq• ( I)) - - · · 63/68
  64. ƒ”H

  65. ƒ”H lxc JP Ì ß ö lxc man pages ¬/

    · Ÿ©Ã žß ÒwÏ äé” ³|\−þ ̃qoqþw¶v›¤”»¿ ßms¢øv¡”»+”“vœ? —›¤ß £ßöw›–—¡ - - - · ×ẃ< º›–—¡ ‚v 0.9.0 ”¬/ÐØÌ ( ¤| Š íß ÞÂáË) ô|(dj́ ƒ”Hcflo¤ »tøu−œ£z“\o - - - 65/68
  66. \o’• 3.8 Çß ì øqºz Linux Çß ì ”̀ “¦

    Ï äé³ |¿ ži x−%Èw}o—|¤ Ï äé³+”•ã¡%ÈØ}›–—¡ Ÿł¿ ∙µ∙µ̇¹wgsœ©¿ &|ò Ø∙µ∙µ|–x–¿ w £¶o )( ¡ :¹ØÏ äéºRRz∼” t £ßöw›–ox¤o ¡ · · · · 66/68
  67. ̨ 5’ http://lxc.sourceforge.net/ http://www.slideshare.net/enakai/lxc-8300191 http://www.slideshare.net/masahide_yamamoto/osc2011-nagoya https://www.nic.ad.jp/ja/materials/iw/2012/proceedings/d1/d1-Ebisawa.pdf http://lc.linux.or.jp/lc2008/slide/bof-04-slide.pdf http://www.landley.net/kdocs/ols/2007/ols2007v2-pages-45-58.pdf http://www.slideshare.net/christophm/linuxcon-barcelon-2012-lxc-best- practices

    http://htaira.fedorapeople.org/hbstudy19/hbstudy19-cgroups.pdf http://www.slideshare.net/mkouhei/lxc-cf201207presen · · · · · · · · · 67/68
  68. <Thank You!> twitter @ten_forward www www.ten-forward.ws/ github github.com/tenforward