Erlang on OS X

Experience report on developing OS X desktop apps development with Erlang and DTrace

Transcript

1. Erlang on OS X @darkproger
2. Erlang on OS X @darkproger
3. When your MacBook is actually a production system
5. OS X Apps Cool UI Animations ??? Profit!
6. OS X Apps distributed bg + ui concurrent event loops + costly computations fault-tolerant if (syscall() != 0) really_do_something(); reliable Force Quit secure .ssh/id_rsa on github
7. OS X launchd(8) xpc(3) codesign(1) sandbox(7) security(1)
8. OS X # dtrace -Cs darwinkit/sys/sockets.d -c 'ping -c1 google.com' PING google.com (173.194.70.101): 56 data bytes 64 bytes from 173.194.70.101: icmp_seq=0 ttl=50 time=38.134 ms --- google.com ping statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 38.134/38.134/38.134/0.000 ms connect_nocancel(un) ping(4052) /var/run/mDNSResponder 24 ok accept(un) mDNSResponder(36) 11 ok bind(in4) mDNSResponder(36) 0.0.0.0:64083 15 ok bind(in6) mDNSResponder(36) ?:64083 4 ok https://github.com/proger/darwinkit
9. OS X for hardcore Linux guys launchd(8) xpc(3) codesign(1) sandbox(7) security(1) /etc/rc.local socket(2) ---- ---- ~/.passwords.txt /etc/openssl
10. OS X for Linux guys launchd(8) xpc(3) codesign(1) sandbox(7) security(1) systemd dbus debsign apparmor gnome-keyring
11. Desktop Apps event-based no polling on-demand activation responsiveness power consumption!
13. Any sufficiently complicated concurrent program in another language contains an ad-hoc, informally specified, bug-ridden, slow implementation of half of Erlang http://article.gmane.org/gmane.comp.lang.erlang.general/27010
14. Erlang Systems distributed scalable concurrent fault-tolerant reliable robust debuggable
15. Erlang Quick Start 2 weeks (emacs: 4 weeks, based on personal experience) http://kirillov.im/erlang101.ru.pdf
16. HelloWorld (Objective-C, Ruby, Python) HelloWorldHelper IPC
17. UI State IPC
18. % pgrep -lf iTunes 22926 /Applications/iTunes.app/Contents/MacOS/iTunes - psn_0_25565280 453 /Applications/iTunes.app/Contents/MacOS/iTunesHelper.app/ Contents/MacOS/iTunesHelper -psn_0_131104 % kill -9 22926 % pgrep -lf iTunes 453 /Applications/iTunes.app/Contents/MacOS/iTunesHelper.app/ Contents/MacOS/iTunesHelper -psn_0_131104 ... % pgrep -lf iTunes 29673 /Applications/iTunes.app/Contents/MacOS/iTunes - psn_0_30412031 453 /Applications/iTunes.app/Contents/MacOS/iTunesHelper.app/ Contents/MacOS/iTunesHelper -psn_0_131104
19. network client filesystem driver UI case study: remote filesystem
20. network client filesystem driver UI case study: remote filesystem server:
21. WCF net.tcp MC-NMF http://msdn.microsoft.com/en-us/library/cc219293.aspx MC-NBFSE http://msdn.microsoft.com/en-us/library/cc219190.aspx SOAP https://github.com/proger/net_tcp
22. MC-NMF
23. Erlang the Language
24. binary syntax preamble(duplex = Mode, Via, Encoding) when is_binary(Via) -> M = mode(Mode), ViaLen = msvlq:encode(byte_size(Via)), Enc = encoding(Encoding), << ?REC_VERSION, 1, 0, ?REC_MODE, M, ?REC_VIA, ViaLen/binary, Via/binary, ?REC_KNOWN_ENCODING, Enc, ?REC_PREAMBLE_END >>.
25. macros -define(DATA(Format, Message), data(Format, Pid) -> Pid ! {record, Message}, data(Rest, Pid)).
26. pattern matching ?DATA( <<?REC_VERSION:8, _Maj:8, _Min:8, Rest/binary>>, version); ?DATAVAR( <<?REC_VIA:8, Rest/binary>>, {via, Str}); ?DATA( <<?REC_MODE:8, Mode:8, Rest/binary>>, {mode, mode(Mode)});
27. enums (pattern matching) mode(1) -> singleton_unsized; mode(2) -> duplex; mode(3) -> simplex; mode(4) -> singleton_sized; mode(_) -> undefined.
28. reverse enums mode(1) -> singleton_unsized; mode(singleton_unsized) -> 1; mode(2) -> duplex; mode(duplex) -> 2; mode(3) -> simplex; mode(simplex) -> 3; mode(4) -> singleton_sized; mode(singleton_sized) -> 4; mode(_) -> undefined.
29. parse transforms -match_reverse(mode). mode(1) -> singleton_unsized; mode(2) -> duplex; mode(3) -> simplex; mode(4) -> singleton_sized; mode(_) -> undefined. * for those who suck at emacs https://github.com/proger/net_tcp/blob/master/src/match_reverse_transform.erl
30. ah, it’s also functional immutability return >>=
31. Erlang the Operating System
32. processes lightweight M:N reductions scheduling
33. shared nothing concurrent gc separate heaps messages always copied except binaries >64b
34. sync https://github.com/rustyio/sync
35. sync ⌃X ⌃S
36. sync
37. rebar % cat rebar.config {deps_dir, "apps"}. {deps, [ {net_tcp, ".*", {git, "../../net_tcp", {branch, "master"}}} ]}. {erl_opts, [ debug_info ]}. {sub_dirs, ["rel"]}.
38. rebar % rebar get-deps % rebar eunit % rebar compile
39. erlang node % rebar create-node nodeid=wcf % emacsclient reltool.config
40. % du -hsx rel/wcf 16M!rel/wcf
41. clustering % erl -sname ewcf \ -s lager -s sync -s net_tcp % erl -sname client -remsh ewcf@uniq
42. Debugging! My favorite part
43. dbg > dbg:c(io, format, ["~p~n", [hello]]). hello (<0.3018.0>) <0.30.0> ! {io_request,<0.3018.0>,<0.30.0>, {put_chars,unicode,io_lib,format, ["~p~n",[hello]]}} (Timestamp: {1359,184994,408882}) (<0.3018.0>) out {io,wait_io_mon_reply,2} (Timestamp: {1359,184994,408894}) (<0.3018.0>) << {io_reply,<0.30.0>,ok} (Timestamp: {1359,184994,408911}) (<0.3018.0>) in {io,wait_io_mon_reply,2} (Timestamp: {1359,184994,408940}) (<0.3018.0>) << timeout (Timestamp: {1359,184994,408945}) ok > self(). <0.50.0>
44. DTrace % dtrace -P erlang$(pgrep -f 'beam.*sname.ewcf') dtrace: description 'erlang4354' matched 1082 probes CPU ID FUNCTION:NAME 2 214164 process_main:process-scheduled 2 214138 process_main:local-function-entry 2 214138 process_main:local-function-entry 2 214138 process_main:local-function-entry 2 214134 fixed_apply:global-function-entry 2 214138 process_main:local-function-entry 2 214138 process_main:local-function-entry 2 214043 copy_struct:copy-struct 2 214136 process_main:global-function-entry ... wtf? the node seems to be doing nothing... run for a little while
45. dtrace -p $pid -Cs darwinkit/sys/machtrace.d ... 4354/0x13e9b: 10429636 11 stat64("/tank/proger/dev/net_tcp/src/net_tcp_app.erl\0", 0xB032D5A8, 0x7F9EC2169E20) = 0 0 4354/0x13e9b: 10429638 4 access("/tank/proger/dev/net_tcp/src/net_tcp_app.erl\0", 0x4, 0x0) = 0 0 4354/0x13e9b: 10429640 4 access("/tank/proger/dev/net_tcp/src/net_tcp_app.erl\0", 0x2, 0x0) = 0 0 4354/0x13e9b: 10429666 6 stat64("/tank/proger/dev/net_tcp/src/net_tcp_sup.erl\0", 0xB032D5A8, 0x7F9EC2169E20) = 0 0 4354/0x13e9b: 10429668 4 access("/tank/proger/dev/net_tcp/src/net_tcp_sup.erl\0", 0x4, 0x0) = 0 0 ... syscall close_nocancel 30 syscall fstatfs64 30 syscall open_nocancel 30 syscall getdirentries64 60 syscall select 114 syscall stat64 239 syscall access 382 wtf?! run for a little while
46. % dtrace -p $pid \ -n 'stat64:entry /pid == $target && strstr(copyinstr(arg0), "src/nmf.erl") != 0/ { self->t = 1; trace(copyinstr(arg0)); }' \ -n 'erlang$target:::process-unscheduled \ /self->t/ \ { printf("%s", copyinstr(arg0)); exit(0); }' CPU ID FUNCTION:NAME 2 815 stat64:entry /tank/proger/dev/net_tcp/src/nmf.erl 2 214166 schedule:process-unscheduled <0.23.0> this guy
47. > erlang:process_info(pid(0,23,0)). [{registered_name,file_server_2}, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, ... ] is that guy
48. % dtrace -p $pid \ -n 'erlang$target:::message-send /copyinstr(arg1) == "<0.23.0>"/ { @[copyinstr(arg0)] = count() }' aggregate by sender pid look for all messages to file_server_2
49. % dtrace -p $pid \ -n 'erlang$target:::message-send /copyinstr(arg1) == "<0.23.0>"/ { @[copyinstr(arg0)] = count() }' dtrace: description 'erlang$target:::message-send ' matched 3 probes ^C <0.63.0> 34 run for a little while aggregate by sender pid look for all messages to file_server_2 this guy this many times
50. > erlang:process_info(pid(0,63,0)). [{registered_name,sync_scanner}, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, ... ] this guy is that sync module which does polling which kinda sucks
51. no code was harmed during this debugging session modified
52. @darkproger http://kirillov.im kthxbai!