$30 off During Our Annual Pro Sale. View Details »

Introductions to Node.js internals

Introductions to Node.js internals

Node.js 底层初探

Minqi Pan

June 17, 2016

More Decks by Minqi Pan

Other Decks in Programming


  1. Node.js 底层初探 Minqi Pan

  2. I’m Minqi Pan github.com/pmq20 twitter @psvr

  3. None
  4. None
  5. (evented)
 a model of I/O
 + (JavaScript)
 a language that

    was well suited to the style of programming = Node.js
  6. 浏览器的影⼦子 • v8 “上下⽂文”的设计:each window and iframe in a browser

    can have its own fresh JavaScript environment. [提问 node] • v8 限制堆的⼤大⼩小,浏览器不太可能⽤用⼤大量量内存 • node::Buffer (跳过 V8的内存分配,堆外内存)
 与 v8::Uint8Array / TypedArray 的故事 • CESU-8 encoding
  7. None
  8. None
  9. C++ 11 (env.cc) auto close_and_finish = [](Environment* env, uv_handle_t* handle,

    void* arg) { … }
  10. 启动过程 node_main.cc * main() void LoadEnvironment(Environment* env) Local<String> script_name =

    FIXED_ONE_BYTE_STRING(env->isolate(), "node.js"); * node.js * [hack]
  11. API 源码定位

  12. JavaScript API • v8 内置 API:v8/src/js/array.js … v8/src/js/weak- collection.js •

    Node.js ⽂文档记载的公开 API:assert.js … zlib.js • Node.js 的私有 API:internal/* 
 与公开 API 同名,internal/bootstrap_node.js 除外 • API ⼦子⽂文件:_debug_agent.js … _tls_wrap.js
  13. C++ API • process.binding 系列列 • 注册发⽣生在每个 *.cc ⽂文件的最后

    • 引⽤用⽅方式 [demo]:
  14. v8 内置 API • create the built-in objects and parse

    the built-in JavaScript code when creating v8::Context’s
  15. v8 内置 API • v8/src/js/array.js => global.Array • utils.InstallFunctions

  16. v8 内置 API • v8/src/js/math.js => global.Math • e.g. Math.atan2

    => MathAtan2JS => src/runtime/ runtime-maths.cc => Runtime_MathAtan2 => std::atan2 => cmath => glibc => __ieee754_atan2 • utils.InstallFunctions
  17. v8 内置 API • v8/src/js/i18n.js => global.Intl • ECMA-402 •

    Default: `small-icu` (English only) support • Intl.DateTimeFormat(['zh-CN']).format(new Date()) • Intl.DateTimeFormat(['en']).format(new Date())
  18. Node.js ⽂文档记载的公开 API
 Node.js 的私有 API • lib/net.js -> internalNet

    (私有⼯工具⽅方法) • lib/dns.js
 -> C++ API:cares_wrap
 -> exports.resolve6
 -> queryAaaa
 -> Query<QueryAaaaWrap>
 -> ares_query
 -> deps/cares
 -> ares_send with callback
  19. API ⼦子⽂文件 • lib/http.js
 -> const client = require('_http_client')

    暴暴露辅助类 ClientRequest • lib/stream.js
 -> Stream.Transform = require('_stream_transform')
 -> const Duplex = require(‘_stream_duplex’)
 -> module.exports = Duplex;
  20. JS API 转存为 C/C++ 代码 • Node采⽤用了了V8附带的js2c.py⼯工具,将所有内置的 JavaScript代码(src/node.js和lib/*.js)转换成C++ ⾥里里的数组,⽣生成node_natives.h头⽂文件 •

    加载JavaScript核⼼心模块->标识符分析->直接定位到 内存中
  21. v8 初探

  22. 重要概念念 • 隔离区 • 句句柄 • 上下⽂文

  23. “隔离区” • class v8::Isolate {} • An isolate is a

    VM instance with its own heap • ⼀一个隔离区:⼀一个 v8 堆
  24. 句句柄之“局部句句柄” • template <class T> class Local {} • Because

    the garbage collector may move objects, it is unsafe to point directly to an object.
  25. 句句柄之“局部句句柄” • template <class T> class Local {} • An

    object reference managed by the v8 garbage collector. • A local handle is a pointer to an object. All V8 objects are accessed using handles, they are necessary because of the way the V8 garbage collector works. • ⼀一个局部句句柄:⼀一个 v8 堆上的对象
  26. 句句柄之“持久句句柄” • Used when keeping a reference to an object

    for more than one function call, or when handle lifetimes do not correspond to C++ scopes. • A UniquePersistent<SomeType> handle relies on C++ constructors and destructors to manage the lifetime of the underlying object. • A Persistent<SomeType> can be constructed with its constructor, but must be explicitly cleared with Persistent::Reset.
  27. “句句柄区域” • class V8_EXPORT HandleScope {} • A stack-allocated class

    that governs a number of local handles. • a container for any number of handles. When you've finished with your handles, instead of deleting each one individually you can simply delete their scope. • ⼀一个句句柄区域:N 个句句柄
  28. ~HandleScope, is called the handle scope is deleted. Objects referred

    to by handles within the deleted handle scope are eligible for removal in the next garbage collection
  29. “上下⽂文”

  30. “上下⽂文” • class V8_EXPORT Context {} • JavaScript provides a

    set of built-in utility functions and objects that can be changed by JavaScript code. • Creation of contexts implies creating the built-in objects and parsing the built-in JavaScript code (with cache) • each window and iframe in a browser can have its own fresh JavaScript environment.
  31. “上下⽂文” • class V8_EXPORT Context {} • A sandboxed execution

    context with its own set of built-in objects and functions. • an execution environment that allows separate, unrelated, JavaScript code to run in a single instance of V8. You must explicitly specify the context in which you want any JavaScript code to be run.
  32. v8 GC 特征 • “stop-the-world” • “generational” • “accurate”

  33. v8 GC 特征 • “stop-the-world”
 stops program execution when performing

    a garbage collection cycle • “generational”
 processes only part of the object heap in most garbage collection cycles. This minimises the impact of stopping the application • “accurate”
 always knows exactly where all objects and pointers are in memory. This avoids falsely identifying objects as pointers which can result in memory leaks.
  34. None
  35. libuv 初探

  36. libuv • supports Linux via epoll • supports Windows via

    Microsoft IOCP • supports Mac OS X via kqueue • same API on Linux, Windows, Mac • Little dependencies (On node-v0.9.0's version of libuv, the dependency on libev was removed)
  37. None
  38. None
  39. #include <stdio.h> #include <stdlib.h> #include <uv.h> int main() { uv_loop_t

    *loop = malloc(sizeof(uv_loop_t)); uv_loop_init(loop); uv_run(loop, UV_RUN_DEFAULT); uv_loop_close(loop); free(loop); return 0; }
  40. “句句柄” • Handles represent long-lived objects capable of performing certain

    operations while active. • e.g.: a prepare handle gets its callback called once every loop iteration when active • e.g.: a TCP server handle get its connection callback called every time there is a new connection.
  41. “句句柄” • 持久性对象 • 句句柄可以对应I/O设备,定时器,进程等 • 句句柄是不透明的数据结构,其中对应的类型 uv_TYPE_t中的type指定了了handle的使⽤用⽬目的

  42. typedef struct uv_loop_s uv_loop_t; typedef struct uv_handle_s uv_handle_t; typedef struct

    uv_stream_s uv_stream_t; typedef struct uv_tcp_s uv_tcp_t; typedef struct uv_udp_s uv_udp_t; typedef struct uv_pipe_s uv_pipe_t; typedef struct uv_tty_s uv_tty_t; typedef struct uv_poll_s uv_poll_t; typedef struct uv_timer_s uv_timer_t; typedef struct uv_prepare_s uv_prepare_t; typedef struct uv_check_s uv_check_t; typedef struct uv_idle_s uv_idle_t; typedef struct uv_async_s uv_async_t; typedef struct uv_process_s uv_process_t; typedef struct uv_fs_event_s uv_fs_event_t; typedef struct uv_fs_poll_s uv_fs_poll_t; typedef struct uv_signal_s uv_signal_t;
  43. “请求” • Requests represent (typically) short-lived operations. These operations can

    be performed over a handle • e.g.: write requests are used to write data on a handle • e.g.(standalone): getaddrinfo requests don’t need a handle they run directly on the loop
  44. “请求” • 短暂性对象(通常只维持在⼀一个回调函数的时间) • 通常对映着句句柄上的⼀一个I/O操作 • ⽤用来在初始函数和回调函数之间,传递上下⽂文

  45. typedef struct uv_req_s uv_req_t; typedef struct uv_getaddrinfo_s uv_getaddrinfo_t; typedef struct

    uv_getnameinfo_s uv_getnameinfo_t; typedef struct uv_shutdown_s uv_shutdown_t; typedef struct uv_write_s uv_write_t; typedef struct uv_connect_s uv_connect_t; typedef struct uv_udp_send_s uv_udp_send_t; typedef struct uv_fs_s uv_fs_t; typedef struct uv_work_s uv_work_t;
  46. 事件循环 • tied to a single thread • all (network)

    I/O is performed on non-blocking sockets which are polled using the best mechanism available on the given platform • the loop will block waiting for I/O activity on sockets which have been added to the poller and callbacks will be fired indicating socket conditions (readable, writable hangup) so handles can read, write or perform the desired I/O operation.
  47. None
  48. 线程池 • a global thread pool on which all loops

    can queue work on • runs blocking file I/O operations • runs DNS functions (getaddrinfo and getnameinfo) • runs User specified code via uv_queue_work()
  49. (possibly) • linux AIO (supported in the kernel) • posix

    AIO (supported by linux, Mac OS X, BSD, solaris, AIX etc.) • Windows’ overlapped I/O
  50. Thank you https://github.com/pmq20/