Slide 1

Slide 1 text

kernel offload with complete host kernel offload with complete host kernel functionalities kernel functionalities  1

Slide 2

Slide 2 text

TCP Offload Engine (ToE) TCP Offload Engine (ToE) 2

Slide 3

Slide 3 text

TCP Offload Engine (ToE) TCP Offload Engine (ToE) 2

Slide 4

Slide 4 text

attempt to upstream ToE in Linux attempt to upstream ToE in Linux 3

Slide 5

Slide 5 text

why ToE was rejected ? why ToE was rejected ? 4

Slide 6

Slide 6 text

reasons of ToE rejected reasons of ToE rejected 5

Slide 7

Slide 7 text

what can we do ? what can we do ? 6

Slide 8

Slide 8 text

our attempt: kernel offload our attempt: kernel offload 7

Slide 9

Slide 9 text

internals internals 8

Slide 10

Slide 10 text

Host part: minoc Host part: minoc 9

Slide 11

Slide 11 text

NIC part: minod NIC part: minod copy_from_user() copy_to_user() 10

Slide 12

Slide 12 text

multiple implementations of NIC multiple implementations of NIC side side 11

Slide 13

Slide 13 text

alternatives alternatives 12

Slide 14

Slide 14 text

alternatives alternatives 13

Slide 15

Slide 15 text

Demo Demo 14

Slide 16

Slide 16 text

benchmark setup benchmark setup 15

Slide 17

Slide 17 text

1. netperf 1. netperf time 16

Slide 18

Slide 18 text

1. netperf (cont'd) 1. netperf (cont'd) 17

Slide 19

Slide 19 text

2. nginx/wrk2, non-TLS/kTLS 2. nginx/wrk2, non-TLS/kTLS --with-openssl-opt=enable-ktls sendfile on; 18

Slide 20

Slide 20 text

2. nginx/wrk2, non-TLS/kTLS 2. nginx/wrk2, non-TLS/kTLS 19

Slide 21

Slide 21 text

2. nginx/wrk2, non-TLS/kTLS 2. nginx/wrk2, non-TLS/kTLS 19

Slide 22

Slide 22 text

2. nginx/wrk2, non-TLS/kTLS 2. nginx/wrk2, non-TLS/kTLS 19

Slide 23

Slide 23 text

2. nginw/wrk2 flamegraph (nginx) 2. nginw/wrk2 flamegraph (nginx) Flame Graph Search ic direct_splice_actor __nanosleep crypto_aes_encrypt [unknown] inet_sendpage [.. crypto_gcm_encrypt clock_nanosleep@@G.. bpf_exec_tx_verdict ghash_update g.. gcm_encrypt_continue minod_client_thread __crypto.. minod_lkl_syscall [unknown] tls_sw_sendpage gcm_hash_init_continue e.. [unkno.. __splice_from_pipe crypto_skcipher_encrypt kernel_sendpage [unknown] tls_sw_do_sendpage shash_ahash_update [unknown] g.. gf128mul_4k_lle r.. splice_direct_to_actor m.. sock_sendpage s.. dpdkio_rx_poll_thread f.. f.. lkl_syscall [unknown] crypto_ctr_crypt minod [unkno.. pipe_to_sendpage gcm_hash_assoc_remain_continue generic_splice_sendpage [un.. crypto_aead_encrypt gcm_hash gcm_hash_assoc_continue [unk.. splice_from_pipe rte_delay_us_sleep d.. minod_process_rsyscall thread_start sys_sendfile64 [unknown] gf128mul_x8_lle [unknown] do_splice_direct shash_async_update do_sendfile d.. crypto_shash_update __crypto_xor start_thread tls_push_record crypto_aead_encrypt 20

Slide 24

Slide 24 text

What we saw ? What we saw ? 21

Slide 25

Slide 25 text

Observations Observations 22

Slide 26

Slide 26 text

To move forward... To move forward... 23

Slide 27

Slide 27 text

Summary Summary 24

Slide 28

Slide 28 text

kernel offload with complete host kernel offload with complete host kernel functionalities kernel functionalities 25