Slide 1
Slide 1 text
Adventures in the Dungeons of
OpenSSL
Ryo Kajiwara/
梶原 龍
(sylph01)
2023/12/15 @ RubyConf Taiwan 2023
1
Slide 2
Slide 2 text
Slides are
available at:
https:/
/speakerdeck.co
m/sylph01/adventures
-in-the-dungeons-of-
openssl
2
Slide 3
Slide 3 text
Hi!
3
Slide 4
Slide 4 text
I do stuff
Play rhythm games (especially DanceDanceRevolution)
Play the bassoon/contrabassoon
Ride a lot of trains (Rails!) (travelled on 99% of JR)
Build keyboards
if anything catches your interest let's talk!
4
Slide 5
Slide 5 text
And I do stuff
that is more relevant to this talk:
Freelance web developer focused on Digital Identity and Security
Mainly working at codeTakt, developing ID platforms for schools
Worked/ing on writing/editing and implementing standards
HTTPS in Local Network CG / Web of Things WG @ W3C, OAuth /
Messaging Layer Security WG @ IETF
Worked as an Officer of Internet Society Japan Chapter (2020-23)
5
Slide 6
Slide 6 text
I'm from the
Forgotten
Realm of
Japan called
Shikoku
image https:/
/twitter.com/Mitchan_599/status/994221711942971392 6
Slide 7
Slide 7 text
My ticket this
time was from
松山
(Matsuyama) to
松山
(Songshan)
7
Slide 8
Slide 8 text
8
Slide 9
Slide 9 text
Caution before I start...
9
Slide 10
Slide 10 text
I'm going to talk about
Dungeons
10
Slide 11
Slide 11 text
... so there be Dragons
Dungeons & Dragons, am I right?
11
Slide 12
Slide 12 text
Caution
Cryptographic API can be very easy to misuse
What's different from dungeons like parse.y -vania is that you
can actually hurt yourself. Do you like security breaches?
I've done my research, but I don't consider myself a cryptography expert
I don't have a PhD/Master in this field, so yeah...
If you're not sure, please have your system audited by a security
expert before going to production
12
Slide 13
Slide 13 text
Caution
The level of production-readiness in this talk:
The hpke gem: Almost ready
Minor changes in the gem API are possible
Mostly uses pretty safe stuff
But not audited by an external security expert
OpenSSL extension version of HPKE: Definitely not
13
Slide 14
Slide 14 text
Past Related Work
Do Pure Ruby Dream of Encrypted Binary Protocol? / Yusuke Nakamura
@ RubyKaigi 2021
Implementing QUIC in Ruby
Talks about the pain of handling hex-encoded and raw strings in
Ruby
14
Slide 15
Slide 15 text
15
Slide 16
Slide 16 text
HPKE
Hybrid Public Key Encryption, RFC 9180
16
Slide 17
Slide 17 text
Hello
Alice!
Alice's
private key
Encrypt
6EB69570
08E03CE4
Hello
Alice!
Decrypt
Alice's
public key
Alice
Bob
Recap: Public
Key Encrypytion
Alice generates key pair
Bob encrypts with Public
Key
Alice can decrypt with
Private Key
https:/
/commons.wikimedia.org/wiki/File:Public_key_encryption.svg 17
Slide 18
Slide 18 text
What is HPKE
In the past:
Encrypt a session key using Public Key Cryptography
like RSA
Then send your messages using symmetric ciphers
like AES
You don't do everything with PKC because it's costly
18
Slide 19
Slide 19 text
What is HPKE
The problem: There is lots of misuse and implementation bugs in
cryptography
PKCS#1 padding in RSA leading to Bleichenbacher's Oracle Attack
Nonce reuse
DSA/ECDSA: leading to leakage of private key in PlayStation
3's code signing
AES's initialization vector re-use is everywhere!
19
Slide 20
Slide 20 text
What is HPKE
You (typically) don't encrypt/decrypt with Elliptic Curve Cryptography
You get key exchange (ECDH) and digital signatures (ECDSA)
so the "encrypt with Public Key" trick doesn't work
20
Slide 21
Slide 21 text
What is HPKE
HPKE solves this by:
Using standardized/reviewed protocols to exchange keys and
encrypt/decrypt
Using best known cipher suites possible
combination of cryptographic algorithms
Using high-level APIs that prevent misuse
21
Slide 22
Slide 22 text
What is HPKE
HPKE uses
Agreement of symmetric keys using a Key Encapsulation
Mechanism (KEM)
that internally uses a Key Derivation Function (KDF)
Then use the symmetric keys to perform Authenticated
Encryption with Associated Data (AEAD)
22
Slide 23
Slide 23 text
How to use HPKE
HPKE is a one-way protocol from Sender to Receiver
Agreement of symmetric key using secret exporter is possible
Receiver creates a public/private key pair, publishes public key
Sender uses public key to encapsulate session key, then uses the
session key to encrypt message
Sender sends the encapsulation and the message
Receiver uses the encapsulation to decapsulate session key, uses the
session key to decrypt the message
23
Slide 24
Slide 24 text
So I wanted HPKE in Ruby...
24
Slide 25
Slide 25 text
What is needed for HPKE?
Key Encapsulation Mechanism
Diffie-Hellman Key Exchange
use of Elliptic Curves: P-256, P-384, P-521, X25519, X448
Key Derivation Function
HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
HMAC-SHA256, HMAC-SHA512
Authenticated Encryption with Associated Data
AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305
25
Slide 26
Slide 26 text
All of this is supported by OpenSSL,
right?
26
Slide 27
Slide 27 text
Well, kinda
27
Slide 28
Slide 28 text
What we had readily available
AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305
test/openssl/test_cipher.rb
ChaCha20-Poly1305 isn't in this test code, but uses the same API
as AES-GCM
HMAC-SHA256, HMAC-SHA512
Even with SHA256 and SHA512 only, I can implement HMAC
Actually we have HKDF itself but we need to use some parts of
HKDF and customize it
28
Slide 29
Slide 29 text
What we 'kinda' had
Elliptic curves P-256, P-384, P-521
test/openssl/test_pkey_ec.rb
X25519, X448
test/openssl/test_pkey.rb
29
Slide 30
Slide 30 text
Q: Really? These seem to be
undocumented
A: Yes.
30
Slide 31
Slide 31 text
Really? These seem to be
undocumented
Some APIs in OpenSSL are intentionally left undocumented to avoid misuses
by people who are not well-versed in cryptography.
Yes, misuse in cryptography can hurt yourself.
31
Slide 32
Slide 32 text
OpenSSL::PKey
Set of classes that handle everything Public Key Cryptography, including
RSA, DSA, Diffie-Hellman, and Elliptic Curve Cryptography.
Wraps OpenSSL's EVP_PKEY struct.
Note that X25519/X448 are actually ECC but does not use
OpenSSL::PKey::EC .
32
Slide 33
Slide 33 text
Raw public/private key support for
X25519/X448
Raw public/private key support for EC "kinda" existed
33
Slide 34
Slide 34 text
Raw public/private key support for
X25519/X448
There was a pull request that worked on this
But it was 3 years old and not working on some platforms
So I started working on this by
Fixing errors on the CI matrix
Addressing unfixed review comments
34
Slide 35
Slide 35 text
https:/
/github.com/ruby/openssl/pull/
646
my first Ruby C extension experience!
35
Slide 36
Slide 36 text
While I was at it...
36
Slide 37
Slide 37 text
OpenSSL::PKey is immutable in
OpenSSL 3.0
https:/
/github.com/ruby/openssl/issues/619 37
Slide 38
Slide 38 text
So you can't do this
pkey = OpenSSL::PKey::EC.new('prime256v1')
pkey.private_key = SecureRandom.random_bytes(32)
Note: A private key of P-256 elliptic curve consists of a 32 byte number
(=scalar).
38
Slide 39
Slide 39 text
In OpenSSL 3.0, we were not just
missing APIs to make X25519/X448
key pairs,
we didn't even have APIs to make EC
key pairs!
39
Slide 40
Slide 40 text
How can we create a key pair with a
specified private key?
40
Slide 41
Slide 41 text
This was available
der = # write something in ASN.1 DER format
OpenSSL::PKey.read(der)
41
Slide 42
Slide 42 text
ASN.1 Sequence -> DER -> PKey::EC
it's even in Ruby 3.3 preview1!
42
Slide 43
Slide 43 text
Enter ASN.1
Abstract Syntax Notation One
Set of binary notation rules for encoding data structures
DER (Distinguished Encoding Rules) is the ruleset to describe data
structures in exactly one way, so as to ensure digital signatures produce
a unique output
PEM files encode certificates private keys written in DER with Base64
43
Slide 44
Slide 44 text
PKey -> DER -> ASN.1 Sequence
pk = OpenSSL::PKey::EC.generate('prime256v1')
pk.private_to_der.unpack1('H*')
=>
"308187020100301306072a8648ce3d020106082a8648
ce3d030107046d306b02010104203baadea1f85b96b3
3b8a895ab2c44a4b72e827ff1b1ac23e7b756daef2f0
892ba14403420004b9f0779b47f432fd7df13b67cf21
0c58db276653a13b862db2eb6d47f3eabe9ac4317181
8e19e78f37fe5e86a7439d471d0f3442a2b727f9560a
ff417d432391"
(formatting just for visibility purposes)
44
Slide 45
Slide 45 text
PKey -> DER -> ASN.1 Sequence
https:/
/lapo.it/asn1js/ 45
Slide 46
Slide 46 text
ASN.1 Sequence -> DER -> PKey::EC
Do it backwards!
def create_key_pair_from_secret(secret)
asn1_seq = OpenSSL::ASN1.Sequence([
OpenSSL::ASN1.Integer(1),
OpenSSL::ASN1.OctetString(secret),
OpenSSL::ASN1.ObjectId(curve_name, 0, :EXPLICIT)
])
OpenSSL::PKey.read(asn1_seq.to_der)
end
46
Slide 47
Slide 47 text
I did this in the HPKE gem for
X25519/X448 too
def create_key_pair_from_secret(secret)
asn1_seq = OpenSSL::ASN1.Sequence([
OpenSSL::ASN1.Integer(0),
OpenSSL::ASN1.Sequence([
OpenSSL::ASN1.ObjectId(asn1_oid)
]),
OpenSSL::ASN1.OctetString("\x04\x20" + secret)
])
OpenSSL::PKey.read(asn1_seq.to_der)
end
47
Slide 48
Slide 48 text
But does this look good?
No, it's ugly!
48
Slide 49
Slide 49 text
API to generate PKeys with a private
key
is being worked on @ Issue #555
49
Slide 50
Slide 50 text
Now we have HPKE in Ruby
GH: sylph01/hpke-rb
gem install hpke now!
note: Still in beta. I will likely tinker around with the API of the gem 50
Slide 51
Slide 51 text
51
Slide 52
Slide 52 text
Well actually, we had HPKE itself in
OpenSSL 3.2
blog post in OpenSSL Blog: https:/
/www.openssl.org/blog/blog/2023/10/18/ossl-hpke/ 52
Slide 53
Slide 53 text
Why not write a Ruby wrapper for this?
53
Slide 54
Slide 54 text
so I did.
GH: sylph01/openssl, hpke branch
(note: this is super experimental territory, DO NOT use
in production!)
54
Slide 55
Slide 55 text
How to develop C extensions
GH: ruby/ruby/doc/extension.rdoc is your friend.
You want to know how to:
Convert C values into Ruby values and vice versa
Wrap a C struct into a Ruby object
55
Slide 56
Slide 56 text
Handling HPKE Context
OSSL_HPKE_CTX *OSSL_HPKE_CTX_new(int mode, OSSL_HPKE_SUITE suite, int role,
OSSL_LIB_CTX *libctx, const char *propq);
int OSSL_HPKE_encap(OSSL_HPKE_CTX *ctx,
unsigned char *enc, size_t *enclen,
const unsigned char *pub, size_t publen,
const unsigned char *info, size_t infolen);
int OSSL_HPKE_seal(OSSL_HPKE_CTX *ctx,
unsigned char *ct, size_t *ctlen,
const unsigned char *aad, size_t aadlen,
const unsigned char *pt, size_t ptlen);
https:/
/www.openssl.org/docs/manmaster/man3/OSSL_HPKE_CTX_new.html 56
Slide 57
Slide 57 text
Handling HPKE Context
static void ossl_hpke_ctx_free(void *ptr) {
OSSL_HPKE_CTX_free(ptr);
}
const rb_data_type_t ossl_hpke_ctx_type = {
"OpenSSL/HPKE_CTX",
{
0, ossl_hpke_ctx_free,
},
0, 0, RUBY_TYPED_FREE_IMMEDIATELY
};
57
Slide 58
Slide 58 text
Handling HPKE Context
static VALUE hpke_ctx_new0(VALUE arg){
OSSL_HPKE_CTX *ctx = (OSSL_HPKE_CTX *)arg;
VALUE obj;
obj = rb_obj_alloc(cContext);
RTYPEDDATA_DATA(obj) = ctx;
return obj;
}
VALUE ossl_hpke_ctx_new(OSSL_HPKE_CTX *ctx){
VALUE obj;
int status;
obj = rb_protect(hpke_ctx_new0, (VALUE)ctx, &status);
if (status) {
OSSL_HPKE_CTX_free(ctx);
rb_jump_tag(status);
}
return obj;
} 58
Slide 59
Slide 59 text
Handling HPKE Context
static VALUE ossl_hpke_ctx_alloc(VALUE klass) {
return TypedData_Wrap_Struct(klass, &ossl_hpke_ctx_type, NULL);
}
void Init_ossl_hpke_ctx(void) {
mHPKE = rb_define_module_under(mOSSL, "HPKE");
cContext = rb_define_class_under(mHPKE, "Context", rb_cObject);
...
rb_define_alloc_func(cContext, ossl_hpke_ctx_alloc);
}
59
Slide 60
Slide 60 text
Generating the key
int OSSL_HPKE_keygen(OSSL_HPKE_SUITE suite,
unsigned char *pub, size_t *publen, EVP_PKEY **priv,
const unsigned char *ikm, size_t ikmlen,
OSSL_LIB_CTX *libctx, const char *propq);
priv is an EVP_PKEY so we can wrap that into OpenSSL::PKey
pub is an unsigned char * = String, but which format?
60
Slide 61
Slide 61 text
Generating the key
Apparently:
for EC keys it's the equivalent of
priv.public_key.to_octet_string(:uncompressed)
for X25519/X448 keys it's the equivalent of priv.raw_public_key
61
Slide 62
Slide 62 text
Encapsulating the key
VALUE ossl_hpke_encap(VALUE self, VALUE pub, VALUE info) {
(snip definitions)
GetHpkeCtx(self, sctx); // extract C pointer to context from object
enclen = sizeof(enc);
if (OSSL_HPKE_encap(
sctx, enc, &enclen,
(unsigned char*)RSTRING_PTR(pub), RSTRING_LEN(pub),
(unsigned char*)RSTRING_PTR(info), RSTRING_LEN(info)) != 1) {
ossl_raise(eHPKEError, "could not encap");
}
enc_obj = rb_str_new((char *)enc, enclen);
return enc_obj;
}
62
Slide 63
Slide 63 text
How to develop (OpenSSL)
ruby/openssl's CONTRIBUTING.md has a "Testing: With different versions of
OpenSSL" section:
git checkout openssl-3.2
OPENSSL_DIR=$HOME/.openssl/openssl-something
./Configure --prefix=$OPENSSL_DIR --libdir=lib enable-
fips enable-trace '-Wl,-rpath,$(LIBRPATH)' -O0 -g3 -ggdb3
-gdwarf-5
make -j4 , make install
63
Slide 64
Slide 64 text
How to develop (OpenSSL)
I wanted to peek inside the OSSL_HPKE_CTX , but I got an invalid use
of incomplete typedef error. This is because the full typedef is not
exposed to the public header files.
I moved the internal definitions to public header file
then recompiled OpenSSL.
64
Slide 65
Slide 65 text
How to develop (OpenSSL gem)
Then to build with that version:
bundle exec rake clean
bundle exec rake compile -- --with-openssl-
dir=$OPENSSL_DIR
irb -I lib -r openssl
65
Slide 66
Slide 66 text
How to develop (debugging)
I wanted to see the C string in hex printed into the console, so:
void rbdebug_print_hex(const unsigned char *str, size_t len) {
VALUE rbstr;
rbstr = rb_str_new((char *)str, len);
rb_p(rb_funcall(rbstr, rb_intern("unpack1"), 1, rb_str_new_cstr("H*")));
}
rb_p is your friend. Also rb_funcall is nice too.
66
Slide 67
Slide 67 text
Example
irb(main):001:0> suite = OpenSSL::HPKE::Suite.new_with_names(
:dhkem_p256_hkdf_sha256, :hkdf_sha256, :aes_128_gcm)
=> #
irb(main):002:0> priv = OpenSSL::HPKE.keygen_with_suite(suite)
irb(main):003:0> pub = priv.public_key
=> #>
Generate a public/private key pair
67
Slide 68
Slide 68 text
Example
# sender
irb(main):004:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite)
=> #
irb(main):005:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info")
irb(main):006:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08",
"a message not in a bottle")
=> "\x93\xC1\xDF7\x87\xA8\xBER\xC9&\xA4\xD1\xB7\x10\x8E..."
Create sender context
Encapsulate key into the public key of the receiver
Using the generated shared secret, seal message
before commit 4548dd0e it uses a different style of initialization API 68
Slide 69
Slide 69 text
Example
# receiver
irb(main):007:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite)
=> #
irb(main):008:0> rctx.decap(enc, priv, "Some info")
irb(main):009:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct)
=> "a message not in a bottle"
Receiver gets the encapsulation enc and sealed message ct
Create receiver context
Decapsulate key from enc with private key and derive shared secret
Then using the derived shared secret, open message
69
Slide 70
Slide 70 text
By the way...
Is anyone familiar with Ruby's encoding? I don't want the force_encoding
but I kinda need it now
irb(main):010:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite)
irb(main):011:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite)
=> #
irb(main):012:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info")
=> "\x04\x17M\b\xE0\x9D\xEEg\xE2..."
irb(main):013:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08", "
ニッホンゴー")
=> "E\x80\xEFP\x95Z9N\x1F\t..."
irb(main):014:0>
irb(main):015:0> rctx.decap(enc, priv, "Some info")
=> true
irb(main):016:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct)
irb(main):017:0> puts pt.force_encoding('UTF-8')
ニッホンゴー
=> nil
70
Slide 71
Slide 71 text
Will this go into actual OpenSSL gem?
This still needs a lot of work:
This is limited to OpenSSL 3.2, so needs guards against older versions
Currently supports Base mode only
Hardcoded length values need to be fixed
Is the C coding actually safe?
71
Slide 72
Slide 72 text
72
Slide 73
Slide 73 text
Conclusion
73
Slide 74
Slide 74 text
So here were my "Adventures in the
Forgotten Realm" called OpenSSL
74
Slide 75
Slide 75 text
Why HPKE in Ruby?
It's the building block for modern security/privacy protocols
TLS Encrypted ClientHello
Oblivious HTTP
Messaging Layer Security
75
Slide 76
Slide 76 text
Why HPKE in Ruby?
If we don't have the building block for modern networking protocols, people
would not implement them
... and just go to Python / Go / Rust / whatever because they have the
building blocks readily available.
76
Slide 77
Slide 77 text
We need modern cryptography in Ruby
for Ruby to stay relevant
77
Slide 78
Slide 78 text
It's not for everyone
... but it surely is for someone
78
Slide 79
Slide 79 text
The adventure continues...
HPKE is just a building block for other protocols
I am working on implementing protocols that rely on HPKE
Also now that I came back from the OpenSSL dungeon alive, I might
continue digging into this "Forgotten Realm"
79
Slide 80
Slide 80 text
Shoutouts
(@ mentions are in GitHub ID)
Ruby OpenSSL maintainers, esp. @rhenium
Past RubyKaigi speakers, esp. @unasuke and @shioimm
HPKE implementers, esp. @dajiaji
And to the organizers of RubyConf Taiwan 2023!
80