or, how to make your ID
generation less like this:
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
and more like this:
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
Requirements
Slide 7
Slide 7 text
3
main requirements
Slide 8
Slide 8 text
1
an ID for every event
Slide 9
Slide 9 text
2
IDs must be unique
Slide 10
Slide 10 text
3
it must scale well
Slide 11
Slide 11 text
Theory
Slide 12
Slide 12 text
Time
Slide 13
Slide 13 text
Time is hard. Really hard.
(Diamond. Hard. See what I did there?)
Slide 14
Slide 14 text
Time oracles could solve this
problem.
Slide 15
Slide 15 text
(No, not that oracle)
Slide 16
Slide 16 text
Time oracles could solve this
problem.
Slide 17
Slide 17 text
NTP tries to solve this
problem for the “rest of us”.
Slide 18
Slide 18 text
(No, not that NTP?)
Slide 19
Slide 19 text
NTP tries to solve this
problem for the “rest of us”.
Slide 20
Slide 20 text
However, expect ±10ms.
(at least)
Slide 21
Slide 21 text
That’s ±10ms per machine.
Slide 22
Slide 22 text
Hey Bob, what’s
the time?
It’s 1970! Back in
my day...
Fucking Bob
always thinks it’s
1970. It’s clearly
1980. Look at my
hair!
No, Karen. It’s 1990. I’m not
wearing this antiquated
Disney shirt for kicks.
Slide 23
Slide 23 text
Time can move backwards or
forwards.
Slide 24
Slide 24 text
1s ≠ 1000ms
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
Representing Numbers In
Binary
Slide 27
Slide 27 text
Java has many numeric data
types.
Slide 28
Slide 28 text
However it has no unsigned
variants.*
This is no longer strictly true in Java 8, however it’s a trick: they’re still signed types, but now
there’s a bunch of functions to do unsigned operations on them (eg. unsignedDivide).
Slide 29
Slide 29 text
So in Java the MSB is for the
sign:
1000000000000
1 when negative and 0 when positive. So the above is a short and is -4096.
Slide 30
Slide 30 text
Some languages make it
hard to use the signed bit.
Slide 31
Slide 31 text
A Java long is 64-bits.
So we have 63 usable bits.
Slide 32
Slide 32 text
The Epoch
Slide 33
Slide 33 text
An epoch is a marker of time
relative to true time.
Slide 34
Slide 34 text
Unix time is an epoch
measured relative to
00:00:00.000 1/1/1970.
Slide 35
Slide 35 text
We can define our own
epoch.
Slide 36
Slide 36 text
So in 1 year it will be
31,536,000.
Unix time will be
~1,456,348,092,000.
Slide 37
Slide 37 text
Why is this useful? Because
it allows us to compress the
storage of time.
Slide 38
Slide 38 text
Redis
Slide 39
Slide 39 text
Redis is awesome, fast and
stable.
Slide 40
Slide 40 text
It supports scripting via Lua.
Slide 41
Slide 41 text
We can create a Lua script to
make an ID inside Redis.
Slide 42
Slide 42 text
Redis is not distributed.*
Redis clustering will arrive in v3.
Slide 43
Slide 43 text
We can round-robin between
a bunch of Redis servers to
achieve distribution.