that you’ll need to generate thumbnails. • If you have a Phoenix application, you’d probably like to generate thumbnails within your application so everything is in the same place. • You’d probably want a solution that just works but is also quite performant.
for completion. 2. Daemon: Swap messages with a long-running daemon. 3. NIF: Run C code in BEAM directly. 4. Pattern Match: implement scaling code in Erlang/Elixir directly. 5. Persistent C Server: swap messages with a supervised process.
one thumbnail at a time. • Start a child process with appropriate arguments. • Wait for the child process to finish. • Look at what the child process has sent.
resource cap. • Safe: Crashes isolated to external OS processes; resource cleanup done by OS. • Slow: code/data needs to be reloaded on each run. • Expensive: bigger servers, smaller conference budget.
and returns results at the same time is very easy to reason with. This can be attractive when you do not have a concurrency requirement. • Thorough cleanup is almost guaranteed upon process exit.
process needs to first load data into memory, or has a heavy initialisation process. • You may create a fork bomb if multiple forks can happen concurrently and there is no safeguard. • You will most likely need a timeout.
code, or find a project that has one and use that. • The daemon will field your requests either over a port directly or via forked child processes that pass messages. • Some daemons may even have concurrency support.
reload data on each call. • Less Memory Pressure: Possible to share some memory among all processes. • Faults Isolated: Crashes isolated to an external OS process and its children. Possible to have an OS-level process manager restart the daemon(s). • Multiple Failure Modes: Errors can propagate and cause grief because the daemon is probably not written in Erlang.
project, has two variants. It can run a daemon which then accepts work, or it can be run standalone. The daemon is about 10 to 100 times faster to scan a file in practice because it does not have to repeatedly load virus definitions. • This is an example of a proper daemon not written in Erlang (and you can still supervise a daemon using Erlang).
half of Erlang in another language. It takes longer to do that than learning Erlang. • If you do not have the daemon supervised by your application, you will not have a common root for all activities and that leads to madness. • You need to find a way to send a message to a daemon. You may need to make a binary/text interface or you may need to take the hit of forking something, which does that. Either way it is a lot of work.
your NIF brings down the BEAM. • Hard to De-Risk: image formats can be complex; it would be difficult to proclaim any code manipulating them bug-free. Images can come from the Internet (i.e. user-provided input). • Elbow Grease Required: Special care is required to mark a NIF dirty. Failure to cover all bases may cause issues.
using pattern-matching. • Requires intimate understanding of all image format specifications and of the BEAM as you will be moving a lot of binary data around. • A good weekend project for the tenacious.
C Server reading from STDIN and writing to STDOUT/STDERR. • Supervise the C Server with appropriate Erlang code which restarts the process as needed. Crash the C Server whenever. • Put as many of these pairs in a connection pool as needed.
Level C binding. This is a proven solution and is faster than ImageMagick. Its functions can be picked-and-choosed in our custom C Server. • Protocol: Text-Based. This means the C Server can be tested in isolation without an elaborate test harness, will be able to work over STDIN/STDOUT, and will not require code to handle a binary protocol.