Lunatic

In my career as a software developer I had the opportunity to work with many different programming languages (Java, Python, JavaScript, Clojure, Elixir, C and Rust) and runtimes (JVM, Node, CPython, Erlang/OTP, etc). These tools share many similarities, but some of them left a stronger impression on me than others.

Erlang/OTP and the way of structuring execution into actors left a specially strong impression. In a world where many are arguing between a monolithic and microservice based application structure approach, Erlang lets you start building your application on a single server (monolithic), but later allows you to scale up and get distribution for "free". This is possible because the programming model is based on independent actors communicating through messages. There is no difference to these actors if they run on the same machine or are spread across a cluster of machines. In most cases you don't need to adapt your code at all to scale up.

Fault tolerance is another benefit of Erlang. The idea of isolating the state of each actor allows you to create truly fault tolerant environments. If failure is limited to one actor, the whole system becomes more resilient.

Sadly, Erlang also has some drawbacks. I would not consider it a fast language, therefore the only way of achieving high performance is dropping down to C and writing NIFs. In this case you would lose all the benefits mentioned earlier, and a bug in your C code could take down the whole application. If you don't have bugs in your code, but don't follow strict rules of writing NIFs, you could end up blocking the execution and endangering your whole application.

I wanted to bring the power of Erlang to other programming languages and at the same time address these shortcomings. I felt like there was a unique opportunity to do so with the recent development of WebAssembly. That's how Lunatic's idea was created.

Lunatic is a set of libraries and a WebAssembly runtime which allows developers to build resilient actor systems.

The people involved in the creation of WebAssembly made it fast, efficient and secure in the browser. We can benefit from this development on the backend too. That's why Lunatic chose WebAssembly instances as the abstraction for actors. Each instance has its own stack, heap and syscalls. This allows us to have completely isolated execution environments per actor, without reaching out to much heavier technologies like Docker. Another benefit of using WebAssembly is that Lunatic applications can be built in any language supporting WebAssembly as a target (like C/C++, Rust, AssemblyScript, etc).

The other part of Lunatic are the libraries exposing Lunatic's features to programming languages. This includes actor creation, message sending, networking and other functionalities. For now there is only support for Rust, but other languages are coming too. Hold tight!

Lunatic is a good choice for any backend application, such as web applications, real time messaging servers and others. It's especially powerful in the networking context, by allowing you to model each connection with an actor. This way you get sandboxing per connection and, therefore, additional security. Even if you have a vulnerability in your app, it will only be limited to the connection exploiting it.

Moreover, Lunatic tries to stay out of your way. If possible, it will provide bindings to its APIs directly through the standard library of your language of choice. We are still not quite there yet, but as standards around WebAssembly (like WASI) keep maturing we are getting closer.

There are many features still in development which will bring other benefits to Lunatic, for example hot reloading and supervision trees.

Lunatic is written in Rust and uses Mozilla's Wasmtime to create processes as WebAssembly instances. Similar to how it's done in Erlang, Lunatic uses the name Process for actors. Lunatic's processes are lightweight, fast to create and destroy, and the scheduling overhead is low. They are designed for massive concurrency.

All processes are scheduled using an async executor and all code running on top of Lunatic will be automatically transformed to be non-blocking. This is also true for low level languages like C and Rust. From the perspective of the developer, only blocking syscalls are used. However, the runtime will take care of actually scheduling another process' execution when one is waiting on networking traffic or is blocked for another reason. This way we get the best of both worlds, straightforward development (without async keywords) and the highest performance possible.

Lunatic is open source and still early in development. There are many interesting problems that need to be solved before it becomes production ready. If you are a Rust developer and would like to help, please reach out to me on [email protected] or on Twitter or join our discord server.