Event-loop server design

[2021-04-06]

I briefly go over the event-loop design for servers that need to handle a high number of simultaneous connections

In the last decades best-practices of server design have been updated to support many concurrent clients. Rise of the Internet, service-oriented architectures and demand for online services all contributed to a push for designs that can handle more clients with less resources. There are many strategies how to design IO intensive servers, most of them are described in the popular C10K article.

In the web-server space the design moved from thread-based request handling (vulnerable to the SlowLoris attack) to event-loop driven architectures represented by the NGINX and Node.js projects.

The event in the event-loop usually representens any activity on a socket (e.g. socket is ready to be read/written, timeouts, errors ..). There is a single thread (the “loop”) that is notified about such activity and creates the event that is then stored on some queue. Events from the queue are processed by a dedicated thread pool. The point is not to block the event-loop.

The implementaion details of the event-loop may vary from system to system. For example:

  • is it faster if the event is handled directly by the event-loop thread (i.e. does the event need to be queued)?
  • Are there more event queues?

These considerations are specific to the system you are building.

The socket activity notifications in Linux (inside the event-loop) can be done by a system call. Performance-wise the epoll API is currently considered as the most scalable. Some systems do not support epoll so it’s better to use a library that picks the fastest kernel API that is available on the given OS. Some of those libraries are libuv (created by and used in Node.js), libev and libevent. There is a benchmarking test of libevent vs. libev if you are interested in performance comparisons.

To summarize the key points of the event-loop design are:

  • one thread (event-loop) can handle all socket activity,
  • the kernel API can provide the notifications about socket activity,
  • use a library (libuv, libev, libevent…) for software portability,
  • processing of events (i.e. when long blocking synchronous calls are triggered) must happen outside the event-loop.