Spacer - A tool to inflate unikernels

Unikernels are on the rise in the cloud. These lightweight virtual machines (VMs) specialized to a single application offer the same level of isolation as full-blown VMs, while providing performance superior to standard Linux-based VMs or even to containers. However, their inherent specialization renders memory deduplication ineffective, causing unikernels, in practice, to consume more memory than their small memory footprint would suggest. This makes them less advantageous when thousands of SaaS and/or FaaS unikernels instances have to run on the same server.

We introduce a novel approach to build the next generation of networked services and lambda functions by improving unikernel's memory layout so that it is more likely to share identical pages with other unikernels deployed on the system. Our approach supports SaaS and FaaS architectures and can be used with ASLR. Our experiments show that our approach can reduce the amount of physical memory used by a set of unikernels running on the same server by as much as 3x, with next to no overhead on applications performance.

Spacer and its toolset is released under an open-source license and are available on Github.

@gaulthiergain/spacer

Unikernels are based on the concept of a library OS, where the functionality of the OS is split into independent libraries. However, not all unikernels use the same libraries, and even for those that use the exact same ones their memory layout may differ. Indeed, the insertion of a new library on an instance shifts following libraries in memory and then makes the cross-reference addresses different in instructions such as call or lea. Both mechanisms concur to prevent memory deduplication.

In order to circumvent these issues, we introduce a new methodology based on page alignment which consists of:

1. Aligning sections and libraries at the same absolute addresses in the virtual address space of all instances.
2. Keeping a global map of all libraries used and mapping them to a specific address between instances

When there are several unikernels instances, it is necessary to align them to specific addresses, which will create some gaps in the virtual memory space of some instances. These gaps will be filled with zeros when the unikernel is loaded, so that these zero-filled pages can be shared.

The figure above shows a scenario where the library ukring is aligned by leaving gaps in the second instance in order to have the same addresses in the binary instructions on all other instances.

Spacer's functionality is divided into three parts.

1. The binary analyzer framework disassembles unikernel images and object files in order to retrieve information about symbols, sections, etc. With this information, it is able to detect the location of each library in the binary file. The tool aims to have a global knowledge of all the libraries used by all unikernels on the same machine (workspace). In addition, it is also able to process several unikernel binaries in order to provide theoretical statistics on the sharing by comparing binary content.
2. Spacer generates a global map of the different libraries. In a first stage, libraries common to all instances are placed sequentially without being aligned on a page boundary. Then, all other libraries, depending on their frequency (i.e. the number of occurrences for all unikernels), are associated with an absolute address (aligned on page boundaries). With this in place, Spacer generates a custom linker script that places libraries according to the previous map. To do this, our tool relies on the current output location counter from the linker script specification and associates an absolute address per library and per section.
3. This linker script is used to build aligned unikernels through a final re-linking procedure and existing unikernel binaries (default ones) are overwritten.

Spacer also supports ASLR (step 4). Explanations about its support is described in our SoCC'22 paper.

We have based our implementation on Unikraft since it supports a wide range of libraries and is under active development. However, our approach should work for other library OSes and unikernels.

• Email author
• SoCC'22 (paper)
• Spacer
• Unikraft
• Gaulthier Gain (main website)