Some years ago, the lack of any sort of kernel-supported container object
was arguably the right approach; there was still a lot to be learned about
how this concept should be integrated into Linux. Some would surely argue
that it is still a good idea. But there are downsides as well, starting
with the complexity of creating a container. Beyond that, though, since
the kernel has no notion of containers, it cannot enforce a container's
properties and cannot even answer questions about which container a process
might belong to. Containers are created by walling off parts of the larger
system. Namespaces are created then entered, filesystems are unmounted or
overmounted to make them inaccessible inside the container, and so on.
This sequence means that any given process might be only partly inside a
container, a situation that, according to Bacik, "is where a depressing
number of our bugs come from
". The current approach to container
creation also makes it easy to let unwanted resources leak into the
container from the outside.
Introducing cells
Bacik's proposal is an attempt to address these problems. A container (called a "cell" in this series), when created, contains nothing at all. No processes or filesystems show up there by default. Anything that the creator wants to exist in a cell must be explicitly put there. Processes and other resources created within a cell are not allowed to to leave that cell. In theory, cells should be easier to manage and easier to keep secure than containers as they exist in Linux now.
A cell's life starts with a call to cell_create():
int cell_create(int parent, unsigned int flags);
The parent parameter is seemingly meant to designate which process will be the parent of the new cell, but in the current implementation the only allowed value is CELL_SELF, meaning that the creator is the parent. The flags argument must be zero. The return value, if all goes well, is a "cellfd" — a file descriptor referring to this cell. The cellfd is special, in that it is the handle that is used to perform all other operations on the cell. There is no way to create a duplicate cellfd that possesses the same capabilities. Creating a cell is an unprivileged operation.
If the cellfd is closed, the cell (and anything contained within it) is immediately killed. There is also a system call to kill a cell explicitly:
int cell_kill(int cellfd, unsigned int flags);
Where cellfd identifies the cell to kill, and flags must be zero.
An empty cell is secure and bug-free, but it is also rather boring. There is another system call that is used to set various properties of this cell to make it more interesting:
int cell_configure(int cellfd, unsigned int cmd, const char *key,
const void *value, int aux);
This system call applies the given configuration cmd to the cell identified by cellfd; the key string identifies which property is to be changed. The available commands are:
- CELL_SET_FLAG
- Sets the flag named by key; there are no flags defined in the current proposal, though.
- CELL_SET_STRING
- Sets the string value given by key to the given value. Two strings are supported in the initial patch set: hostname to set the host name within the cell, and net, which sets the network namespace for the cell. By default, the cell has no network access, and only AF_UNIX sockets can be created within it. Setting the net key to new creates a new network namespace that the builder will subsequently configure, while own creates a network namespace owned by the cell itself and configured internally. In the new case, there is another new system call (cell_open()) that must be used to obtain a file descriptor for the newly created namespace. The network namespace can also be explicitly set with CELL_SET_FD, described below.
- CELL_SET_BINARY
- Sets the property identified by key to the provided binary data. The only value accepted is ids, which is used to define which user IDs will be used within the cell. There are only two allowed values. Setting it to one causes processes within the cell to run with the same user ID as the creator; this mode is not allowed for the root user.
Otherwise, the ids key must be set to 65,536, which causes a block of that many IDs to be allocated from the upper part of the 32-bit user-ID space. These IDs will be unique on the running system, and the cell creator has no control over what they will be. It is not possible to create files owned by those IDs in any filesystem outside of the cell. This part of the design has drawn some comments and may change (perhaps by providing an entirely separate user-ID space for cell inmates) in the future.
- CELL_SET_FD
- Sets a value requiring a file descriptor. There are three of those implemented in this patch set. root is a filesystem namespace that contains the root filesystem within the cell, net is a network namespace to be used by the cell, and budget is a control-group directory. The budget is mandatory, and each cell must have its own control group.
- CELL_CMD_SEAL
- Once the configuration of the cell is complete, this command must be issued; it completes the initialization of the cell and prevents any further configuration changes.
Once the cell is sealed, it is set up with a filesystem tree, control-group membership, and the rest of the namespaces that control its interactions with the rest of the system. It is still otherwise empty, though, with no processes running within it. In Bacik's design, there is no way for an existing process to enter a cell; setns() cannot be used to enter its namespaces. Instead, processes must be born within the cell from the beginning, and they can never exit the cell.
Creating inmate processes
Allowing the creation of processes with life sentences inside a cell involves yet another proposal for a process-builder API; the last such attempt was covered here in August. Here, the first step is to create the process within the cell with a variant of the pidfd_open() system call. Rather than passing a process ID to pidfd_open(), the cell creator passes the cellfd and the PIDFD_EMPTY flag. The return value is a pidfd that does not, yet, refer to a running task; it is in the "embryo" state at this point.
Once the pidfd is created, it must be configured with another new system call before it can run anything useful:
int pidfd_configure(int pidfd, unsigned int cmd, const char *key,
const void *value, int aux);
The interface looks much like that of cell_configure(), and it operates in mostly the same way. The process to configure (which must be in the embryo state) is specified by pidfd; the cmd value is one of:
- PIDFD_SET_STRING
- The only supported key is exec, which names the executable file (within the cell's mount namespace) that the process will run.
- PIDFD_SET_BINARY
- Sets a key to binary data. Recognized keys are argv to set the processes argument vector, envp to set its environment variables, and cred, which sets the process's user and group IDs within the cell's user namespace.
- PIDFD_SET_FD
- Sets a file descriptor in the embryo process; the key is the file-descriptor number (as a string), and the aux parameter is an open file descriptor in the caller's file table. The only open file descriptors the process will have are those set up with PIDFD_SET_FD, so there should be no problems with file descriptors accidentally leaking into the cell.
- PIDFD_CMD_START
- This command sets the process running once all of the necessary parameters have been configured. After that, it is no longer an embryo, and pidfd_configure() can no longer be used with it.
The first process to be started within the cell becomes the init process for the cell; if it dies, the cell dies with it. It is possible to inject other processes into a running cell using this API, so the owner can, for example, start a shell within the cell if need be. But that must be done by creating a new process; existing processes are forbidden from crossing that boundary.
Other details
A lot of other restrictions at the cell boundary exist as well. Every process within the cell runs with the "no new privileges" flag set, so it cannot gain new capabilities. It is not possible to send open file descriptors across the boundary (in either direction) using Unix-domain sockets. Cells cannot share a control group with any other processes. And so on; the purpose is to make the containment of cells as tight and controlled as it can be.
There are a lot of details to this proposal that have not been covered
here; it also contains a long list of open issues and shortcomings. As
Bacik noted, the purpose is not to provide a complete implementation;
instead, it "exists so that there is code to argue about
". Readers
wanting to know more should read through his entire proposal. The code
exists in a
GitHub branch; it contains a
design document that covers much of what was discussed here and more as well.
So far, there has been little discussion of this proposal, perhaps because
it was only posted to the linux-fsdevel mailing list. Nonetheless, it has
evolved somewhat since the first
version was posted in mid-October. This is not a small proposal, and
it is trying to make changes that have proved controversial in the past, so
it is not hard to imagine that discussion will eventually happen. At the
far end of it, though, the kernel might just have the container abstraction
that its community has resisted for so long.
| Index entries for this article | |
|---|---|
| Kernel | Containers |
Did you like this article?? Claim your free trial subscription now to get a lot more like it.