The Unix Column: 'Sandboxes Using Chroot'
You've just obtained a new application that will run networked over an Internet. How do you know its secure? How do you know that its code doesn't contain any oversights that may lead to a system compromise? You probably don't, especially if its a large application.
Unintentional holes may be introduced to applications with as little as a one line coding oversight, such as copying data between two memory locations without checking the bounds of the data first (such a crack can be leveraged through a so-called 'stack overflow' exploit). An insecure but trusted application may by-pass firewalls, opening a potential entry point. For example, an institution may have a strict firewall policy, but allow incoming HTTP connections. The strength of a chain is as strong as its weakest link, and in this case, access to the underlying system could be obtained through the web server (for example, through an insecure CGI script).
This article discusses running applications in what can be thought of as a virtual environment. The underlying idea is that a process is launched in a restricted environment that it believes to be a 'full' machine. However, its view of the world is one that has been created to simulate the machine. When an attacker gains access to the machine through an exploitable hole in some application, he can only damage the restricted environment.
The key to the whole idea is the chroot system call. When a program calls chroot, it irreversibly changes its view of the filesystem. The processes idea of the extent of the filesystem can be reduced to a much smaller directory hierarchy by reducing the position of root (/) lower down in the 'real' filesystem.
Some applications can operate in a specific chrooted mode, such as Acme Labs thttpd [1]. Other applications cannot, and have to be 'fooled' into believing that the environment in which they live is the whole machine.
Note that many applications utilise a chrooted environment, or may use chroot in some way, yet not offer the kind of security described in this article. For example, wu-ftpd [2] can run in a chrooted environment for anonymous access, but the server at login runs non-chrooted (so that users can login and get to their home directories).
Chrooted environments are not a new concept and are used increasingly in connection with security policy. Famously, chroot was used to build a 'jail' to contain and observe a malicious hacker, as described in An Evening With Berford [3].
A General Recipe
The section gives an idea on how to set-up a fairly general secure environment. It can be tailored to individual applications. Specific details will depend on your flavour of Unix.
The secure environment operates as a kind of miniature Unix system. Its construction consists of the following steps,
- creating a directory structure;
- creating appropriate symbolic links;
- building devices;
- including required system files.
These tasks would usually be automated via makefile or shell scripts. When the environment has been built and installed, applications can be launched from within it.
Note that a number of files need be modified for the virtual environment such as user database files (passwd, shadow, master.passwd etc, which would contain minimal users) and device files.
1. Creation of directory structure
One approach to maintaining a miniature Unix filesystem, is to gather together a minimal filesystem which can later be copied into an appropriate parent directory (which will become the new root directory). The result should be a stripped down version of your Unix installation. For FreeBSD [4], the top level directory would contain the subdirectories, bin, dev, etc, sbin, src, tmp, usr, var.
2. Creation of symbolic links
When the basic filesystem has been created, a number of conventional symlinks need to be made. Some of these will depend on your particular system. Common examples include lib -> usr/lib and bin -> usr/bin.
3. Building devices
In general, it may be more secure to decide which devices to include based on the needs of your application. The idea is to have the minimal devices necessary for the application(s) that will be run from the environment.
FreeBSD comes supplied with a MAKEDEV script that builds the /dev directory. This can be altered to build the virtual /dev directory. The contents of the devices directory in the test environment that I built, include the console, pseudo terminal devices, and memory devices.
4. Copying files
Key system files are likely to live under etc, usr/lib and bin. Library dependencies for usr/lib can be discovered using the ldd command. This process is likely to require a degree of experimentation.
5. Running the application
Running the application and restricting it to the environment could be as simple as:
chroot <new root directory> <new root directory>/<full path to application>.
It is at this point that most errors are likely to occur, indicating the absence of application dependencies from the new filesystem.
A more elaborate "virtual machine" could be set-up to initiate a pseudo boot procedure based on /etc/rc startup files that also startup required application daemons. This would require running a suitable startup script (perhaps from the real startup scripts of the real machine).
Example: Webserver
This section looks at setting up Apache [5] in a chrooted environment. Although the details here are more specific to the system that I am using, the principles should be the same for most Unices.
I have created a directory to contain the contents of the new root directory. This has been pre-built so that the symbolic links are already in place.
drwxrwxr-x 2 root wheel 512 May 17 17:01 bin drwxrwxr-x 2 root wheel 512 May 17 17:01 dev drwxrwxr-x 4 root wheel 512 May 17 17:02 etc drwxrwxr-x 2 root wheel 512 May 17 17:02 sbin drwxrwxr-x 8 root wheel 512 May 17 17:03 src drwxrwxr-t 2 root wheel 512 May 17 17:03 tmp drwxrwxr-x 8 root wheel 512 May 17 17:04 usr drwxrwxr-x 3 root wheel 512 May 17 17:04 var
This container directory also contains a Makefile, so that the virtual environment can be installed from this directory using make(1).
The Makefile to install the environment involves copying (or rsync'ing etc) the directories into a new root directory. Then the device files will be built by running the customised MAKEDEV script.
My environment will run in its own disk partition which has been mounted at /var/virtual. To build the environment in here I run
make ROOT=/var/virtual/apache.
The next stage is to install Apache into the virtual environment. Firstly, the compiled Apache and source tree needs to be copied to /var/virtual/src/apache. I can then
chroot /var/virtual "cd /src/apache && make install"
which will install Apache into the virtual environment into the directories specified during its configuration. My Apache configuration arguments included:
PREFIX = /usr/local APACHE_CONFIGURE_ARGS = \ --prefix=${PREFIX} \ --with-layout=GNU \ --enable-module=rewrite \ --enable-module=auth_dbm \ --enable-module=most \ --sysconfdir=${PREFIX}/etc/apache \ --includedir=${PREFIX}/include/apache \ --logfiledir=/var/log \ --runtimedir=/var/run \ --datadir=${PREFIX}/www \ --proxycachedir=${PREFIX}/www/proxy \ --libexecdir=${PREFIX}/libexec/apache \ --without-confadjust \ --enable-shared=remain \
this means that my Apache data directory will be /var/virtual/usr/local/www as viewed from the "real" machine, or /usr/local/www as viewed from within the restricted environment. If building a secure environment that corresponds to an IP address (see below), then obviously the Apache configuration should attempt to run services on this address only.
To ensure that Apache starts up when the machine boots, an etc/rc.d type startup file is put into place within the virtual environment. For FreeBSD, this involves putting in file named something like 50apache.sh into /var/virtual/etc/rc.d. This file looks like a conventional startup file:
#!/bin/sh # this tool makes life easy :-) apachectl=/usr/local/sbin/apachectl # prevent runaway httpd's ulimit -t 300 # start if [ "x$1" = "x" -o "x$1 = "xstart" ] ; then echo -n 'Starting Apache' ${apachectl} start # restart if [ "x$1" = "xrestart" ] ; then echo -n 'Restarting Apache' ${apachectl} restart # stop if [ "x$1" = "xstop" ] ; then echo -n 'Stopping Apache' ${apachectl} stop else echo "usage $0 [start|restart|stop]" 1>&2 exit 1 fi exit 0
When the 'real' machine boots, the startup script /usr/sbin/runvirtual is called. This is a script that 'boots' the virtual environment by emulating a part of the normal boot procedure. The script also configures an IP address corresponding to networked applications running in the environment*, which is then securely configured using an IP firewall utility. The host configuration information stored in system files within the secure environment lists only this address.
Apache should now be running from a virtual environment. Should somebody gain access to the machine through the filesystem, they will find themselves in "chroot jail". In the worst case a rm -fr / will result in the removal of the virtual environment. The real machine should still be operational, and the environment can be rebuilt from its installation directory.
User home directories (~user corresponding to /userdir/user/public_html or similar) will not work, because Apache will not have access to the real filesystem. One idea to enable these is to put a symbolic link from /userdir/user/public_html to /var/virtual/home/user, so that the user automatically writes to the virtual environment.
* Note that in order to assign an IP address to the virtual environment, two changes to the virtual machine's libc were necessary. bind() was altered to associate IN_ADDR_ANY to only the virtual environment IP address. gethostname() was altered to obtain the hostname as given in the virtual environment's etc/hostname.
Summary
Running applications from virtual environments, where possible, adds an extra line of defence between your real machine and the malicious user of a network application running on the machine. Such an environment can be thought of as a virtual machine, but there are important and security relevant differences between the strict idea of virtual machine and a restricted filesystem. Perhaps most importantly is the fact that the virtual environment and real machine are running directly under the same kernel. The implications are that kernel tables and other configuration are shared between both systems. One simple exploitation of this could be filling the process table to capacity having gained access to the virtual environment, in order to incapacitate the real machine.
Improvements to the model described in this article include greater scrutiny regarding general security. For example, resource restraints per user (or on the application process) could offer defence against resource exhaustion attacks as described above. Access to the virtual environment, other than through applications, could be restricted; for example, by confining administrative access to authenticated users of a VPN.
Security policy should not be eased on the virtual machine because it is assumed to be at less risk. Conventional security tools (such as TCP wrappers [6]) can still be run on the host machine. Administrators should be careful not to defeat the object of the virtual environment (for example, by running another web server on the same machine, non-chrooted).
Relevant reading
Many of the points mentioned are relevant to configuring virtual hosts/virtual services; readers may wish to explore this area further.
- Safe and friendly read-only chroot jails for FTP and WWW,
URL: http://www.daemonnews.org/199905/chroot.html.
References
- An Evening With Berferd, in which a Hacker is Lured, Endured, and Studied,
- URL: ftp://ftp.research.bell-labs.com/dist/ches/berferd.ps
Author Details
Ian Peacock
Netcraft
Bath
email address: ip@netcraft.co.uk