On online judging, part 2: the Windows sandbox

In the last chapter of this topic, we talked about creating a sandbox for Linux-based systems.

On Linux, we have a fine-grained control over untrusted code by intercepting all system calls via the ptrace(2) API. Since a similar API does not exist on Windows, alternative methods must be used.

Unfortunately, there isn't a nice solution as for Linux: we have to deal with securing the filesystem, network, and resources separately. Furthermore, we will require administrator rights — Windows APIs are simply not powerful enough to sandbox in any way without them.

Securing the Filesystem

For every submission, we can create an unprivileged user programatically with NetUserAdd. This user has no initialized profile (i.e., no C:\Users directory, and no registry). Untrusted user submissions are ran under this user, such that any malicious code is not able to tamper with the filesystem. This user is deleted after the submission terminates.

Of course, the user needs to have read access to a directory where the program executable is stored. We may mount a small virtual disk for the sandbox to use as a temporary storage device, or use some arbitrary C:\-level folder instead.

Securing the Network

Submissions should not have access to network resources (they could use this to initiate DOS attacks, download answers, or perform any other myriad of unwanted things). We can add a temporary firewall rule (INetFwRules) for each submission binary, disallowing all internet access. The rule is cleaned up at the end of grading.

Resource Limiting

User code is generally run under strict time and memory limits that must be enforced by the sandbox. Ideally, submissions should not be allowed to tamper with global settings (e.g., swapping the mouse buttons), and should not attempt to create UIs.

Windows has Job objects that may be used for this purpose. We can create a Job object with the desired constraints, and attach the submission process to it.

Caveats and workarounds

At this point, one might think that a sandbox is complete: the filesystem is secured, submissions cannot access the network, and cannot tamper with user settings. There is, as always, a catch.

When CreateProcessWithLogonW is used to spawn the submission process under an unprivileged user, the Secondary Logon Service (SLS) is invoked. This would not be an issue, except that it puts the process into its own Job object. Windows 8 added support for processes being part of multiple Job objects, but compatibility is important. The problem is worsened by the inexistence of an API to fetch the Job object associated with a process.

We can work around the issue by using undocumented ntdll API to enumerate all system handles. We can filter these by type, and for all Job handles call IsProcessInJob to determine which Job the user process belongs to. Once this SLS Job object is found, it must be modified directly to add our restrictions. Since Windows 8, Job objects may be chained; however, for compatibility, we instead overwrite the SLS Job object with our own.

Silencing Runtime Errors

In online judging, Runtime Error verdicts are common. Whenever a user submission crashes, Windows "helpfully" opens a dialog informing the host that "the process has stopped unexpectedly", and that Windows is "searching online for a solution". Since the dialog is modal, execution of the sandbox is stopped until the host manually closes the dialog.

Thankfully, the Windows API contains a SetErrorMode that can indicate to the operating system that the submission can handle runtime errors itself. Calling SetErrorMode silences the dialogs, and since this property is inherited by child processes, setting the error mode for the sandbox process should be a global solution.

More caveats… and more workarounds

When CreateProcessWithLogonW is used, the created process is not considered a child of the calling process: it is created as a child of the Secondary Logon Service. Therefore, calling SetErrorMode on the sandbox will have no effect on submission processes.

The rabbit hole just keeps getting deeper and deeper… we must inject a DLL into the user process to call SetErrorMode from inside it.

Further Restrictions

win32k.sys handles system calls pertaining to NtUser/GDI — calls that submissions to an online judge should never have to call. Since Windows 8, the Windows API has a SetProcessMitigationPolicy that allows complete disabling of all win32k.sys calls. Since we're injecting the user process anyway for SetErrorMode, we can use this opportunity to SetProcessMitigationPolicy as well.

This feature is not critical, but does decrease attack surface of a sandbox (a number of exploits have been discovered using the win32k API).

Conclusion (and Viability for Online Judging)

Setting all this up is definitely a lot of work, and is really a collection of hacks piled sky-high. Is it worth it, to be able to run Windows judges?

In my experience, the answer would have to be no. Unless your entire infrastructure is tied to Windows, you'll be getting more mileage out of a Linux-based sandbox rather than a hack. Furthermore, in running a Windows judge over at DMOJ for half a year, the performance difference between Windows and Linux machines for contests is obvious: under the same hardware specs, Linux submissions typically ran 3-4x faster than their Windows counterparts.

Furthermore, as of Windows 10, Windows users have the option of enabling the Windows Subsystem for Linux, which allows them to run a ptrace(2)-based Linux sandbox… on Windows.