Blazing-fast Java2D rendering
Anyone who has ever attempted to draw anything more than almost-static scenes with Java2D can attest that it sluggishly chugs along. Some will even say it's even unusable for repainting at 60Hz or higher without taking a toll on CPU.
Today, we'll look at what we can do to speed up rendering, in ways that (at the time of writing) I have not seen discussed anywhere online. Probably because it's a big hack.
Vanilla Java2D Rendering, and Caveats
The reader may be familiar with the way Java handles drawing in swing. If not, the official documentation is a good starting place.
What's important to note is that when you wish to update a frame in your Java2D application, you must first request a repaint, which is processed by putting a repaint event onto the event queue. If there are things earlier in the event queue, they must first get processed before you can repaint.
It's not even guaranteed that a repaint request will cause a repaint — sometimes,
multiple repaint events can get "squashed" into one, causing jittery animations. Of course, there are workarounds like
paintImmediately,
for example, but none provide outstanding performance for even the simplest scenes: there's simply too much abstraction,
which is a killer when every millisecond counts to obtain an immersive rendering experience.
A simple benchmark
Below is a simple Swing application that does nothing more than draw a red, full-window rectangle. We'll be using it as a benchmark for the purposes of this post — though it is not a particularly good real-life example, the effects of Java2D abstraction are fairly uniform across the entire API: if we can get this to run quickly, so too will everything else.
class PaintFrame extends JFrame {
int frameCount;
{
setSize(720, 680);
JPanel canvas;
add(canvas = new JPanel() {
@Override
public void paint(Graphics gfx) {
gfx.setColor(Color.RED);
gfx.fillRect(0, 0, getWidth(), getHeight());
frameCount++;
}
}, BorderLayout.CENTER);
setLocationRelativeTo(null);
setVisible(true);
new Thread(() -> {
while (true) {
canvas.paintImmediately(0, 0, getWidth(), getHeight());
}
}).start();
}
}
We can also hook our frame up to a simple, but illustrative, test.
public class PaintTest {
public static void main(String[] argv) {
PaintFrame frame = new PaintFrame();
new Timer().schedule(new TimerTask() {
double seconds;
@Override
public void run() {
seconds++;
System.out.printf("Averaging %.2f fps!\n", frame.frameCount / seconds);
}
}, 1000, 1000);
}
}
Running on my Intel HD 5500 integrated graphics, the example code above averages ~1,200 frames per second. This quickly drops to below 400fps when making the window fullscreen, which is unacceptable for anything where framerate matters.
The significance of this deserves a bit more explanation (400fps is far more than the human eye can see!) Here, we're doing nothing more than drawing a red rectangle as fast as possible; there is no application logic taking up resources at the same time, and a single frame takes 2.5ms to process.
To maintain a 120fps framerate, each frame should be processed in ~8.3ms. If we're taking 2.5ms just to draw a single red rectangle, that leaves 5.8ms per frame for application logic: rendering would consume ~30% of application time. Naturally, rendering time increases the more you have to draw per frame, and our 2.5ms measurement is for a single rectangle.
Now that we've seen how vanilla Java2D rendering performs, let's see if we can do better.
A Hack for Fast Rendering
Java2D provides output with OpenGL, Direct3D, GDI, and more, depending on platform. Most of these are inherently active-rendering APIs, so there should be no technical barrier preventing us from rendering directly to them… except for abstraction.
Disclaimer: if your application needs to run on more than just Oracle's VM (or equivalently, OpenJDK), your mileage may vary with this approach. As I mentioned earlier, it's a hack specific to the internals of the Oracle API implementation, so it's unlikely to work anywhere else.
Let's start off with an observation. If we try printing out a Graphics object passed to paint, we'll see that it's implemented by
sun.java2d.SunGraphics2D.
We can also see that we're passed a different object each frame, so that's already a waste of GC resources, if we're pumping out
hundreds of frames a second.
If we could construct our ownSunGraphics2D object, we'd be able to reuse it and any underlying resources outside of our
paint method, directly in our rendering thread. The SunGraphics2D constructor is pretty benign, so that's promising.
public SunGraphics2D(SurfaceData sd, Color fg, Color bg, Font f)
At first glance, this seems fairly mild for such a fundamental class. The only thing that appears tricky is the SurfaceData parameter.
Obtaining a SurfaceData
SurfaceData sounds exactly like what one would expect an abstraction of a native surface to be called, and if we
dig into its source,
it becomes evident that SurfaceData implementations (the class itself is marked abstract) do the heavy lifting
in rendering Java2D. If we search for implementations, we get names like D3DWindowSurfaceData, GDIWindowSurfaceData,
XSurfaceData, and so on.
It's clear that any rendering we do will have to be platform-dependent, so let's stick to the
GDIWindowSurfaceData for now. Naturally, this is will work only on Windows, but idea is what's important, and generalizes to other
platform-specific surface implementations.
If we take a look at the
source for GDIWindowSurfaceData,
we find a very helpful function:
public static GDIWindowSurfaceData createData(WComponentPeer peer) {
SurfaceType sType = getSurfaceType(peer.getDeviceColorModel());
return new GDIWindowSurfaceData(peer, sType);
}
…and all we need to use it is a WComponentPeer, which we can obtain from our panel's (deprecated) getPeer method!
Note: getPeer is removed in the Java 9 EAP; equivalently, you can use reflection to fetch the peer field directly.
Importantly, all SurfaceData implementations provide a createData static method, so it's possible to use reflection to make
accessing code more portable and elegant. But, that's beyond the scope of this post.
An improved benchmark
Putting these together, we can come up with a solution that allows us to draw at our own pace, outside of Swing/AWT entirely.
class PaintFrame extends JFrame {
int frameCount;
{
setSize(720, 680);
// Note that this now a heavyweight Panel: JPanels don't have real native peers
Panel canvas = new Panel();
add(canvas, BorderLayout.CENTER);
setLocationRelativeTo(null);
setVisible(true);
ComponentPeer peer = canvas.getPeer();
SurfaceData surfaceData = GDIWindowSurfaceData.createData((WComponentPeer) peer);
SunGraphics2D gfx = new SunGraphics2D(surfaceData, Color.BLACK, Color.BLACK, null);
new Thread(() -> {
gfx.setColor(Color.RED);
while(true) {
gfx.fillRect(0, 0, getWidth(), getHeight());
frameCount++;
}
}).start();
}
}
On the same hardware, our new rendering approach can pump out ~14,000 frames per second, which drops to ~6,000fps when fullscreen. That's a 20x speed improvement over regular rendering!
A Practical Conclusion
It's nice to be able to say we can render 20x faster by employing this approach. But, it's not without caveats: you need to implement a backend for each platform you wish to be able to render on, or at least provide a fallback to regular Swing drawing when you cannot.
In other words, it's not practical for simple one-off Swing applications. Nor is it practical for speeding up general rendering of Swing components. However, if your task involves repainting a large component as fast as possible, this is definitely the fastest you can get without linking 3rd party libraries to perform Direct3D/OpenGL rendering yourself.
You can view a more complete implementation of the ideas expressed in this post in a Gameboy Color emulator I wrote, where Java2D was taking more time to render than the rest of the emulation combined (which spurred me to develop this approach).