3D ingame UI with scene2d.ui in LibGDX

2D user interfaces in a 3D environment are something pretty cool. Probably the most well-known game for utilizing UIs in very immersive way is Dead Space.

Dead Space three dimensional ingame UI

Another possible application of this are ingame computer screens with minigames, or menus.

Computer screens in Dead Space with interactive interfaces

How can anything like this be achieved with LibGDX? That’s super easy!

UIs in LibGDX are usually made with the help of a Stage and Actors. The scene2d.ui package offers a bunch of useful widgets like Label, Button, ScrollPane and many more. But by default those will be drawn with an OrthogonalCamera into the X/Y plane.

Now how to get those in a 3D enironment? We just replace the camera! See this forum thread for NateS original post about that.

float width = Gdx.graphics.getWidth();
float height = Gdx.graphics.getHeight();
PerspectiveCamera camera = new PerspectiveCamera(90, width, height);
camera.position.set(width / 2, height / 2 - 200, 200);
camera.near = 0.1f;
camera.far = 1000;
camera.rotate(30, 1, 0, 0);

Since the standard Stage class doesn’t expect a PerspectiveCamera, we need to adjust the way picking (conversion from screen coordinates to stage coordinates) is done.

public class Stage3D extends Stage {

    public Vector2 screenToStageCoordinates (Vector2 screenCoords) {
        Ray pickRay = getViewport().getPickRay(screenCoords.x, screenCoords.y);
        Vector3 intersection = new Vector3(0, 0, 1);
        if (Intersector.intersectRayPlane(pickRay, new Plane(new Vector3(0, 0, 1), Vector3.Zero), intersection)) {
                screenCoords.x = intersection.x;
                screenCoords.y = intersection.y;
        } else {
                screenCoords.x = Float.MAX_VALUE;
                screenCoords.y = Float.MAX_VALUE;
        return screenCoords;


Another problem is that clipping (which is done via the ScissorStack) won’t be correct anymore, due to the new perspective. That might look like the next picture. Note that it doesn’t look like it, but it’s actually a rectangle which has been cut out because of the clipping.

Parts of the UI being cut off because of clipping

By default clipping is used for all widgets that are based on Table. Some of them need clipping, for example the ScrollPane, but a Window usually doesn’t. As a quick workaround for that problem it is possible to disabled clipping on all widgets via setClip(false).

The end result could look like the following. The camera is controlled via a CameraInputController. Notice that the input is perfectly accurate.

An inventory in a 3D environment.

Leave a Reply

Your email address will not be published. Required fields are marked *