Tobii Interaction Library SDK

[Back to the SDK start page]

Coordinate systems, scaling, multiple displays and display enumeration

tl;dr

The problems with multiple monitors and/or coordinate scaling descibed here are usually easily solved on Windows consumer systems by using the display enumeration helpers included in the SDK; please see the display enumeration samples linked from the SDK start page.

Introduction

Most samples in this SDK start with a basic assumption: That a standard Tobii eye tracking device is being used, that that device is attached to the main monitor in the system and that you know the resolution of that monitor in your application coordinate space, including whatever scaling factors or dpi settings are in play. (They also assume that the end user has already performed calibration and display setup.)

This is taken from the basic interactor C++ sample:

constexpr float width = 2560.0f;
constexpr float height = 1440.0f;
constexpr float offset = 0.0f;
intlib->CoordinateTransformAddOrUpdateDisplayArea(width, height);
intlib->CoordinateTransformSetOriginOffset(offset, offset);

After this call, Interaction Library will scale gaze data stream values to (0 < x < 2560, 0 < y < 1440) when the user is looking at the screen (negative values or higher values indicate the user is looking just outside the screen). Interaction Library will also assume that interactors you add are in that same coordinate space, and that the eyetracking device is attached to whatever monitor you intend when you add interactors.

However, it is quite common for users to have multiple monitors, and the eyetracking device may not be on the main monitor of the system, or the monitor where your application is located. The recommended solution to this problem is the following:

  1. (Prerequisite) The end user performs display setup and calibration. During display setup, the configuration tool will write the ID of the display the eyetracking device is attached to to the eyetracking device.
  2. When initializing the Interaction Library, you should enumerate ALL displays on the system, including virtual screen bounds and IDs for each one and tell Interaction Library about ALL of them.
  3. You should set the origin offset either to your window location and use window-local coordinates, or to (0, 0) to use global coordinates.

This will assure that when you add interactors, they will receive events when they are globally located where the user is looking, and that gaze data streams have values that are globally correct.

About the CoordinateTransformAddOrUpdateDisplayArea() and CoordinateTransformSetOriginOffset() parameters

When telling the Interaction Library about all the displays using CoordinateTransformAddOrUpdateDisplayArea(), you supply 3 pairs of values and lastly an ID.

The first two values are effectively scaling parameters; you tell the Interaction Library the width and height of the display as you see them, and Interaction Library will scale values accordingly (input and output).

The next four values are width, height, and origin of the display in global virtual screen coordinates. These are used to know where the any interactors are located globally, after considering the current offset origin (that you set using CoordinateTransformSetOriginOffset()) and scaling. Importantly, setting the offset is always done in the virtual screen coordinate space, regardless of your supplied scaling. Lastly, the given ID is matched to the display ID on the eyetracking device being used.

On Windows systems

The normal case

Assuming Tobii devices on Windows, on a standard consumer setup, the installed Tobii configuration tools will automatically perform the prerequisite step 1 (see above) for you.

Step 2 requires you to know how to enumerate all attached monitors and how Tobii identifies displays. This involves a lot of arduous Win32 calls, so we have written a header only helper library to help you deal with it. You can see how to use it in the samples linked from the main page. The scaling values are up to you, but a simple approach is to use the virtual screen width and height as that probably will correspond to your UI frameworks idea of scaling for that display.

Step 3 is easy if you hav a single window and are not interested in anything outside of that; simply set the window lcoation whenever it moves (in virtual screen coordinates) and you are good to go. If you want screen wide data streams coordinates or have multiple windows, the easiest approach is to use global coordinates and calculate proper global locations for your interactors before adding them to the Interaction Library. In this case, remember to update your interactors whenever a window containing them moves.

When using the WPF binding

The WPF binding already uses the display enumeration helpers under the hood if you let the WPF binding initilize the Interaction Library, so you don't need to do anything in that case. This is the benefit of using a UI binding! If you supply the Interaction Library instance to the WPF binding before initializing it however, you are responsible for display enumeration as in the normal case above.

Custom scenarios

Using the Windows virtual screen coordinates and the Tobii IDs for displays is of course all optional.

  • If you have a custom setup with custom display configuration tools, you need to make sure the IDs you tell Interaction Library about match the IDs that the configuration tool you are using generates.
  • You can also use custom virtual screen coordinate systems if you like, just remember to use the same coordinate space when setting the origin offset.

Other operating systems

On other operating systems, there currently are no Tobii consumer setups, Tobii defined display IDs or Tobii standardized configuration tools (and no display enuemration helper libraries). On these systems, you need to make sure what you tell the Interaction Library regarding display IDs matches your setup. The general guidelines in this document still apply, however.