Interface to the features exposed by the Interaction Library. More...
Adapters | |
Adapters are the abstraction layer between the Interaction Library and the eyetracking data provider. The Interaction Library ships with a default adapter for Tobii consumer eyetracking devices so setting an adapter is NOT required to use the Interaction Library. Setting a custom adapter can however be useful e.g. when testing by supplying a "fake" adapter implementation that generates controlled test data. | |
virtual Result | SetAdapter (IL_AdapterFunctions *adapterFunctions)=0 |
Sets a custom adapter for this InteractionLib instance, or removes the current adapter. More... | |
Display Properties and Coordinate Transforms | |
In order for the interaction library to function properly, you need to setup the appropriate coordinate system for your environment. This coordinate system will be used for all interactor operations, and all display based coordinates in data streams. Important: If you are using a UI binding with the Interaction Library it should set all of these values for you! If you are, you can skip this section. If not, or if you are developing a UI binding, read on. For most use cases:
Special cases:
Usage tips:
| |
virtual Result | CoordinateTransformAddOrUpdateDisplayArea (float widthUserUnits, float heightUserUnits, float widthVirtualUnits=0, float heightVirtualUnits=0, float xOriginVirtualUnits=0, float yOriginVirtualUnits=0, const char *virtualDisplayAreaId=nullptr)=0 |
Adds a display area (usually a screen) to the coordinate transform setup. Typically call this method multiple times to add information on dpi scaling and location of all screens in the system (see sample/separate documentation for details). More... | |
virtual Result | CoordinateTransformSetOriginOffset (float xOriginOffsetVirtualUnits, float yOriginOffsetVirtualUnits)=0 |
Sets your origin position, as it is in the virtual coordinate system. Typically used to set the window position to use window local coordinates for streams and interactors, or one could set the display center to use a centered, left-handed coordinate system. More... | |
virtual Result | CoordinateTransformSetDisplayDensity (float xUnitsPerMm, float yUnitsPerMm)=0 |
An alternative to CoordinateTransformAddOrUpdateDisplayArea(), normally only used in special circumstances. More... | |
Interactors | |
Interactors is what we call regions on screen that can be interacted with using gaze. Registering interactors that represent objects (e.g. user interface elements or game objects) will elevate gaze based interaction to be object centric rather than pixel/position centric. The Interaction Library contains specialized algorithms to make an informed decision on which object is being interacted with, taking into account factors such as physiological characteristics of eye movements, end user eye movement behavior, momentaneous noise and errors in the raw data, relative sizes and positions of interactors in space, and many more things. Various interaction events are produced for declared interactors when the user looks around (see Interactor Events). Add or update multiple interactors and/or set/change their properties by subsequent calls to AddOrUpdateInteractor(), UpdateInteractorBounds(), UpdateInteractorZ(), RemoveInteractor(), and ClearInteractors(). Batches of updates are performed between calls to BeginInteractorUpdates() and CommitInteractorUpdates(). Calling both of them, and in the correct order, is required (even if updating only a single interactor). Using BeginInteractorUpdates() and CommitInteractorUpdates() is both a performance benefit and an insurance of transaction safety: All changes take effect at the time of a successful commit, not for each individual add/update/remove/clear call. Note however that the interactor functions are not thread local. | |
virtual Result | BeginInteractorUpdates ()=0 |
Begins an interactor update transaction, in which many add/update/remove/clear calls are batched. More... | |
virtual Result | CommitInteractorUpdates ()=0 |
Commits all batched interactor add/update/remove/clear calls done since BeginInteractorUpdates() was called. More... | |
virtual Result | AddOrUpdateInteractor (InteractorId id, Rectangle bounds, float z)=0 |
Adds an interactor or updates an existing one, specifying its bounds and z value (the value increases towards the user). More... | |
virtual Result | UpdateInteractorBounds (InteractorId id, Rectangle bounds)=0 |
Updates the bounds of an existing interactor. More... | |
virtual Result | UpdateInteractorZ (InteractorId id, float z)=0 |
Updates the z value for an existing interactor (the value increases towards the user). More... | |
virtual Result | RemoveInteractor (InteractorId id)=0 |
Removes an existing interactor and all its related data. More... | |
virtual Result | ClearInteractors ()=0 |
Removes all existing interactors. More... | |
Interactor Weight Distributions | |
Weight distributions can be used to describe the visual attention properties of an interactor. As an example, UI buttons tend to have their visual content in the center, so they benefit from being center weighted. Standard Weight Distributions Standard weight distribution types are built into Interaction Library and can be used to specify, for example, center weighted or flat distributions. Use UpdateInteractorStandardWeightDistributionTypes() to change these. Setting a flat weight distribution type removes any previous weight distribution settings. Custom Weight Distributions Custom weight distributions can be used for three purposes:
For stencil maps, values are checked for 0 or >0, with 0 being transparent. For weight distributions, values are in the range 0 to 255, with 0 still obscuring elements behind but not adding weight to selection of focused interactors. A value of 1 denotes normal weight. A combination of both work like weight distributions but with 0 being transparent (like in stencil maps). Custom weight distribution bitmaps can be bound to (used for) multiple interactors. They are stretched to fit the bounds of the bound interactor. You can add a maximum of one stencil map and one weight distribution per interactor (or one combined).
| |
virtual Result | UpdateInteractorStandardWeightDistributionTypes (InteractorId id, WeightDistributionType weightDistributionTypes)=0 |
Sets the built-in standard weight distribution of an interactor, e.g., center weighted or flat. More... | |
virtual Result | AddOrUpdateCustomWeightDistributionBitmap (WeightDistributionId weightDistributionId, bool isWeightDistribution, bool isStencilMap, int bitmapColumns, int bitmapRows, const unsigned char *bitmap)=0 |
Adds a new custom weight distribution bitmap or updates an existing one. More... | |
virtual Result | RemoveCustomWeightDistributionBitmap (WeightDistributionId weightDistributionId)=0 |
Removes an existing custom weight distribution bitmap. More... | |
virtual Result | AddOrUpdateBindingToCustomWeightDistribution (InteractorId id, WeightDistributionId weightDistributionId)=0 |
Binds a (already added) custom weight distribution to an (already added) interactor, or changes an existing binding. More... | |
virtual Result | RemoveBindingToCustomWeightDistribution (InteractorId id, WeightDistributionId weightDistributionId)=0 |
Removes a binding to a custom weight distribution bitmap for the specified interactor. More... | |
Interactor Events | |
Gaze focus events are generated when the user looks at an interactor, as well as when the user's gaze leaves the interactor. Leave events are also produced if the user physically leaves his position in front of the tracker or the tracking stops, etc. | |
virtual Result | SubscribeGazeFocusEvents (GazeFocusEventCallback callback, void *context)=0 |
Subscribes to interactor gaze focus events. More... | |
virtual void | UnsubscribeGazeFocusEvents ()=0 |
Unsubscribes from interactor gaze focus events. More... | |
Data Streams | |
Various device data streams are exposed in the Interaction Library. The data streams can be subscribed to and unsubscribed from, at will, as long as only one callback is registered per data stream. Callbacks are made on the thread calling Update() or UpdateStreamingDataOutput().
| |
virtual Result | SubscribeGazePointData (GazePointDataCallback callback, void *context)=0 |
Subscribes to gaze point data. More... | |
virtual Result | SubscribeGazeOriginData (GazeOriginDataCallback callback, void *context)=0 |
Subscribes to gaze origin data. More... | |
virtual Result | SubscribePresenceData (PresenceDataCallback callback, void *context)=0 |
Subscribes to presence data. More... | |
virtual Result | SubscribeHeadPoseData (HeadPoseDataCallback callback, void *context)=0 |
Subscribes to head pose data. More... | |
virtual void | UnsubscribeGazePointData ()=0 |
Unsubscribes from gaze point data. More... | |
virtual void | UnsubscribeGazeOriginData ()=0 |
Unsubscribes from gaze origin data. More... | |
virtual void | UnsubscribePresenceData ()=0 |
Unsubscribes from presence data. More... | |
virtual void | UnsubscribeHeadPoseData ()=0 |
Unsubscribes from head pose data. More... | |
virtual void | GetDataStreamCapability (StreamType streamType, Capability *capability)=0 |
Get information on the current system capabilities for a data stream. More... | |
virtual void | SubscribeDataStreamCapabilitiesChanged (DataStreamCapabilitiesChangedCallback callback, void *context)=0 |
This callback will be triggered on UpdateStreamingDataOutput()/Update()/WaitAndUpdate() if capabilities have changed. Querying capabilities with GetDataStreamCapability(...) on this callback is allowed. More... | |
virtual void | UnsubscribeDataStreamCapabilitiesChanged ()=0 |
Unsubscribes from data stream capabilities changed notifications. More... | |
Control | |
Various ways of controlling the Interaction Library processing flow. Since the Interaction Library does not have any threads by itself, periodic update calls are needed to collect data from the device, perform all calculations, select the currently focused interactor, produce suitable interaction events and trigger any callbacks including for data streams. WaitForDeviceConnection() is used to connect to the hardware data providers. It needs to be called until a connection is established for any streams to work and any interaction events to be generated through the Update() calls below. It is also needed for reconnection if device connections fail. The Update() method performs all update operations while the UpdateStreamingDataOutput() and UpdateInteraction() methods can be used instead of Update() if more fine-grained control is needed (eg if you need data streams at native frame rates but interaction events less often). WaitForDeviceDataAvailable() can be used to block until new device data is available. This is optional.
| |
virtual Result | WaitAndUpdate (unsigned long connectionIntervalMs=1000)=0 |
Checks/waits for device connection, waits for data, and updates. More... | |
virtual Result | WaitForDeviceConnection ()=0 |
Waits for a device to be available and a connection to be established. More... | |
virtual Result | Update ()=0 |
Updates the system and receives event callbacks that are called on the calling thread. More... | |
virtual Result | WaitForDeviceDataAvailable ()=0 |
Waits for data from the current device. More... | |
virtual Result | UpdateStreamingDataOutput ()=0 |
Updates stream buffers and process filters only, and publishes results on the calling thread via the registered streaming data callbacks. More... | |
virtual Result | UpdateInteraction ()=0 |
Updates interaction buffers, performs interactor selection calculations, and publishes results on calling thread via the registered interaction event callbacks. More... | |
Interface to the features exposed by the Interaction Library.
|
pure virtual |
Sets a custom adapter for this InteractionLib instance, or removes the current adapter.
adapterFunctions | Pointer to an instance of the structure that defines the custom adapter. nullptr is a valid parameter value which will cause the current adapter to be removed. Without a valid adapter the following functions will do nothing and return Result::Warning_NoAdapterSet: |
WaitAndUpdate(), WaitForDeviceConnection(), Update(), UpdateStreamingDataOutput(), UpdateInteraction().
However, it is still possible to add interactors and subscribe to gaze focus events and data streams.
|
pure virtual |
Adds a display area (usually a screen) to the coordinate transform setup. Typically call this method multiple times to add information on dpi scaling and location of all screens in the system (see sample/separate documentation for details).
If you are in a single screen environment, or a defacto single screen environment (see above), or use other (external) mechanisms to handle multiple displays areas, you may call this only once and then omit all parameters except the first two.
If you are unable to determine this information in your environment, please use CoordinateTransformSetDisplayDensity() instead (see below).
widthUserUnits | The width in your preferred units (eg. with dpi scaling in your environment, or other preferred units) |
heightUserUnits | The height in your preferred units (eg. with dpi scaling in your environment, or other preferred units) |
widthVirtualUnits | The width in system virtual screen units (or custom units that you use for system display setup) |
heightVirtualUnits | The height in system virtual screen units (or custom units that you use for system display setup) |
xOriginVirtualUnits | The display origin x coordinate in the system virtual screen coordinate system (or your custom space for system display setup) |
yOriginVirtualUnits | The display origin y coordinate in the system virtual screen coordinate system (or your custom space for system display setup) |
virtualDisplayAreaId | The virtual display id as a null terminated string (eg. on Windows this is a number) |
|
pure virtual |
Sets your origin position, as it is in the virtual coordinate system. Typically used to set the window position to use window local coordinates for streams and interactors, or one could set the display center to use a centered, left-handed coordinate system.
xOriginOffsetVirtualUnits | The x offset in the system virtual screen coordinate system (or your custom space) |
yOriginOffsetVirtualUnits | The y offset in the system virtual screen coordinate system (or your custom space) |
|
pure virtual |
An alternative to CoordinateTransformAddOrUpdateDisplayArea(), normally only used in special circumstances.
This method exists as an alternative or addition to CoordinateTransformAddOrUpdateDisplayArea(). It is mainly intended to be used in edge cases where you are in a sandboxed environment and cannot access information about displays, or if a specialized adapter is used that you cannot access or modify for some reason and CoordinateTransformAddOrUpdateDisplayArea() is insufficient to solve the dpi scaling problem. You can see if you need to use CoordinateTransformSetDisplayDensity() by checking the return value from CoordinateTransformAddOrUpdateDisplayArea(). In almost all situations, using only CoordinateTransformAddOrUpdateDisplayArea() is sufficient.
xUnitsPerMm | The number of horizontal units per mm. |
yUnitsPerMm | The number of vertical units per mm. |
|
pure virtual |
Begins an interactor update transaction, in which many add/update/remove/clear calls are batched.
|
pure virtual |
Commits all batched interactor add/update/remove/clear calls done since BeginInteractorUpdates() was called.
|
pure virtual |
Adds an interactor or updates an existing one, specifying its bounds
and z
value (the value increases towards the user).
id | |
bounds | |
z |
|
pure virtual |
Updates the bounds
of an existing interactor.
id | |
bounds |
|
pure virtual |
Updates the z
value for an existing interactor (the value increases towards the user).
id | |
z |
|
pure virtual |
Removes an existing interactor and all its related data.
id |
|
pure virtual |
Removes all existing interactors.
|
pure virtual |
Sets the built-in standard weight distribution of an interactor, e.g., center weighted or flat.
id | |
weightDistributionTypes |
|
pure virtual |
Adds a new custom weight distribution bitmap or updates an existing one.
weightDistributionId | |
isWeightDistribution | |
isStencilMap | |
bitmapColumns | |
bitmapRows | |
bitmap |
|
pure virtual |
Removes an existing custom weight distribution bitmap.
weightDistributionId |
|
pure virtual |
Binds a (already added) custom weight distribution to an (already added) interactor, or changes an existing binding.
id | |
weightDistributionId |
|
pure virtual |
Removes a binding to a custom weight distribution bitmap for the specified interactor.
id | |
weightDistributionId |
|
pure virtual |
Subscribes to interactor gaze focus events.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Unsubscribes from interactor gaze focus events.
|
pure virtual |
Subscribes to gaze point data.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Subscribes to gaze origin data.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Subscribes to presence data.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Subscribes to head pose data.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Unsubscribes from gaze point data.
|
pure virtual |
Unsubscribes from gaze origin data.
|
pure virtual |
Unsubscribes from presence data.
|
pure virtual |
Unsubscribes from head pose data.
|
pure virtual |
Get information on the current system capabilities for a data stream.
Data stream availability is dependent on if a device is present, what type of device it is and current system state of the device being used. Data streams are typically not enabled unless the they are subscribed to, either internally by the Interaction Library or from this api.
streamType | The stream type to get capability information on. | |
[out] | capability | Will be set to the system capability for the stream type. |
|
pure virtual |
This callback will be triggered on UpdateStreamingDataOutput()/Update()/WaitAndUpdate() if capabilities have changed. Querying capabilities with GetDataStreamCapability(...) on this callback is allowed.
callback | |
context | Will be passed to the callback when the callback is invoked. |
|
pure virtual |
Unsubscribes from data stream capabilities changed notifications.
|
pure virtual |
Checks/waits for device connection, waits for data, and updates.
This is a convenience method that aggregates all necessary control methods into one, and sleeps the calling thread for the appropriate amount of time between connection attempts when no device is connected. The default connection attempt interval is one second, passing 0 means no sleep.
If you need granular control over updates and/or non-blocking calls use a suitable combination of the various other control methods.
|
pure virtual |
Waits for a device to be available and a connection to be established.
|
pure virtual |
Updates the system and receives event callbacks that are called on the calling thread.
|
pure virtual |
Waits for data from the current device.
May timeout or hang indefinitely depending on adapter implementation. Waiting is optional but if you do not wait, remember to call Update() or UpdateStreamingDataOutput() at least 10 times per second. Do call Update() (or some combination of the other update methods) after waiting regardless of return code to make sure all proper events are generated.
|
pure virtual |
Updates stream buffers and process filters only, and publishes results on the calling thread via the registered streaming data callbacks.
Use instead of Update() if you need more granular control or use multiple threads.
|
pure virtual |
Updates interaction buffers, performs interactor selection calculations, and publishes results on calling thread via the registered interaction event callbacks.
Use instead of Update() if you need more granular control or use multiple threads.