Sync Unity and physiology

Let’s say you want to record the position of a ball or whatever gameObject you use in Unity.
Or if you are using VR — you wish to stream the position/rotation of the head and controllers while in VR.
Also, at the same time you are measuring some physiology, e.g., the galvanic skin response (GSR).
During the experiment, you have certain triggers/events, also called Markers, that you also want to keep track of…

You wish to sync everything together? Well, my advice is use Lab Streaming Layer (LSL)

Why LSL?

It is easy to implement in many languages and easy to use.
It can sync many data automatically, meant for physiological recording.
It handles multiple machines at the same time with high precision, so one could sync even ERPs.
And no need for ip addresses anymore when communicating between machines.
It works on Local Network as well as WAN
(I advise you use a local router or hub to avoid interference)

Send data streams from Unity :

(Send data streams to Unity – bellow.)

Most importantly, when using LSL, make sure you have DEACTIVATED FIREWALL for Unity and Openvibe and other apps in use!!!

1. For offline analysis :

(How to : bellow)

One can record all data in an .xdf file with the Lab Recorder,
and analyze offilne in Matlab for example, with eeglab or other toolboxes.

How To: (offline)

1. download LSL4Unity (this is a modified version from the original xfleckx )
2. download latest Lab Recorder
3. download Unity,
you would need to have a game already made:
e.g., Rolling A Ball — from the Unity tutorial,
or if you do VR, you would need an environment ready to use,
for HTC Vive for example: from htc-vive-tutorial-unity
4. For this tutorial Windows 7+ is needed (I tested on windows 10)

Once you have downloaded and unzipped all the folders,
I advice you to follow the nice tutorials for Unity before using LSL.

Make a copy of the Unity project you intend to use and rename it
e.g., Roll_a_Ball-LSL (to keep your original project without lsl).
Then copy the LSL4Unity folder in the Assets folder from your Unity project.
So when you enter your project in Unity it looks like this:

Send streams from Unity

such as position, rotation of the gameObject or of head and controllers,

Send Marker streams from Unity

such as collision between game objects etc.

 

    1. once you followed the instructions above, start game in Unity (press play)
    2. open Lab Recorder


(Save the .xfd file in the desired destination and click on Update to see active streams)

  1. check all the streams you are interested in recording (Marker streams with continuous streams)
    and press Start to Record!

Receive data in Unity :

Receive streams in Unity

physiological stream which can influence your gameObjects’ positions for instance,

Receive Marker/ events in Unity

such as openvibe stimuli .

2. Online processing:

(How to : soon)

You can use openvibe which has LSL already integrated,
or Simulink (working on it)!