libLSL
Lately I’ve been looking at Lab Streaming Layer (LSL) for my day job. LSL is an open-source cross-platform software framework for collecting time-series measurements that handles the networking, time-synchronization, collection, viewing, and recording of data. Their docs are pretty good and they have plenty of examples in their github repository. This blog post probably won’t teach you anything you couldn’t get from the docs, but I just felt like writing some notes on my experiences with it.
Install
They have arm64, amd64, and x86 files for various operating systems pre-built in their GitHub Release page. So if you’re on one of the supported systems, the easiest thing to do is download one of their packages.
I prefer building from source, so I downloaded the tarball for version 1.16.2 and unpacked it.
wget https://github.com/sccn/liblsl/archive/refs/tags/v1.16.2.tar.gz
tar -xvf v1.16.2.tar.gzThey include a script standalone_compilation_linux.sh that can handle compiling the library for you.
$ ./standalone_compilation_linux.shThe script assumes the source tarball is a repository, so it reports a fatal error when it calls git.
+ git describe --tags HEAD
fatal: not a git repository (or any of the parent directories): .git
+ echoThis doesn’t seem to matter as the script continues anyway.
+ g++ -fPIC -fvisibility=hidden -O2 -Ilslboost -DBOOST_ALL_NO_LIB -DASIO_NO_DEPRECATED -DLOGURU_DEBUG_LOGGING=0 -DLSL_LIBRARY_INFO_STR="built from standalone build script" src/api_config.cpp src/buildinfo.cpp src/cancellation.cpp src/common.cpp src/consumer_queue.cpp src/data_receiver.cpp src/info_receiver.cpp src/inlet_connection.cpp src/lsl_inlet_c.cpp src/lsl_outlet_c.cpp src/lsl_resolver_c.cpp src/lsl_streaminfo_c.cpp src/lsl_xml_element_c.cpp src/netinterfaces.cpp src/resolve_attempt_udp.cpp src/resolver_impl.cpp src/sample.cpp src/send_buffer.cpp src/socket_utils.cpp src/stream_info_impl.cpp src/stream_outlet_impl.cpp src/tcp_server.cpp src/time_postprocessor.cpp src/time_receiver.cpp src/udp_server.cpp src/util/cast.cpp src/util/endian.cpp src/util/inireader.cpp src/util/strfuns.cpp thirdparty/pugixml/pugixml.cpp -Ithirdparty/pugixml thirdparty/loguru/loguru.cpp -Ithirdparty/loguru -Ithirdparty/asio lslboost/serialization_objects.cpp -shared -o liblsl.so -lpthread -lrt -ldl
In file included from /usr/include/c++/12/bits/stl_algobase.h:64,
from /usr/include/c++/12/memory:63,
from thirdparty/asio/asio/detail/memory.hpp:21,
from thirdparty/asio/asio/execution/detail/as_invocable.hpp:20,
from thirdparty/asio/asio/execution/execute.hpp:20,
from thirdparty/asio/asio/execution/executor.hpp:20,
from thirdparty/asio/asio/execution/allocator.hpp:20,
from thirdparty/asio/asio/execution.hpp:18,
from thirdparty/asio/asio/any_io_executor.hpp:22,
from thirdparty/asio/asio/basic_socket_acceptor.hpp:19,
from thirdparty/asio/asio/ip/tcp.hpp:19,
from src/socket_utils.h:4,
from src/time_receiver.h:4,
from src/time_receiver.cpp:1:
/usr/include/c++/12/bits/stl_pair.h: In instantiation of ‘constexpr std::pair<typename std::__strip_reference_wrapper<typename std::decay<_Tp>::type>::__type, typename std::__strip_reference_wrapper<typename std::decay<_Tp2>::type>::__type> std::make_pair(_T1&&, _T2&&) [with _T1 = double&; _T2 = double&; typename __strip_reference_wrapper<typename decay<_Tp2>::type>::__type = double; typename decay<_Tp2>::type = double; typename __strip_reference_wrapper<typename decay<_Tp>::type>::__type = double; typename decay<_Tp>::type = double]’:
src/time_receiver.cpp:176:40: required from here
/usr/include/c++/12/bits/stl_pair.h:741:5: note: parameter passing for argument of type ‘std::pair<double, double>’ when C++17 is enabled changed to match C++14 in GCC 10.1
741 | make_pair(_T1&& __x, _T2&& __y)
| ^~~~~~~~~
+ gcc -O2 -Iinclude testing/lslver.c -o lslver -L. -llsl
+ LD_LIBRARY_PATH=. ./lslver
LSL version: 116
built from standalone build script
2415.792841The build compelted for me and created a shared library named liblsl.so and an executable named lslver in the folder. lslver just tells you the version of your LSL and how it was made.
Building with CMake
You can also use CMake to build the
sudo apt install cmakeThe steps I took with CMake were
- Change directories into liblsl’s source folder
cd liblsl-1.16.2- make a build directory
mkdir build
cd build- Configure it with CMake
cmake ..- Build and install
make
sudo make installThis installs the files into /usr/local/ by default.
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/lib/liblsl.so.1.16.2
-- Installing: /usr/local/lib/liblsl.so.2
-- Installing: /usr/local/lib/liblsl.so
-- Installing: /usr/local/lib/cmake/LSL/LSLTargets.cmake
-- Installing: /usr/local/lib/cmake/LSL/LSLTargets-noconfig.cmake
-- Up-to-date: /usr/local/include
-- Installing: /usr/local/include/lsl_c.h
-- Installing: /usr/local/include/lsl
-- Installing: /usr/local/include/lsl/types.h
-- Installing: /usr/local/include/lsl/resolver.h
-- Installing: /usr/local/include/lsl/xml.h
-- Installing: /usr/local/include/lsl/streaminfo.h
-- Installing: /usr/local/include/lsl/outlet.h
-- Installing: /usr/local/include/lsl/common.h
-- Installing: /usr/local/include/lsl/inlet.h
-- Installing: /usr/local/include/lsl_cpp.h
-- Installing: /usr/local/lib/cmake/LSL/LSLCMake.cmake
-- Installing: /usr/local/lib/cmake/LSL/LSLConfig.cmake
-- Installing: /usr/local/lib/cmake/LSL/LSLConfigVersion.cmake
-- Installing: /usr/local/bin/lslver
-- Set runtime path of "/usr/local/bin/lslver" to "$ORIGIN:$ORIGIN/../lib"When the build completes, you should have files for liblsls.so and lslver Running lslver shows the version of LSL.
./lslver
ggallard@muninn:~/lsl/liblsl-1.16.2/build $ ./lslver
LSL version: 116
git:unknown/branch:unknown/build:/compiler:GNU-12.2.0/link:SHARED
738.962552Python Installation
LSL supports several languages including Python. On my Raspberry Pi Zero I used pip to install it in a virtual environment
$ python3 -m venv venv
$ . venv/bin/activate
(venv)$ pip3 install pylslAfter installation, I needed to point pylsl at my libsls.so file. How you do this will depend on where you’ve installed it.
If you used, standalone_compilation_linux.sh you can use PYLSL_LIB to tell pylsl where to find liblsl.so.
PYLSL_LIB=/home/ggallard/LSL/lib/liblsl.so or use LD_LIBRARY_PATH to tell it where to search for the shared library.
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/libSending Data
LSL uses Stream Outlets to transmit time-series data over a local network.
You can potentially have many stream outlets transmitting over your network, so outlets have metadata (names, content-type, format type, source-id, etc ) to help you pick the streams you want to listen to.
I use lsl::stream_info to specify the metadata for an LSL stream.
// Create stream info with custom metadata for channel names
const size_t channelCount = 5;
lsl::stream_info streamInfo("ExampleDoubleStream", // Stream Name - Describe the device here
"RandomValues", // Content Type of the Stream.
channelCount, // Channels per Sample
1, // Sampling Rate (Hz)
lsl::cf_double64, // Format/Type
"c++source987456" // SourceID (Make Unique)
);Note that streams consist of a single format type (floats, ints, strings, etc) and have a known number of channels per sample. In my example I set the channels per sample to 5 and set the format to double.
After you have your meta data ready, use it to create a stream_outlet
lsl::stream_outlet streamOutlet( streamInfo );The outlet will transmit the stream metadata to all devices on the local network using UDP multicast. This lets any program or device interested in the streams find them.
Sending data is easy. You just call push_sample() with the data you want to send
// setup random samples for outlet stream
std::vector<double> sample(channelCount);
...
streamOutlet.push_sample(sample);The following code is a simple C++ example that continuously sends 5 channels of random doubles.
#include <iostream>
#include <vector>
#include <random>
#include <thread> // for sleep
#include <lsl_cpp.h>
int main(int argc, char* argv[]) {
// Create stream info with custom metadata for channel names
const size_t channelCount = 5;
lsl::stream_info streamInfo("ExampleDoubleStream", // Stream Name
"RandomValues", // Content Type
channelCount, // Channels per Sample
1, // Sampling Rate (Hz)
lsl::cf_double64, // Format/Type
"c++source987456" // SourceID (Make Unique)
);
lsl::stream_outlet streamOutlet( streamInfo );
// setup random samples for outlet stream
std::vector<double> sample(channelCount);
std::random_device rd;
std::mt19937 gen(rd());
std::uniform_real_distribution<> uni(0.0, 1.0);
while (true) {
// fill sample vector with random values.
for( auto &val : sample ) {
val = uni(gen);
}
streamOutlet.push_sample(sample);
// print sample for verification
std::cout << "Sample sent: ";
for (const auto &val : sample) {
std::cout << val << " ";
}
std::cout << std::endl;
// Sleep for 1 second.
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return 0;
}The equivalent in Python looks like this:
import random
import time
from pylsl import StreamInfo, StreamOutlet
# Create stream_info with mixed channels: 2 for numeric (double and int), 1 for string
channel_count = 5
info = StreamInfo('ExampleDoubleStream', 'RandomValues', channel_count, 1, 'double64', 'pysourceid814709')
# Create an outlet
outlet = StreamOutlet(info)
while True:
# Fill sample with random values
sample = [random.random() for _ in range(5)]
# Send the sample
outlet.push_sample(sample)
print(f"samples send: {sample}")
# Sleep for 1 second
time.sleep(1)Reading a Stream
Now that we have a stream outlet, we need a way to read it in. LSL provides stream inlets for this purpose.
We’ll need to get the stream_info in order to create a stream_inlet. The lsl::resolve function is used to search for LSL outlets on your network.
std::vector<lsl::stream_info> streamInfo =
lsl::resolve_stream("name", // property (name, type, source_id, desc/manufacture
"ExampleDoubleStream", // value the property should have
1, // minimum number of streams
lsl::FOREVER // timeout
);LSL will look at UDP multicast messages to find streams matching your search criteria. In my case, I"m looking for outlet streams with the name ‘ExampleDOubleStream’. You might get several streams with this name, so lsl::resolve_stream() returns a vector of stream_info.
Note I used "name" to search for my outlet, but content-type ("type") is the preferred way to find streams on your network.
When reading a stream, you’ll need to know how many channles each sample has. You can get this from the stream info with the channel_count() method.
size_t channelCount = streamInfo[0].channel_count();Once you have the channel count, you can allocate enough space for the sample and read it with pull_sample()
std::vector<double> sample(channelCount);
// Pull a sample from the inlet
streamInlet.pull_sample(sample);The following code is a simple C++ example that continously reads 5 channles from our random stream of double values.
#include <iostream>
```c++
#include <cmath>
#include <iostream>
#include <vector>
#include <thread>
#include <lsl_cpp.h>
int main(int argc, char* argv[]) {
// Look for streams
std::vector<lsl::stream_info> streamInfo = lsl::resolve_stream("name", // property (name, type, source_id, desc/manufacture
"ExampleDoubleStream", // value the property should have
1, // minimum number of streams
lsl::FOREVER // timeout
);
std::cout << "Found " << streamInfo.size() << " streams\n";
if( streamInfo.size() == 0 ) {
std::cerr << "No streams found. Exiting.\n";
return -1;
}
size_t channelCount = streamInfo[0].channel_count();
std::cout << "Channel Count: " << channelCount << std::endl;
// Create an inlet to receive data
lsl::stream_inlet streamInlet(streamInfo[0]);
// Buffer to hold the received sample data
std::vector<double> sample(channelCount);
while (true) {
// Pull a sample from the inlet
streamInlet.pull_sample(sample);
// Print the received sample
std::cout << "Received sample: ";
for (const auto &val : sample) {
std::cout << val << " ";
}
std::cout << std::endl;
}
return 0;
}The python equivalent is
from pylsl import StreamInlet, resolve_stream
# Resolve the stream
streams = resolve_stream( 'name', 'ExampleDoubleStream', 1, 0)
# Create an inlet
inlet = StreamInlet(streams[0])
# Retrieve the stream info to get the number of channels
info = inlet.info()
num_channels = info.channel_count()
print(f"Number of channels: {num_channels}")
while True:
# Pull a sample from the inlet
sample, timestamp = inlet.pull_sample()
# Print the received sample
print(f"Received sample: {sample}")Multiple data types.
Unfortunately streams appear to be restricted to a single type only. Their docs state:
All data within a stream is required to have the same type (integers, floats, doubles, strings).
This feels restrictive to me, but I imagine it helps with efficiency and multi-language support.
I work with eye trackers (and other data sources) for my day job. So the data streams I normally work with often have more than one type of data. So this restriction is less than ideal. Their solution appears to be to create multiple streams and
For a simple test I’ll pretend I have an eye-tracker that sends:
- The 3d position of a gaze (GazeOriginX, GazeOriginY, and GazeOriginZ) represented by three double values
- the 3d direction of a gaze (GazeDirectionX, GazeDirectionY, and GazeDirectionZ) represented by three double values
- The 2D location where the gaze intersects a computer monitor (IntersectionX, IntersectionY) represented by two double values.
- The numeric index of the Display the Subject is currently looking at (assuming multiple monitors, 1,2 or 3)
- The name of the Display
This gives us 8 doubles, a single int, and a string. Normally I’d pack all of the data into a JSON string or even a binary blob to transmit it, but I can’t do this in LSL. So instead I’ll make 3 streams and push them all with the same LSL timestamp
double timestamp = lsl::local_clock();
lslGazeFloatOutlet.push_sample(lslGazeFloatValues, timestamp);
lslGazeIntOutlet.push_sample(lslGazeIntValues, timestamp);
lslGazeStringOutlet.push_sample(&message, timestamp);The following code creates 3 streams with meta data describing the channles and continously transmit them.
#include <cmath>
#include <iostream>
#include <vector>
#include <thread>
#include <random>
#include <lsl_cpp.h>
int main(int argc, char** argv) {
// Create stream info with custom metadata for channel names
// initialize a vector of strings with channel names
const size_t numFloats = 8;
lsl::stream_info lslGazeFloatInfo("EyeTrackingFloatData", "Gaze", numFloats, LSL_IRREGULAR_RATE, lsl::cf_float32);
std::vector<float> lslGazeFloatValues(numFloats);
// Add channel names to metadata
lsl::xml_element channels = lslGazeFloatInfo.desc().append_child("channels");
channels.append_child("channel").append_child_value("label", "IntersectionX");
channels.append_child("channel").append_child_value("label", "IntersectionY");
channels.append_child("channel").append_child_value("label", "HeadPosX");
channels.append_child("channel").append_child_value("label", "HeadPosY");
channels.append_child("channel").append_child_value("label", "HeadPosZ");
channels.append_child("channel").append_child_value("label", "HeadDirectionX");
channels.append_child("channel").append_child_value("label", "HeadDirectionY");
channels.append_child("channel").append_child_value("label", "HeadDirectionZ");
const size_t numInts = 2;
lsl::stream_info lslGazeIntInfo("EyeTrackingIntData", "Gaze", numInts, LSL_IRREGULAR_RATE, lsl::cf_int64);
std::vector<int64_t> lslGazeIntValues(numInts);
lslGazeIntValues[0] = -1;
lslGazeIntValues[1] = 4;
lsl::xml_element intChannels = lslGazeIntInfo.desc().append_child("channels");
intChannels.append_child("channel").append_child_value("label", "FrameNumber");
intChannels.append_child("channel").append_child_value("label", "IntersectionObjectIndex");
// Create stream info with custom metadata for channel names
lsl::stream_info lslGazeStringInfo("EyeTrackingStringData", "Gaze", 1, lsl::IRREGULAR_RATE, lsl::cf_string);
lsl::xml_element objectNameChannels = lslGazeStringInfo.desc().append_child("channels");
objectNameChannels.append_child("channel").append_child_value("label", "intersectionObjectName");
lsl::stream_outlet lslGazeFloatOutlet(lslGazeFloatInfo);
lsl::stream_outlet lslGazeIntOutlet(lslGazeIntInfo);
lsl::stream_outlet lslGazeStringOutlet(lslGazeStringInfo);
std::random_device rd;
std::mt19937 gen(rd());
std::uniform_real_distribution<> uni(0.0, 1.0);
while (true) {
// fill sample vector with random values.
for( auto &val : lslGazeFloatValues ) {
val = (float)uni(gen);
}
std::string message = "CenterDisplay";
// send all samples with a timestamp
double timestamp = lsl::local_clock();
lslGazeFloatOutlet.push_sample(lslGazeFloatValues, timestamp);
lslGazeIntOutlet.push_sample(lslGazeIntValues, timestamp);
lslGazeStringOutlet.push_sample(&message, timestamp);
// Sleep for a while before sending the next sample
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return 0;
}The equivalent in python looks like this:
import time
import random
from pylsl import StreamInfo, StreamOutlet, local_clock
# Create stream info with custom metadata for channel names
num_floats = 8
lsl_gaze_float_info = StreamInfo('EyeTrackingFloatData', 'Gaze', num_floats, 0, 'float32')
# Add channel names to metadata
channels = lsl_gaze_float_info.desc().append_child('channels')
channels.append_child('channel').append_child_value('label', 'IntersectionX')
channels.append_child('channel').append_child_value('label', 'IntersectionY')
channels.append_child('channel').append_child_value('label', 'HeadPosX')
channels.append_child('channel').append_child_value('label', 'HeadPosY')
channels.append_child('channel').append_child_value('label', 'HeadPosZ')
channels.append_child('channel').append_child_value('label', 'HeadDirectionX')
channels.append_child('channel').append_child_value('label', 'HeadDirectionY')
channels.append_child('channel').append_child_value('label', 'HeadDirectionZ')
num_ints = 2
lsl_gaze_int_info = StreamInfo('EyeTrackingIntData', 'Gaze', num_ints, 0, 'int64')
int_channels = lsl_gaze_int_info.desc().append_child('channels')
int_channels.append_child('channel').append_child_value('label', 'FrameNumber')
int_channels.append_child('channel').append_child_value('label', 'IntersectionObjectIndex')
lsl_gaze_string_info = StreamInfo('EyeTrackingStringData', 'Gaze', 1, 0, 'string')
object_name_channels = lsl_gaze_string_info.desc().append_child('channels')
object_name_channels.append_child('channel').append_child_value('label', 'intersectionObjectName')
# Create stream outlets
lsl_gaze_float_outlet = StreamOutlet(lsl_gaze_float_info)
lsl_gaze_int_outlet = StreamOutlet(lsl_gaze_int_info)
lsl_gaze_string_outlet = StreamOutlet(lsl_gaze_string_info)
lsl_gaze_float_values = [random.random() for _ in range(num_floats)]
lsl_gaze_int_values = [random.randint(0, 1000) for _ in range(num_ints)]
message = "CenterDisplay"
while True:
# Send the samples
lsl_gaze_float_outlet.push_sample(lsl_gaze_float_values)
lsl_gaze_int_outlet.push_sample(lsl_gaze_int_values)
lsl_gaze_string_outlet.push_sample([message])
# Sleep for a while before sending the next sample
time.sleep(1)
Reading Multiple Streams
To read the eye tracking data, I make 3 inlets. One for each outlet stream.
#include <cmath>
#include <iostream>
#include <vector>
#include <thread>
#include <iomanip>
#include <lsl_cpp.h>
int main(int argc, char** argv) {
try {
// Resolve the stream
std::vector<lsl::stream_info> floatResults = lsl::resolve_stream("name", "EyeTrackingFloatData", 1, 5.0);
std::cout << "Found " << floatResults.size() << " float streams" << std::endl;
std::vector<lsl::stream_info> intResults = lsl::resolve_stream("name", "EyeTrackingIntData", 1, 5.0);
std::cout << "Found " << intResults.size() << " int64 streams" << std::endl;
std::vector<lsl::stream_info> stringResults = lsl::resolve_stream("name", "EyeTrackingStringData", 1, 5.0);
std::cout << "Found " << stringResults.size() << " string streams" << std::endl;
if (floatResults.empty() || intResults.empty() || stringResults.empty()) {
std::cerr << "Missing a stream" << std::endl;
return 1;
}
// Create inlets for the streams
lsl::stream_inlet floatInlet(floatResults[0]);
lsl::stream_info floatInfo = floatInlet.info();
std::cout << "The float stream's XML meta-data is: \n" << floatInfo.as_xml();
lsl::stream_inlet intInlet(intResults[0]);
lsl::stream_info intInfo = intInlet.info();
std::cout << "The int64 stream's XML meta-data is: \n" << intInfo.as_xml();
lsl::stream_inlet stringInlet(stringResults[0]);
lsl::stream_info stringInfo = stringInlet.info();
std::cout << "The string stream's XML meta-data is: \n" << stringInfo.as_xml();
std::cout << std::fixed;
std::cout << std::setprecision(7);
while (true) {
// Buffer to hold the received data
std::vector<float> floatSamples(8);
// Pull sample from the inlet
double timestamp = floatInlet.pull_sample(floatSamples.data(), floatSamples.size());
// Print the received data
std::cout << "Received at " << timestamp << ":\n";
for (size_t i = 0; i < floatSamples.size(); ++i) {
std::cout << "Channel " << i << ": " << floatSamples[i] << std::endl;
}
// same for int
std::vector<int64_t> intSamples(2);
timestamp = intInlet.pull_sample(intSamples.data(), intSamples.size());
std::cout << "Received at " << timestamp << ":\n";
for (size_t i = 0; i < intSamples.size(); ++i) {
std::cout << "Channel " << i << ": " << intSamples[i] << std::endl;
}
std::vector < std::string> stringSamples(1);
timestamp = stringInlet.pull_sample(stringSamples.data(), stringSamples.size());
std::cout << "Received at " << timestamp << ":\n";
for (size_t i = 0; i < stringSamples.size(); ++i) {
std::cout << "Channel " << i << ": " << stringSamples[i] << std::endl;
}
// Sleep for a short duration to simulate processing time
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
}
catch (std::exception& e) {
std::cerr << "Error: " << e.what() << std::endl;
return 1;
}
return 0;
}The python equivalent looks like:
from pylsl import StreamInlet, resolve_stream
import time
try:
# Resolve the streams
float_results = resolve_stream('name', 'EyeTrackingFloatData', 1, 5.0)
print(f"Found {len(float_results)} float streams")
int_results = resolve_stream('name', 'EyeTrackingIntData', 1, 5.0)
print(f"Found {len(int_results)} int64 streams")
string_results = resolve_stream('name', 'EyeTrackingStringData', 1, 5.0)
print(f"Found {len(string_results)} string streams")
if not float_results or not int_results or not string_results:
print("Missing a stream")
exit(1)
# Create inlets for the streams
float_inlet = StreamInlet(float_results[0])
float_info = float_inlet.info()
print(f"The float stream's XML meta-data is: \n{float_info.as_xml()}")
int_inlet = StreamInlet(int_results[0])
int_info = int_inlet.info()
print(f"The int64 stream's XML meta-data is: \n{int_info.as_xml()}")
string_inlet = StreamInlet(string_results[0])
string_info = string_inlet.info()
print(f"The string stream's XML meta-data is: \n{string_info.as_xml()}")
while True:
# Buffer to hold the received data
float_samples, timestamp = float_inlet.pull_sample()
print(f"Received at {timestamp}:")
for i, sample in enumerate(float_samples):
print(f"Channel {i}: {sample}")
# Same for int
int_samples, timestamp = int_inlet.pull_sample()
print(f"Received at {timestamp}:")
for i, sample in enumerate(int_samples):
print(f"Channel {i}: {sample}")
# Same for string
string_samples, timestamp = string_inlet.pull_sample()
print(f"Received at {timestamp}:")
for i, sample in enumerate(string_samples):
print(f"Channel {i}: {sample}")
# Sleep for a short duration to simulate processing time
time.sleep(0.01)
except Exception as e:
print(f"Error: {e}")
exit(1)
