Skip to content

Fix for non-ascii characters in text annotations of open-ephys files #1827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MarinManuel wants to merge 6 commits into
base: master
Choose a base branch
from
+20 −8
Open

Fix for non-ascii characters in text annotations of open-ephys files #1827

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
675cfa7
added utf8 decoding for strings if first attemp fails
MarinManuel Mar 23, 2026
cb1db66
added utf8 decoding for strings if first attemp fails
MarinManuel Mar 23, 2026
3f457af
Merge remote-tracking branch 'origin/master'
MarinManuel Mar 23, 2026
3f194f8
replaced try/except with logic test
MarinManuel Apr 1, 2026
e89119c
replaced try/except with logic test
MarinManuel Apr 1, 2026
89043a5
Merge remote-tracking branch 'origin/master'
MarinManuel Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 20 additions & 8 deletions neo/rawio/openephysbinaryrawio.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -391,30 +391,42 @@ def _parse_header(self):
# check that events have timestamps
assert "timestamps" in info, "Event stream does not have timestamps!"
# Updates for OpenEphys v0.6:
# In new vesion (>=0.6) timestamps.npy is now called sample_numbers.npy
# In new version (>=0.6) timestamps.npy is now called sample_numbers.npy
# The timestamps are already in seconds, so that event times don't require scaling
# see https://open-ephys.github.io/gui-docs/User-Manual/Recording-data/Binary-format.html#events
if "sample_numbers" in info:
self._use_direct_evt_timestamps = True
else:
self._use_direct_evt_timestamps = False

# for event the neo "label" will change depending the nature
# for event the neo "label" will change depending on the nature
# of event (ttl, text, binary)
# and this is transform into unicode
# all theses data are put in event array annotations
# and this is transform into Unicode
# all these data are put in event array annotations
if "text" in info:
# text case
info["labels"] = info["text"].astype("U")
if info["text"].dtype.kind in ["U", "S"]:
info["labels"] = np.array([e.decode('utf-8') for e in info['text']], dtype='U')
else:
info["labels"] = info["text"].astype("U")
elif "metadata" in info:
# binary case
info["labels"] = info["channels"].astype("U")
Copy link
Copy Markdown
Contributor

@zm711 zm711 Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a previous typo on our part? Why are we looking at channels here if we are checking for metadata? I'm not super familiar with this format so would love to know what is going on here? I'm just brainstorm if there is a simpler way to make this work and noticed this pattern that seems off?

if info["metadata"].dtype.kind in ["U", "S"]:
info["labels"] = np.array([e.decode('utf-8') for e in info['channels']], dtype='U')
else:
info["labels"] = info["channels"].astype("U")
elif "channels" in info:
# ttl case use channels
info["labels"] = info["channels"].astype("U")
if info["channels"].dtype.kind in ["U", "S"]:
info["labels"] = np.array([e.decode('utf-8') for e in info['channels']], dtype='U')
else:
info["labels"] = info["channels"].astype("U")
elif "states" in info:
# ttl case use states
info["labels"] = info["states"].astype("U")
if info["states"].dtype.kind in ["U", "S"]:
info["labels"] = np.array([e.decode('utf-8') for e in info['states']], dtype='U')
else:
info["labels"] = info["states"].astype("U")
else:
raise ValueError(f"There is no possible labels for this event!")

Expand Down
Loading