Skip to content

GPS time correlation on log file parsing#71

Draft
LelsersLasers wants to merge 20 commits into
masterfrom
millan/parse_timestamp_correlation
Draft

GPS time correlation on log file parsing#71
LelsersLasers wants to merge 20 commits into
masterfrom
millan/parse_timestamp_correlation

Conversation

@LelsersLasers
Copy link
Copy Markdown
Member

  • After splitting into chunks, look for all GPS messages
  • Linear regression to find correlation between the DAQ timestamp and the time contained in the GPS message
  • Added column to the parsed CSV with the "real" (GPS correlated time)
  • Log files with valid correlated times are named out_###_TimeOfFirstMessage

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds GPS-based time correlation to DAQ log parsing so parsed CSV outputs can include a “real” timestamp derived from GPS time messages, and correlated chunks can be named with the time of the first message.

Changes:

  • Introduces a daq_log_parse::correlate module that extracts gps_time messages per chunk and fits a linear regression between DAQ ms and Unix ms.
  • Updates the log parsing UI flow to run correlation after chunking and pass correlated chunks into CSV generation.
  • Extends CSV output to include a correlated time column and optionally suffixes output filenames with the first correlated timestamp.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
src/ui/log_parser.rs Runs chunk time-correlation before writing CSV tables.
src/daq_log_parse/table.rs Accepts correlated chunks, adds correlated time column, and names output files with correlated first time when available.
src/daq_log_parse/mod.rs Exposes the new correlate module.
src/daq_log_parse/correlate.rs Implements GPS timestamp extraction and linear regression correlation per chunk.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/daq_log_parse/correlate.rs Outdated
Comment thread src/daq_log_parse/correlate.rs
Comment thread src/daq_log_parse/correlate.rs Outdated
Comment thread src/daq_log_parse/table.rs Outdated
Comment thread src/daq_log_parse/table.rs
@LelsersLasers
Copy link
Copy Markdown
Member Author

Currently gives weird results:
GPS correlation successful: slope=1.565244263, intercept_ms=1778437407030.705, rms_error_ms=771.89, points=144

DAQ ticks are 1 ms so the slope should be exactly 1 and the RMS is very high, but from debug printing the algorithm is working correctly, it is the data itself that is noisy

Noisy/inccorect GPS:
Figure_1
Figure_2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants