Support speed and NMEA heading from Simrad .raw files

[gavinmacaulay]

This PR addresses issue #1346

• Add speed over ground to the EchoData object (obtained from NMEA RMC and VTG messages),
• Add heading data to the EchoData object for EK60 files and EK80 files (if heading is not available from MRU datagrams). Obtained from NMEA HDT messages,
• Add tests for the new code. The test for EK80 heading from HDT uses a .raw file (D20260613-T230914.zip) that needs to be added to the test assets in an ek80/heading directory and is used by test_convert_ek80::test_parse_NMEA_heading .

Notes:

• An existing comment in convert/set_groups_base.py for a function this PR modifies says "move this to be part of parser as it is not a 'set' operation". This has not been done as it would muddy the changes in this PR,
• The new speed over ground and heading variables have their own time coordinates,
• The Provide vessel speed over ground when available #1346 issue was only about speed, but while adding that I noticed that heading was missing for raw files without MRU datagrams so included that.

[gavinmacaulay]

The unit tests look to all be failing due to not finding the extra .raw file that needs to be added to the test assets.

[LOCEANlloydizard]

Hi @gavinmacaulay, thx for the PR! i'll had the data first thing tomorrow and will then re-trigger the set of test! cheers!

[gavinmacaulay]

@LOCEANlloydizard - just pushed a new set of changes that should address most of the test failures not related to that new data file

[LOCEANlloydizard]

@gavinmacaulay i've added the new test data file as a separate release asset (ek80_heading.zip) in #1704
I did it this way because ek80.zip is already quite large (~1 GB), so I thought it would be cleaner to avoid rebuilding and re-uploading it every time we need to add a new test file..

So the PR needs small changes:

• add an ek80_heading_path fixture in the EK80 test file

@pytest.fixture
def ek80_heading_path(test_path):
    return test_path["EK80_HEADING"]

• update test_parse_NMEA_heading to use ek80_heading_path

raw_file=ek80_heading_path / "heading" / "D20260613-T230914.raw"

let me know if something doesnt work!

[LOCEANlloydizard]

@gavinmacaulay I've merged #1704 into main, so you can(/need) pull the latest changes! cheers!

[gavinmacaulay]

@LOCEANlloydizard Thanks for adding the extra raw file. Tests under Windows now all pass but those under Ubuntu don't.

It looks to be because when ek80_heading.zip is unzipped the raw file ends up in ek80_heading\heading under Windows (as expected) but in ek80/heading under Ubuntu (not expected) and the test fails because it can't find the raw file.

I suspect (am not 100% sure) that the directory structure in the .zip file needs to be ek80_heading/heading/D...raw instead of ek80/heading/D...raw.

[gavinmacaulay]

Similar problem still:) On windows the raw file ends up in ek80_heading/ but on ubuntu it is in ek80_heading/heading/

[LOCEANlloydizard]

@gavinmacaulay i've updated the ZIP file to match the actual test path (ek80_heading_path / "D20260613-T230914.raw")! Yes, I ran into this issue before between Windows and Ubuntu... and I think the flattening function may need a bit more investigation. In any case, it passes CI now!

By the way, is this PR ready for review, or is it still a draft? Thanks!

[gavinmacaulay]

@LOCEANlloydizard Ready for review now (was just being careful and waiting for all checks to properly pass).

[LOCEANlloydizard]

did not review the code, just putting this as a note for later: how are duplicate timestamps handled for these new variables? e.g. #1627

[gavinmacaulay]

Duplicate timestamps are removed in set_groups_base.py:_extract_selected_NMEA() (lines 227-233). This applies to the existing lat/lon parsing as well as the added heading and speed parsing. The first of any duplicated timestamps are removed.

I've just now noticed #1478 (comment) and note that removing duplicate timestamps for lat/lon resolves that problem in a different, perhaps unwanted, way. And a similar problem may occur with the speed, which currently reads from RMC and VTG NMEA messages.