Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-full.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
#### 1. Setup: ####
###################

- uses: actions/checkout@v4
- uses: actions/checkout@v6

- name: Prepare for deployment at push
if: github.event_name == 'push'
Expand Down
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: RstoxData
Version: 2.2.1-9006
Date: 2026-06-06
Version: 2.2.1-9007
Date: 2026-06-13
Title: Tools to Read and Manipulate Fisheries Data
Authors@R: c(
person(given = "Arne Johannes",
Expand Down
7 changes: 6 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# RstoxData v2.2.1-9005 (2026-05-21)
# RstoxData v2.2.1-9007 (2026-06-13)
* Added support for reading zipped ICESAcoustic XML files (in getIcesVocabulary()).
* Removed warning occurring when translation table and variable is coerced to numeric in matchVariable().


# RstoxData v2.2.1-9006 (2026-06-06)
* Fixed bug in Translate functions where translating numeric variables did not work for large values that are represented by scientific notation in R. As an example, 200000000 is represented as 2E+08, and after the fix, both 200000000 and 2E+08 are valid values to translate from.


Expand Down
15 changes: 10 additions & 5 deletions R/DefineAndUpdateVariables.R
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ RedefineStoxBiotic <- function(
#' @details The columns of the \code{TranslationTable} can be given in one of two ways: (1) A single value or a string to be evaluated and matched using the "\%in\%" operator, such as "HER" or "c(\"HER\", \"CLU\")"; or (2) a string expressing a function of the variable given by the column name, such as "function(IndividualTotalLength) IndividualTotalLength > 10".
#'
#' Specifying NewValue as a function can be used to transform numeric values, e.g. "function(IndividualTotalLength) IndividualTotalLength * 1.1" to compensate for different length measurement. This can be useful for length that are not total length, in which case TranslateBiotic can be used to translate the lengthmeasurement (NMDBiotic) or LengthType (ICESBiotic) to the accepted "E" or "1", respectively. For BioticData read from ICESBiotic XML files the the LengthType specifies the type of length measurement. These values are not translated using the vocabulary from the XML file, so that total length is represented as "AC_LengthMeasurementType_1" instead of the code "1". For these data the translation can be either to "AC_LengthMeasurementType_1" or to "1".
#'
#' Note that a function string for NewValue must be of the form function(VAR), where VAR is the name of the variable to translate. The function body can however contain other variables. E.g., "function(IndividualTotalLength) 5 * IndividualRoundWeight^(1/3)", will set the IndividualTotalLength 5 times the square root of IndividualTotalLength.
#'
#' Similar to transforming IndividualTotalLength, IndividualRoundWeight can also be transformed if the individualproducttype is not 1 for NMDBiotic XML files.
#'
Expand Down Expand Up @@ -406,7 +408,8 @@ translateOne <- function(
nomatches <- sapply(matches, function(x) length(x) == 1 && is.na(x))

# Translate only if there were ane matches:
if(!any(nomatches)) {
# Here, na.rm = TRUE makes sure we get TRUE/FALSE and not NA:
if(!any(nomatches, na.rm = TRUE)) {
# Remove empty matches:
matches <- matches[sapply(matches, NROW) > 0]

Expand All @@ -422,7 +425,7 @@ translateOne <- function(

if(length(thisMatches)) {

# Kepp pnly the columns present in the table to translate, so that we can use thisMatches to dientify the rows in which to translate:
# Keep only the columns present in the table to translate, so that we can use thisMatches to identify the rows in which to translate:
thisMatches <- subset(thisMatches, select = intersect(names(thisMatches), names(data[[tableName]])))

# Translate
Expand Down Expand Up @@ -458,7 +461,8 @@ translateOne <- function(

thisMatches <- matchVariable(variableToTranslate, translationListOne, data[[tableName]])

if(any(thisMatches)) {
# Here, na.rm = TRUE makes sure we get TRUE/FALSE and not NA:
if(any(thisMatches, na.rm = TRUE)) {
# Translate
if(isFunctionString(translationListOne$NewValue, variableToTranslate)) {
#data[[tableName]] [thisMatches, eval(variableToTranslate) := eval(parse(text = translationListOne$NewValue)) (get(variableToTranslate)), on = names(thisMatches)]
Expand Down Expand Up @@ -643,6 +647,7 @@ matchClass <- function(A, B) {

# Function to match a condition given in the element 'variableName' of 'list' to the corresponding element in 'table', either as a function expression string that can be evaluated, or a replacement string:
matchVariable <- function(variableName, list, table) {

# Special case for NA:
if(is.na(list[[variableName]]) || identical(list[[variableName]], "NA")) {
is.na(table[[variableName]])
Expand All @@ -664,8 +669,8 @@ matchVariable <- function(variableName, list, table) {
vector <- eval(parse(text = deparse(list[[variableName]])))
if(!is.list(vector) && is.vector(vector) && length(dim(vector)) < 2) {
# If both the translation value and the data value is coercible to numeric, treat as numeric:
xnum <- as.numeric(table[[variableName]])
ynum <- as.numeric(vector)
suppressWarnings(xnum <- as.numeric(table[[variableName]]))
suppressWarnings(ynum <- as.numeric(vector))
if(!any(is.na(xnum)) && !any(is.na(ynum))) {
xnum %in% ynum
}
Expand Down
2 changes: 1 addition & 1 deletion R/Filter.R
Original file line number Diff line number Diff line change
Expand Up @@ -511,7 +511,7 @@ expandFilterExpressionList <- function(FilterExpressionList, sep = "/") {
#' Filters \code{\link{BioticData}}.
#'
#' @param BioticData Input \code{\link{BioticData}} data.
#' @param FilterExpression Filter expression given as a list of strings. The name of the list and structures should be identical to the names of the input data list. To extract or exclude missing values (NAs) use the operator \code{\%in\%} or the special operator \code{\%notin\%}, which is defined in RstoxData. For a numeric filter, in the current StoX GUI, NA must be added manually in the text field of the "Expression table dialog" (the "Query builder dialog" will only show numeric values and will show an empty fields if the filter contains NA). Possible values are provided in the filter expression builder in the StoX GUI for character values, integer values, and numeric values that are all whole numbers, but not for keys (CruiseKey, StationKey, HaulKey, SpeciesCategoryKey, SampleKey and IndividualKey in \code{\link{FilterStoxBiotic}}, and CruiseKey, LogKey, BeamKey, AcousticCategoryKey, ChannelReferenceKey and NASCKey in \code{\link{FilterStoxAcoustic}}).
#' @param FilterExpression Filter expression given as a list of strings. The name of the list and structures should be identical to the names of the input data list. To extract or exclude missing values (NAs) use the operator \code{\%in\%} or the special operator \code{\%notin\%}, which is defined in RstoxData. For a numeric filter, in the current StoX GUI, NA must be added manually in the text field of the "Expression table dialog" (the "Query builder dialog" will only show numeric values and will show an empty fields if the filter contains NA). Note that there is a distinction between NA and NaN for numeric variables. To overcome this distinction, use \code{is.na} instead of \code{\%in\%} NA. In the current GUI \code{is.na} must be typed into the parameter field. Possible values are provided in the filter expression builder in the StoX GUI for character values, integer values, and numeric values that are all whole numbers, but not for keys (CruiseKey, StationKey, HaulKey, SpeciesCategoryKey, SampleKey and IndividualKey in \code{\link{FilterStoxBiotic}}, and CruiseKey, LogKey, BeamKey, AcousticCategoryKey, ChannelReferenceKey and NASCKey in \code{\link{FilterStoxAcoustic}}).
#' @param FilterUpwards Whether the filter action will propagate in the upwards direction. Default to FALSE. Use this option with caution, particularly for swept-area survey estimates, where setting \code{FilterUpwards} to TRUE could affect the estimated mean density.
#'
#' @return An object of filtered data in the same format as the input data.
Expand Down
29 changes: 25 additions & 4 deletions R/xsdUtils.R
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,9 @@ readCharZip <- function(x, ...) {
if(length(file) > 1) {
stop("Input data can be zipped, but then each file must be zipped individually, so that each zipfile contains only one file (and this file must have the same name as the zip, excluding file extension).")
}
output <- readChar(unz(x, file), ...)
con <- unz(x, file)
output <- readChar(con, ...)
close(con)
}
else {
output <- readChar(x, ...)
Expand Down Expand Up @@ -408,12 +410,31 @@ checkFileNameInZip <- function(x) {
getIcesVocabulary <- function(xmlFile) {

# Read only first 10e4 character
tmpText <- readChar(xmlFile, 10e4)
nchars <- 10e4
if(tools::file_ext(xmlFile) == "zip") {
tmpText <- readCharZip(xmlFile, nchars = nchars)
}
else {
tmpText <- readChar(xmlFile, nchars = nchars)
}

xmlObj <- read_html(tmpText)

# Apply transformation to get the vocabulary translation table
ret <- rbindlist(lapply(as_list(xml2::xml_find_all(xmlObj, "//vocabulary/*/code")), function(x) if(length(x)>0) return(list(attr(x, "id"), ifelse(length(x) > 0, unlist(x), NA), attr(x, "codetype")))))
setnames(ret, c("id", "value", "codetype"))
ret <- rbindlist(
lapply(
as_list(
xml2::xml_find_all(
xmlObj,
"//vocabulary/*/code"
)
),
#function(x) if(length(x)>0) return(list(attr(x, "id"), ifelse(length(x) > 0, unlist(x), NA), attr(x, "codetype")))
function(x) if(length(x)>0) return(list(attr(x, "id"), ifelse(length(x) > 0, unlist(x), NA)))
)
)
#setnames(ret, c("id", "value", "codetype"))
setnames(ret, c("id", "value"))
return(ret)
}

Expand Down
Binary file modified inst/extdata/functionArguments.rds
Binary file not shown.
Binary file added inst/testresources/ICES_Acoustic_1.zip
Binary file not shown.
6 changes: 6 additions & 0 deletions inst/tinytest/test-read.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
zippedICESAcousticXMLFile <- system.file("testresources","ICES_Acoustic_1.zip", package = "RstoxData")

a <- RstoxData::ReadAcoustic(zippedICESAcousticXMLFile)
expect_true(a$ICES_Acoustic_1.zip$Data[3, Value] == 46.45)


2 changes: 2 additions & 0 deletions man/DefineTranslation.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/FilterAcoustic.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/FilterBiotic.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion man/FilterICESAcoustic.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading