Skip to content

Compress TSV exports#1314

Merged
labkey-tchad merged 4 commits intorelease26.3-SNAPSHOTfrom
26.3_fb_tsvCompression
Mar 26, 2026
Merged

Compress TSV exports#1314
labkey-tchad merged 4 commits intorelease26.3-SNAPSHOTfrom
26.3_fb_tsvCompression

Conversation

@labkey-jeckels
Copy link
Copy Markdown
Contributor

@labkey-jeckels labkey-jeckels commented Mar 20, 2026

Rationale

Spring Boot compresses many text HTTP responses by default. But not TSV.

Changes

  • Add text/tab-separated-values to the default set of compressed MIME types

put("server.compression.enabled", "true");
// Spring Boot compresses HTML, JSON and other types by default, but not TSV. We have to duplicate the
// defaults and add text/tab-separated-values
put("server.compression.mime-types", "text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml,text/tab-separated-values");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see one use of text/json in our code, but I don' think we're using that in responses.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think text/json is a valid mime-type. It should only be application/json according to RFC8259.

@labkey-tchad
Copy link
Copy Markdown
Member

labkey-tchad commented Mar 24, 2026

TSVs are compressed. What about other common mime types?

  • Excel: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
  • CSV: text/csv
  • SVG: image/svg+xml (any large svgs are generated in the browser)
  • PNG: image/png (already comressed)
  • Truetype fonts: font/ttf (Roboto and Titillium add up to about 450KB)

@labkey-jeckels
Copy link
Copy Markdown
Contributor Author

TSVs are compressed. What about other common mime types?

  • Excel: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
  • CSV: text/csv
  • SVG: image/svg+xml (any large svgs are generated in the browser)
  • PNG: image/png (already comressed)
  • Truetype fonts: font/ttf (Roboto and Titillium add up to about 450KB)

I went with CSV and SVG, as they're text formats that will compress well. TTF is less compelling given their static nature, but I added it.

Excel and PNG are already compressed

@labkey-tchad
Copy link
Copy Markdown
Member

CSVs are compressed but none of the SVGs or TTF files are.
Also, no webdav downloads seem to be compressed. Should they be?

@labkey-jeckels
Copy link
Copy Markdown
Contributor Author

CSVs are compressed but none of the SVGs or TTF files are. Also, no webdav downloads seem to be compressed. Should they be?

When serving static files, whether they're part of a module or not, the request is served by DavController. It uses Tomcat's sendfile "API" to send them, which is more efficient for I/O than Stream-based responses but doesn't support compressing. Therefore, for (most) compressible static files, the build pre-creates a .gz version and we serve that instead.

https://github.com/LabKey/platform/blob/7d500be3cfb1119eac1ac77d5bdadc07c7bf6756/core/src/org/labkey/core/webdav/DavController.java#L5004

Dynamically generated content like TSV exports go through a Stream and therefore get compressed per the settings in this change.

I was on the fence about including TTF in the compression list, and just reverted them. They're cached so it shouldn't be much of a difference in real-world perf.

I propose no other changes at this point. The main concern was compressing large TSV exports, which this change accomplishes.

@labkey-matthewb

@labkey-tchad labkey-tchad merged commit 2b7f866 into release26.3-SNAPSHOT Mar 26, 2026
6 of 8 checks passed
@labkey-tchad labkey-tchad deleted the 26.3_fb_tsvCompression branch March 26, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants