Add HttpResponse.writeHeaders

[BV-WebDev]

Woud it be possible to get a writeHeaders function?
Signature could be [[key: string, value: string]]
So we only need one function/c++ call to write all headers.

[Guthib-of-Dan]

Hello. uWS doesn’t really force you to write one header per function call. You can use underlying CRLF string concatenation to make several requests fit into one call. #1217 here I describe my library, and in one or two days I am going to roll out @ublitzjs/core 1.1.0. There you would be able to write several headers at once, without sacrificing typescript safety.

res.writeHeader(“Content-Type: text/plain\r\nAccept-Ranges”, “bytes”)

I have to point out, that on my pc at some benchmark, where I wrote 100 headers with 100 function calls vs 100 concatenated in one call, speed raised from 130000 requests per aecond to 280000 requests per second

[BV-WebDev]

Oh ok thank you, I have to test that. So basically we loose ~1% perf per set header.
Typical website may use 6 headers, maybe more with CORS.
Would be better if it was integrated in uWS than using tricks tho.

[uNetworkingAB]

This is a contended idea because it sacrifices/misdirects the fast path for a (potentially?) less slow slow path:

1. writeHeader is (unless I forgot a detail) ideal candidate for V8 fastcalls. That function does not generate or call any JS code, so it could be a fastcall. However, Node.js people think they know better than us so they hold that feature hostage and censored (we can forcefully use it by shipping the header ourselves if absolutely critical, though) <- this has already been done before, as a test. It can be done.

2. uWebSockets.js is a wrapper of uWebSockets. 99% is a 1-to-1 wrap job. The fundamental design idea of uWS is that functions like writeHeader be just that; simple functions that generate no memory allocations whatsoever. They are entirely inlined as well. For C++ (and thus for the maximally performant path) this is the prime design. Changing that design by incorporating ideas from a GC:d script language and trying to satisfy a 1% loss due to a call is moving the design in the wrong direction.

Before 2018 the library was (pretty much) 2 different libraries (not really); one for C++ and one for JS. They had completely different interfaces and could not be tested as a single library. With this design of being a 1-to-1 wrapper, what is broken for JS is broken for C++ so the stability of both libraries are kept in sync.

Paying 1% for writeHeader call is basically a wrapping tax. You could say: "why don't we completely reimagine the library with a completely different memory model and make everything copy into a shared array buffer and invent our of calling convention using basically a ringbuffer of OP-codes that execute when you call the fastcall "process" function and only calls into JS once per event loop iteration?

Well, yes you can, but that would be to fundamentally rewrite the library from scratch (and all of its internal zero-copy/zero-allocation optimizations) and it kind of reminds me more of Deno design than uWS. I have tested this design and it can improve perf in Node.js by maybe 30% in best cases (like hello world benchmarks) but it introduces a lot of drawbacks and uses a lot more memory and allocations and a lot more copying which can only really work for small messages.

I actually don't remember the exact thought I had, but I know I have tested this before and yes, it is possible to make this faster like this, but then you basically get a hyper-optimized, hyper-incompatible, frowned upon hack that is so beyond in performance what other parts of Node.js is that you basically will get a 50% performance drop by just looking at anything other than uWS in Node.js so it's not really worth sacrificing this 1-to-1 wrapper simplicity/compatibility/commonality/testability/stability for this new thing that never will see any practical differences when you actually build an app in Node.js.

And let me remind you again; none of this will make the C++ library any faster. So the fast path is not getting any faster.

Does this make any sense?

[uasan]

Does this make any sense?

Many servers unrelated to JS use a similar method because it makes sense: the source headers are already merged. This is common with cached responses or when your data source is another server that you need to proxy.

P.S. Glad you're back online and working on server development.

[uNetworkingAB]

I tested at least 3 different ideas before:

1. the DeclarativeResponse -> it basically registers a static array of OP-codes and executes them in C++. This boosted perf. for hello world by maybe 70% because there is no JS involved (this was kept)
2. I also tested Cached responses where a middle layer just responds with a cache for the same URL. This was basically as performant as DeclarativeResponse and kind of is the better solution but it was reverted to reduce complexity and DeclarativeResponse filled the role in simpler ways. Cached responses could return as a paid feature add-on (think open-core model) or something like that.
3. I also tested the idea that the C++ library runs all handlers internally, builds an array of OP-codes and data (by copying to a preallocated buffer), then calls into JS once (per event-loop iteration) and then all sockets run all their handlers by consuming that given buffer, and they in turn build their responses into a "response" buffer in similar ways (with OP-codes and data). Then the C++ code takes that response and executes it for all sockets. This was the design that improved perf. by maybe 30%. It was not as good as the Cached response or DeclarativeResponse but it was close. This design has a shit ton of complexity though, and will be highly problematic if you are sending back large chunks of data (you would need to allocate separate buffers and refer to them in the main one - creating garbage collection issues and so on). Also, it would need an entire clone of the JS interface to use this OP-code kind of solution. It's also highly incompatible with asynchooks and all that since all "async operations" stem from the same C++ call. And even if you managed to build this, you would have an architecture that is so foreign to Node.js and so hyper-optimized compared to everything else in Node.js that it would be simply alien and tank in perf. as soon as you made a single call to anything else.

[uNetworkingAB]

Does this make any sense?

Many servers unrelated to JS use a similar method because it makes sense: the source headers are already merged. This is common with cached responses or when your data source is another server that you need to proxy.

P.S. Glad you're back online and working on server development.

Yes but then you would have an interface like:

block_of_preformatted_headers = uWS.PrepareHeaders(array of headers) // slow path
response.writePreparedHeaderBlock(block_of_preformatted_headers) // fast path

you wouldn't have a fast path interface that takes a dynamically allocated hash map of headers

[uNetworkingAB]

Well anyways, yes you could add a JS function in uws.js that uses the trick with response.writeHeader to write many headers.

I guess it would be something like uWS.writeHeaders(res, [headers])

[uNetworkingAB]

We can let AI add writeHeaders as a test and you can benchmark it, I already added onFullData so I already broke my own rule here. Let's add it as a test only in the wrapper (it can be improved later on)

[uNetworkingAB]

Okay I added it, now you have both onFullData and writeHeaders. Let's benchmark this and compare!

[uNetworkingAB]

I benchmarked the writeHeaders and it absolutely sucks

Two calls to writeHeader:
Req/sec: 130958.500000
Req/sec: 132026.250000

One call to writeHeaders:
Req/sec: 67402.750000
Req/sec: 68474.000000

[uNetworkingAB]

I have reverted this change

[Guthib-of-Dan]

So here I am again. Even though in runtime using CRLF trick is unsafe (but faster than writeHeaders or singular writeHeader), “preparing” headers can be done easily. @ublitzjs/core 1.1.0 has rolled out, as well as its “staticHeaders” function.
https://github.com/ublitzjs/core/blob/main/USAGE.md#headers

[BV-WebDev]

I tested using CRLF trick and a js helper function writeHeaders.
Test code:

import uWS from "./uwsmaster/src/uws.js";

const port = 9001;

const headerList = [];
for (let i = 0; i < 20; i ++) {
    headerList.push(["Referrer-Policy", "strict-origin"]);
}

const writeHeaders = (res, headers) => {
    if (!headers || !headers.length) return;

    let stringHeaders = "";
    const lastIndex = headers.length - 1;
    for (let i = 0; i < lastIndex; i ++) {
        stringHeaders += headers[i][0] +": "+ headers[i][1] +"\r\n";
    }

    res.writeHeader(stringHeaders + headers[lastIndex][0], headers[lastIndex][1]);
};

const app = uWS./*SSL*/App({
    key_file_name: 'misc/key.pem',
    cert_file_name: 'misc/cert.pem',
    passphrase: '1234'
}).get('/multiple', (res, req) => {
    for (const [key, value] of headerList) {
        res.writeHeader(key, value);
    }
    res.end('OK');

}).get('/single', (res, req) => {
    writeHeaders(res, headerList);
    res.end('OK');

}).listen(port, (token) => {
    if (token) {
        console.log('Listening to port ' + port);
    } else {
        console.log('Failed to listen to port ' + port);
    }
});

Results:
Base 0 header: 180K Req/Sec

Test 5 headers:
Multiple call String: 155K Req/Sec
Multiple call Binary: 158K Req/Sec +2%
Single writeHeaders: 158K Req/Sec +2%

Test 10 headers:
Multiple call String: 132K Req/Sec
Multiple call Binary: 145K Req/Sec +10%
Single writeHeaders: 145K Req/Sec +10%

Test 20 headers:
Multiple call String: 95K Req/Sec
Multiple call Binary: 118K Req/Sec +24%
Single writeHeaders: 126K Req/Sec +32%

Sending multiple strings costs a lot, and that's where we lose the most performance.
The writeHeaders function works well, but only supports strings.

I don't know if this helper function should be added or not to the uws.js file.
But it is definitely something someone wrapping around uWS may want to use.

Alternatively, we could add a writeRawHeaders(RecognizedString) function, it would allow for caching and optimizations.
res.writeRawHeaders("Referrer-Policy: strict-origin\r\nX-Content-Type-Options: nosniff");

[uNetworkingAB]

Yes JavaScript strings are extremely inefficient and void all possible optimizations for efficiently handling data. ArrayBuffers everywhere would allow V8 fastcalls basically... everywhere. V8 fastcalls can take POD data like ArrayBuffers, integers, floats but cannot deal with complex objects like String. This is a longstanding friction with how JS works - everything uses Strings which are notoriously undefined in its memory layout and can be anything from contiguous 1 byte arrays to non-contiguous ropes of madness. It makes it impossible to optimize anything that carries data.