Okay, so I've been a bit sad that WebSockets didn't make the transition
over to HTTP/2 because when I needed some realtime data
WebSockets was a great tool to have under your belt. So I've been idly
following whatever progress was made related to speccing
WebSockets over h2, never even having thought about why.
Turns out, there's not much reason to! I've been working on implementing
a reverse proxy in Node, and one of the requirements I had was to
provide a HTTP/2 server frontend. While toying with it, I got to learn
its inner workings better and everything around WS/h2 started to make
sense.
HTTP/2 breaks the HTTP/1 wire compatibility, which some people are not
too happy about, but for a good reason -- its model of interaction is
a superset of h1's.
You can just use a single connection for a single HTTP request, and
receive a single HTTP response. You can also do the equivalent of a
Connection: keep-alive and reuse the same connection for multiple
sequential requests.
However, h2 also allows for a simpler to implement pipelining, which was
technically a part of HTTP/1.1 but rarely implemented,
which means that other hacks for accelerating fetching
multiple resources by doing that in parallel are not that necessary
anymore.
However, it doesn't stop there. The base upon which HTTP/2 allows for
requests with HTTP/1 semantics is the stream. Wherein HTTP/1 had, for
everything you wanted to do, an interlinked pair of request+response,
HTTP/2 only has the stream.
If you take a look at Node's http2 module, you'll discover
that Http2Stream is actually a Duplex stream. With h2 the distinction
between a read-only request and write-only response is gone. No need for
BOSH or the like. Now every request, if it can be called that, is a
two-way lane which carries messages back and forth without any need for
limiting yourself.
This is very close to what WebSockets could do, although I'd even say
that h2 is also a superset of WebSockets in the semantic sense because
it adds HTTP's headers for a preamble of metadata, with which you can
negotiate the content type and the rest. HTTP/2 is already what
WebSockets wanted to be. (Although there's a subtle problem wherein
large payloads might arrive in multiple pieces and it's your
responsibility to reassemble them if your request is long-lived like
that, but that can be solved easily by e.g. using a lightweight prefix
length framing on top of h2's streams.)
Now a bit about h2 Push which everyone is touting as the magical
technology which makes WebSockets (and, indeed, any push protocol)
unnecessary and will replace all of them.
Let me start with this -- the web world seems to have a lot of
misconceptions about what h2 Push actually is. If you take a look at the
spec, you'll notice that the PUSH_PROMISE frame looks an
awful lot like the HEADERS frame which opens a stream (in h1 people call
this a "request"). That's because it is -- an h2 Push is basically a
way to execute synthetic requests (i.e. ones which the client hasn't
made yet), prepackage their response and send it over. That's it.
It's a useful tool for when you need it (like preemptively sending
static assets along with some HTML to be used later), but otherwise it's
nothing more, nothing less. Just a request for the client to use.
I remember reading about some implementation where h2 Push was used to
essentially inject data units into the browser's cache and then push the
URLs over SSE, but that's such overkill. Just use a
long-running h2 stream (SSE, even) and just send the data over that way.
Anyhow. h2 is cool. You should probably use it if you don't already.
It's a shame though that nobody decided to implement it over plaintext.
But, contrary to Dave Winer's opposition, the powers in that
fight seem to have good arguments for putting everything behind TLS.
Some code?
Oh, right.
const h2 = require('http2');
const s = h2.createServer();
s.on('stream', (str, headers) => {
let i = 0, intv;
str.on('close', () => {
if (typeof intv !== 'undefined') {
clearInterval(intv);
}
});
/* Most of HTTP/1 request line metadata and even some of the
headers have been transformed into these "pseudo-headers".
See RFC 7540 §8.1.2.1. */
const rq = headers[':path'];
if (rq === '/echo') {
str.respond({ ':status': 200 });
str.pipe(str);
return;
} else if (rq === '/timer') {
str.respond({ ':status': 200 });
intv = setInterval(() => {
str.write(i++ + '\n');
}, 1000);
return;
} else if (rq === '/ping') {
// To observe the effects of this one, you should use a
// HTTP/2 frame-level dumper, such as nghttp.
str.respond({ ':status': 200 });
intv = setInterval(() => {
str.session.ping(() => { console.log('ponged!'); });
}, 5000);
return;
}
str.respond({ ':status': 404 });
str.end('not found, sorry :(\n');
});
s.listen(() => {
console.log('listening on :%d', s.address().port);
});
Of course, this is just a toy example, but it should be enough for a
demo. I encourage you to explore HTTP/2 Push on your own, but it's fun
nonetheless.
This won't easily work with browsers because, as mentioned previously,
they require HTTP/2 over TLS. This is the so-called "plaintext" variant,
which is not implemented by Firefox, Chrome et al.
You can try it with a self-signed or Let's Encrypt cert though. :)
Errata 2019-06-06
For a while there has been a renewed effort to bringing
WebSockets over to HTTP/2; the HTTP/2 library used as the basis for many other
software has it implemented for almost a year already,
meaning some end software also supports it; some browsers
implement it in an experimental state, and yet others
are still working on it. WebSockets still have some
advantages over raw HTTP/2 (like proper framing and binary/text separation)
so I'm excited to see this RFC come to fruition, even though it's taking a
while for us to get there.
# Random thoughts on HTTP/2
Okay, so I've been a bit sad that [WebSockets didn't make the transition
over to HTTP/2][no-ws-over-h2] because when I needed some realtime data
WebSockets was a great tool to have under your belt. So I've been idly
following [whatever progress][h2ws-draft] was made related to speccing
WebSockets over h2, never even having thought about why.
[no-ws-over-h2]: https://daniel.haxx.se/blog/2016/06/15/no-websockets-over-http2/
[h2ws-draft]: https://github.com/mcmanus/draft-h2ws
Turns out, there's not much reason to! I've been working on implementing
a reverse proxy in Node, and one of the requirements I had was to
provide a HTTP/2 server frontend. While toying with it, I got to learn
its inner workings better and everything around WS/h2 started to make
sense.
HTTP/2 breaks the HTTP/1 wire compatibility, which some people are not
too happy about, but for a good reason -- its model of interaction is
a superset of h1's.
You can just use a single connection for a single HTTP request, and
receive a single HTTP response. You can also do the equivalent of a
`Connection: keep-alive` and reuse the same connection for multiple
sequential requests.
However, h2 also allows for a simpler to implement pipelining, which was
[technically a part of HTTP/1.1][h1-pipelining] but rarely implemented,
which means that [other hacks][h1-sharding] for accelerating fetching
multiple resources by doing that in parallel are not that necessary
anymore.
[h1-pipelining]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Connection_management_in_HTTP_1.x#HTTP_pipelining
[h1-sharding]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Connection_management_in_HTTP_1.x#Domain_sharding
However, it doesn't stop there. The base upon which HTTP/2 allows for
requests with HTTP/1 semantics is the stream. Wherein HTTP/1 had, for
everything you wanted to do, an interlinked pair of request+response,
HTTP/2 only has the stream.
If you take a look at [Node's `http2` module][h2-node], you'll discover
that `Http2Stream` is actually a `Duplex` stream. With h2 the distinction
between a read-only request and write-only response is gone. No need for
[BOSH][] or the like. Now every request, if it can be called that, is a
two-way lane which carries messages back and forth without any need for
limiting yourself.
[h2-node]: https://nodejs.org/dist/v10.6.0/docs/api/http2.html#http2_class_http2stream
[BOSH]: https://xmpp.org/extensions/xep-0124.html
This is very close to what WebSockets could do, although I'd even say
that h2 is also a superset of WebSockets in the semantic sense because
it adds HTTP's headers for a preamble of metadata, with which you can
negotiate the content type and the rest. HTTP/2 is already what
WebSockets wanted to be. (Although there's a subtle problem wherein
large payloads might arrive in multiple pieces and it's your
responsibility to reassemble them if your request is long-lived like
that, but that can be solved easily by e.g. using a lightweight prefix
length framing on top of h2's streams.)
Now a bit about h2 Push which everyone is touting as the magical
technology which makes WebSockets (and, indeed, any push protocol)
unnecessary and will replace all of them.
Let me start with this -- the web world seems to have a lot of
misconceptions about what h2 Push actually is. If you take a look at the
[spec][h2-spec], you'll notice that the PUSH\_PROMISE frame looks an
awful lot like the HEADERS frame which opens a stream (in h1 people call
this a _"request"_). That's because it is -- an h2 Push is basically a
way to execute synthetic requests (i.e. ones which the client hasn't
made yet), prepackage their response and send it over. **That's it.**
It's a useful tool for when you need it (like preemptively sending
static assets along with some HTML to be used later), but otherwise it's
nothing more, nothing less. Just a request for the client to use.
[h2-spec]: https://httpwg.org/specs/rfc7540.html
I remember reading about some implementation where h2 Push was used to
essentially inject data units into the browser's cache and then push the
URLs over [SSE][], but that's such overkill. Just use a
long-running h2 stream (SSE, even) and just send the data over that way.
[SSE]: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
Anyhow. h2 is cool. You should probably use it if you don't already.
It's a shame though that nobody decided to implement it over plaintext.
But, contrary to [Dave Winer's opposition][dw-https], the powers in that
fight seem to have good arguments for putting everything behind TLS.
[dw-https]: http://this.how/googleAndHttp/
## Some code?
Oh, right.
```js
const h2 = require('http2');
const s = h2.createServer();
s.on('stream', (str, headers) => {
let i = 0, intv;
str.on('close', () => {
if (typeof intv !== 'undefined') {
clearInterval(intv);
}
});
/* Most of HTTP/1 request line metadata and even some of the
headers have been transformed into these "pseudo-headers".
See RFC 7540 §8.1.2.1. */
const rq = headers[':path'];
if (rq === '/echo') {
str.respond({ ':status': 200 });
str.pipe(str);
return;
} else if (rq === '/timer') {
str.respond({ ':status': 200 });
intv = setInterval(() => {
str.write(i++ + '\n');
}, 1000);
return;
} else if (rq === '/ping') {
// To observe the effects of this one, you should use a
// HTTP/2 frame-level dumper, such as nghttp.
str.respond({ ':status': 200 });
intv = setInterval(() => {
str.session.ping(() => { console.log('ponged!'); });
}, 5000);
return;
}
str.respond({ ':status': 404 });
str.end('not found, sorry :(\n');
});
s.listen(() => {
console.log('listening on :%d', s.address().port);
});
```
Of course, this is just a toy example, but it should be enough for a
demo. I encourage you to explore HTTP/2 Push on your own, but it's fun
nonetheless.
For requests:
```sh
$ nghttp -v "http://localhost:12345/ping"
$ nghttp -v "http://localhost:12345/timer"
$ # etc...
```
This won't easily work with browsers because, as mentioned previously,
they require HTTP/2 over TLS. This is the so-called "plaintext" variant,
which is not implemented by Firefox, Chrome et al.
You can try it with a self-signed or [Let's Encrypt][] cert though. :)
[Let's Encrypt]: https://letsencrypt.org
## Errata 2019-06-06
For a while there has been a renewed [effort][rfc8441] to bringing
WebSockets over to HTTP/2; the HTTP/2 library used as the basis for many other
software [has it implemented][nghttp2-v1.34] for almost a year already,
meaning [some end software][node-v10.12] also supports it; some browsers
[implement it][chromestatus-rfc8441] in an experimental state, and yet others
[are still working on it][firefox-bug1434137]. WebSockets still have some
advantages over raw HTTP/2 (like proper framing and binary/text separation)
so I'm excited to see this RFC come to fruition, even though it's taking a
while for us to get there.
[rfc8441]: https://tools.ietf.org/html/rfc8441
[nghttp2-v1.34]: https://nghttp2.org/blog/2018/10/04/nghttp2-v1-34-0/
[node-v10.12]: https://nodejs.org/en/blog/release/v10.12.0/
[chromestatus-rfc8441]: https://www.chromestatus.com/feature/6251293127475200
[firefox-bug1434137]: https://bugzilla.mozilla.org/show_bug.cgi?id=1434137