/ technology proposal

HTTPS and caching, a proposal

HTTPS is first-and-foremost end-to-end encryption. 99% of the use cases however, are leveraging HTTPS for its authentication capabilities (I'm generalizing; it depends on the use case). The users of a website want to know with complete certainty that their browsing session is not being intercepted and altered by a third-party. Most users (again, a generalization) do not care if their ISP knows that they requested jquery, font-awesome, bootstrap or some other universally-used library. If you want to know more about the benefits (er, necessities) of authentication, read this post by Troy Hunt. Yes, it is nearly 4 years old, but in that time there has been no progress in breaking up authentication and encryption schemes for websites.

The problem with HTTPS is that it is completely opaque to the entire network between the website's front-end host and the browser. This bypasses the many caching proxies that (may or may not) sit between the website host and the end user.

I have come up with a solution which allows the existing infrastructure to cache and accelerate traffic, while keeping the promise that none of the data can be modified in flight. This proposal would purposely allow the data to be inspected in flight, so that various intermediate servers could cache requests aggressively and short-circuit the round-trip between client and server.

The design is fairly simple. It involves the server setting a cache-expiry header similar to the way they already do, but it also requires the server to sign the responses with the private key corresponding to the domain's https certificate. The intermediate servers are allowed to cache these responses up to the expiration time. Because the responses are cryptographically signed by the origin server, the client can be confident that the response has not been modified (at least, as confident as one can be when trusting the current HTTPS scheme since the trust is inherited from the private key/cert).

This proposed protocol could be called HTTPA, where the A stands for Authenticated. It would replace HTTP for content which is not user-specific nor particularly high-value such as graphics, scripts, css, and other static resources. It could also be used for any data which is requested frequently, but which are still not user-specific or needing privacy such as stock data, game scores, etc. The real tradeoff is frequency of request vs CPU intensity of signing responses.

HTTPA resources referenced from an HTTPS page would not generate a mixed-content warning, as all of the content is authenticated and guaranteed to be pristine and exactly what the server would have sent. Additionally, the user-agent should be able to replace all occurrences of HTTPA with HTTPS, and still receive the same results. Given this parity, users who are paranoid about eavesdropping can turn on full encryption and browse as they would today. Another benefit of this is that you can POST credentials from HTTPA to HTTPS without fear of interception. Your landing page must be authenticated and guaranteed not to be modified in transit. This is guaranteed by HTTPA and not HTTP.

Perhaps the only leakage of information would come in the form of the referer header, which user-agents area already required to mask when requesting a non-secure resource from a secure one. This would remain intact; in the context of the misspelled referer header, HTTPA would be treated the same as HTTP.

In essence, it provides servers with a means to offload some requests to the myriad of caching servers on the internet, without requiring the server operator to give up their private key (a HUGE no-no for many enterprises), fall back to plain HTTP (and risk injected malware), or set up some elaborate CDN (no hate to you CDN providers out there, you're great!). In fact, this would help to augment CDNs because they would not have to make requests for resources if they had cached responses which were signed and unexpired.

The only real drawback to using this is that the server cannot track subresource hits, (or perform analytics on the logs) because these subresources might be cached. Additionally, cookes would be necessarily disabled for HTTPA, because the entire HTTP response is signed, including headers. This removes the reliance on cookieless domains for reducing response packet size (but you still want a few domains for parallelization, which can all benefit even further from HTTPA on initial pageload).

Given a Node.js http/s server, it would not be very difficult to use the crypto module to intercept the http response, sign it, then send it off (possibly with the signature in the trailers, or sign, add the signature as a header, then have the user-agent strip the signature header before verifying the signature). I'll publish a POC soon. Watch this space.


This design has been implemented successfully in another protocol, namely DNSSEC. DNSSEC is what inspired this post.