Mar 01

Apache and Byte-Ranges for resumable downloads

Category: Linux,webserver   — Published by goeszen on March 1, 2017 at 4:51 pm

While preparing a website for serving larger files via Apache 2.2, I recently noticed that downloads were not resumable on this web property. To pinpoint the problem (and fixing it in the end -spoiler: wasnt' Apache's fault) I dug through the whole stack and came up with some findings others might find useful:

First of all: Apache does support the "Accept-Ranges" / "Range" headers out-of-the-box. No special configuration is needed, and you don't have to explicitly enable the Ranges header or load any non-standard modules.

Background on range serving

When a user downloads a larger file, for example with Firefox, she/he can press "PAUSE" on the download, and when the user decides to continue this download, the browser or download manager will be able to resume downloading where it left of. This is done via the Ranges facility. A webserver supporting this standard advertises the support of byte range serving via sending the Accept-Ranges header on responses to GET requests. A downloading client may interpret this header and may ask for a specific part of a file by sending the Range HTTP header on (subsequent) GET requests. There are a few modes of specifying how a range is defined, a common one is "bytes". All this is defined in RFC2616, chapter 14.5.

This range serving is what people refer to when they ask How they can download with pause and resume, for example in this question.

Testing if your server serves byte ranges

You can test if your Apache setup is serving ranges by asking with curl (for example) for a range. In case your server responds with a status code of 206 partial content your server is ready and properly configured to serve byte ranges. Example curl output:

$ curl -i -X HEAD --header "Range: bytes=50-100" http://www.example.com/style.css
HTTP/1.1 206 Partial Content
Date: Wed, 01 Mar 2017 13:58:06 GMT
Server: Apache
Last-Modified: Thu, 03 Nov 2016 15:14:44 GMT
ETag: "a002df-160c-54067aa74702f"
Accept-Ranges: bytes
Content-Length: 51
Vary: Accept-Encoding
Content-Range: bytes 50-100/5644
Content-Type: text/css

curl: (18) transfer closed with 51 bytes remaining to read

In case your server is not accepting byte ranges, it will simply respond with a status code 200, like below, although you asked for a range:

$ curl -i -X HEAD --header "Range: bytes=50-100" http://example.com/image.png
HTTP/1.1 200 OK
Date: Wed, 01 Mar 2017 12:51:47 GMT
Server: Apache
Content-Length: 42760
Last-Modified: Tue, 28 Feb 2017 15:39:32 GMT
Content-Type: image/png

Note: If you want curl to echo your request header as well, add the verbose switch (-v) and curl will dump the header it is sending to your server out on console. (via)

Related discussion here, here, for uploads here and for svn here.

If it's not working

If your Apache server is not answering Range requests with Partial Content 206, ranges must be disabled somewhere in your config. Look into the http.conf, apache.conf, with split configs in conf.d/* and individual vhost config files.

In contrary to what Vivek in his very detailed article suggests, Apache does not need the mod_headers / headers module for byte range serving to work!
The mod_headers module (docs) is useful when you want to modify how, what or which HTTP-Headers Apache is serving. So for example when you want to disable range serving on a specific vhost or a singular directory - but the logic to serve (byte) Ranges is not in the mod_headers module - Apache will allow partial downloads or resumable downloads just fine without mod_headers being loaded.

In case you wonder, you can find out which modules Apache has loaded by issueing apachectl -M or looking into the apache dir /etc/apache2/mods-enabled.

Restricting or disabling the Ranges header

Sometimes you may want to restrict or disable (byte) range serving altogether. You can do so by loading the mod_headers module and turning off the Ranges header on a global, per vhost or even per-directory level. Use something like described here:

Header set Accept-Ranges none

This directive sets Accept-Ranges to none - effectively disabling it. Make sure you test it via a curl get.

Sometimes you'll hear about accepting ranges may impose a security risk. From what I read it is a know vulnerability and fixed in more recent Apache versions (> apache 2.x). Also, you probably already employ mod_security to mitigate common exploits, and mod_security has rules against DDoS and similar attacks against the Range(s) mechanism.

This post here has another simple approach to selectively disable answering range requests.

Issues in relation with dynamic websites

The issue I was facing with Apache serving byte-ranges on some vhosts, while not doing it on others was related with dynamic websites, where a script produces the content on a site - in contrast to static content simply served by Apache. In short, for anyone interested, with my setup, a rewrite rule wasn't sending dynamic-pages-only to the script but was asking for static content as well through the dynamic mechanics of the site. Thus the script and not Apache natively was handling the serving. And the script, as with most web-frameworks, wasn't supporting serving Range requests. Read more about this issue in RewriteCond REQUEST_FILENAME file test not working.

For keywords:
apache2 enable resumable downloads
apache 2.2 accept-ranges header
apache2 range header
Apache Range Header

Leave a Reply

=