Background

In web application servers it is useful to perform operations on "finished" responses before they are sent to a client. You might want to set caching or other headers, or compress the body data. Some web frameworks, including Rocket, have a system for user-defined middleware or response wrappers that can perform these operations in a reusable and composable way.

There are a wide variety of operations you might want to apply to a response after it has been built:

• Add a Server or other global header
• Set the Content-Type, Content-Disposition, or another response-specific header
• Compress the response body and set the Content-Encoding header
• Checksum the response and save the checksum in the ETag header
• Compare the If-None-Match header in the request to the ETag of the response, and respond with a 304 Not Modified if they match
• Serve a portion of the body selected by the Range header in the request
• Add an X-Response-Time header indicating how much time the server spent processing the request

A server might perform some operations on every response, or only on specific routes or when certain conditions are met. In Rocket, the idiomatic way to operate on individual responses is the Responder. Some responder types, such as String or Vec, set the response body to the bytes they contain. Other responders, such as Content, modify the result of another Responder. These are known as wrapping Responders, and they are the building blocks of composable operations on responses.

Consider the following route:

use rocket::http::ContentType;
use rocket::response::Content;

#[get("/hello")]
fn hello() -> Content<&'static str> {
    Content(ContentType::JSON, "{\"hello\": \"world\"}")
}

When this route returns, Content::respond_to() is called. It is a wrapping responder: its implementation is "Run the inner responder, then set the Content-Type header to the specified ContentType (in this case application/json)." Here the inner responder type is &str, and its respond_to implementation is "set the response body to the UTF-8 bytes underlying this string and set the Content-Type to text/plain".

Because Content sets the Content-Type header after the inner responder has run, the final response sent to the client will have Content-Type: application/json as desired.

Handling Content-Range

Another useful wrapping responder might be the "Range request handler". Range requests are commonly used for resuming downloads and skipping around in streaming media. Suppose we had a wrapping responder called Range, used like this:

#[get("/video.mp4")]
fn video() -> Range<File> {
    Range(File::open("video.mp4"))
}

Range might implement respond_to in the following way:

1. Run the inner responder - in this example, File
2. Check if the client sent a Range header
3. Grab the requested portion of the response body
4. Set the response body to be only the portion
5. Set the Content-Range header on the response

In a real project this wouldn't show as a video in most browsers, because we forgot to set the Content-Type header. Let's fix it:

#[get("/video.mp4")]
fn video() -> Range> {
    Range(Content(ContentType::MP4, File::open("video.mp4")))
}

Hmm. What about Content>?

#[get("/video.mp4")]
fn video() -> Content> {
    Content(ContentType::MP4, Range(File::open("video.mp4")))
}

That works too! Content and Range can safely be reordered because they don't interfere with each other in any way.

Suppose we decided Range is a really nice feature and we built it into Rocket directly, so every File will handle range requests automatically:

#[get("/video.mp4")]
fn video() -> Content {
    Content(ContentType::MP4, File::open("video.mp4"))
}

Much simpler, and now we don't have to worry about Range handling because it's already done for us!

Handling ETag

Now imagine another useful responder. The HTTP ETag header carries a checksum of the response body which can be used to make repeated requests more efficient. A browser can send a checksum it already has in the If-None-Match header, and if it matches the current ETag the server can send 304 Not Modified with no body instead of a 200 OK.

Let's use our hypothetical ETag responder

#[get("/hello")]
fn hello() -> ETag<&'static str> {
    ETag("Hello there!")
}

ETag might work like this:

1. Run the inner responder
2. Checksum the body
3. Respond with a 304 Not Modified if the checksum matches
4. Set the ETag header

Just like the order of Range and Content don't matter, the order of ETag and Content does not matter.

But the order of Range and ETag does matter! The checksum of a whole file is different from the checksum of any section of the file. But that's easy to fix: always use Range> and never ETag>. That way, the checksum will always be calculated for the whole file.

Disappointment

Now we have a problem.

#[get("/video.mp4")]
fn video() -> Content>> {
    Content(ContentType::MP4, ETag(File::open("video.mp4")))
}

Remember that we wanted to make File handle range requests automatically, so ETag will process after Range handling. But as I just pointed out, Range handling can only correctly be done after ETag handling.

This is disappointing to me, because it would have been really nice for Range requests to be handled automatically in Rocket. If you were hoping for a clever solution or idea I'm afraid you will leave disappointed too, because I don't have one yet.

The Dfam consortium is excited to announce the release of Dfam 3.0. This release represents a major transition for Dfam from a proof-of-concept database into a funded open community resource. Central to this transition is a major infrastructure and technology update, enabling Dfam to handle the increasing pace of genome sequencing and TE library generation.

—Xfam blog, https://xfam.wordpress.com/2019/03/06/dfam-3-0-is-out/

Since I started at the Institute for Systems Biology (ISB) last September we have been hard at work on this update. My own contributions to the 3.0 release were to the new REST API, the new web interface written in Angular, and updating some existing backend scripts and visualizations.

I would like to thank Robert Hubley and Arian Smit at ISB, Travis Wheeler at the University of Montana, and all the previous contributors to Dfam. Without them none of this would have been possible, and I am excited to be here working on this and other projects.

The opinions expressed herein are my own and do not necessarily represent the views of ISB or any of its collaborators.

About six months have passed since I announced the Washington programming language, but not much progress has been made in that time. I have completed a draft specification and an interpreter, but much work remains to be done. The next steps are to define more presidents and write more example programs.