Extending your ASP.NET Web API responses with useful metadata - StrathWeb

Strath

Extending your ASP.NET Web API responses with useful metadata

Because heplpful information make it easier for the client

If you ever worked with any API, which, in this day of age, you must have, you surely noticed that in most situations the API response isn’t just the result (requested data), but also a set of helpful metadata, like “total Results”, “timestamp”, “status” and so on.

In Web API, by default, you just serialize your models (or DTO) and such information are not present. Let’s build something which will solve this problem and help you decorate your response with hepful information. This would make it very easy for the client to implement paging, auto-loading scenarios, caching (if you return last modified information) and a lot more.

What are we going to build

First let’s go through a plan of what we are going to build.

Assume you have some sample repository. In the recent samples, I have always been using a dummy repository from this post, so let’s use it again.

Now, by default, the response will look like this (for a single item):

If we request for IQueryable, we obivously get an Array.

Now, what we’d like to have is something like this:

So much nicer isn’t it?

Designing the service

So how are we going to accomplish this? Pretty simple:
1. We will implement a DelegatingHandler which will capture all HttpResponseMessages, extract the response object and wrap into our custom generic Metadata. Then it will be flushed to the client.
2. Additionally, we’ll have a CustomQueryableAttribute, which we will use on IQueryable Actions, to keep the information about the size (count) of the IQueryable. This way we will be able to provide information about what is the total size of the collection and thus support OData filtering. This way the client can request i.e. $top=3 results, but still have information about the total size of the IQueryable.

Models

As mentioned, we will use a repository from here – it’s really simple and perfect for testing. Let’s have a look at the Metadata instead.

It is a generic class, which will provide information such as:
– total results (since the data may be filtered)
– returned results (since the data may be filtered)
– results object (which is equal to the default WebAPI response)
– timestamp of the response
– status – to indicate a successful or unsuccessful response

I will leave the constructor empty for now, it will be more clear to what’s happening there, once we go through the handler and filter.

The Data annotations are there for compatibility with DataContractSerializer. Content negotation is supported, but in principle, this functionality is better suited for JSON.NET, but more on that later.

MetadataHandler

MetadataHandler will inherit from a DelegatingHandler, which means it will have access to the HttpResponseMessage before it is flushed to the client and before applying MediaFormatting.

As mentioned, we need to modify the response, and to do that we need to override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) method and add a ContinueWith to it.

In there, we extract the object from the response and pass it to a private method for altering.

Now, here is a note. If you serialize to JSON, using JSON.NET you can do just that, it can handle that just fine. On the other hand, if you are willing to use DataContractSerializer, it will not be able to serialize object types properly. You would need known types, so you’d need either reflection or typecasting:

Anyway, going back to our example, the ProcessObject method:

In this case, we instantiate a new Metadata and set it as the content of the response. As you see, I arbitrairly set the formatter to JSON.NET. The commented out code preservers content negotiation, but if you use that you’d need to type cast the objects as mentioned before.

We also used a simple helper method to check if the response is even worth processing:

Let’s now revisit the constructor we omitted earlier.

The constructor takes the HttpResponseMessage and builds up useful information based on it – i.e. setting the “sucess” or “error”, and also calculating the number of returned results.

Note, if single item is requested (via regular Get(int id) type of action, not OData) it will show 1 total result, because that’s how many matches there were. Total results is greater than returned results only if you filter data with OData.

Adding support for OData and IQueryable

All of this wouldn’t be very useful if we didn’t add our key functionality, which is “total results”. To do that, we need to introduce a new attribute filter, CustomQueryableAttribute, inheriting from QueryableAttribute.

We’d then decorate all IQueryable actions with it.

We override the method IQueryable ApplyResultLimit(HttpActionExecutedContext actionExecutedContext, IQueryable query), which gives us access to IQueryable prior to applying the filtering. What it means, is that we can easily save aside the total number of results of the IQueryable and then use it later in our Metadata object.

We use a bit of a hack here, since I save that in a custom header field “originalSize”. In the MessageHandler later on, we will read that value from the headers and remove it so that they don’t get sent to the client – so it only saves the purpose of transporting a variable from the ActionFilter to the MessageHandler. If you want to pass data between two ActionFilters you could use ControllerContext.RouteData.Values, but for our scenario I couldn’t find a better way.

Now we need to update the MessageHandler, to make it aware of the “originalSize”:

So we read the header value and get rid of the redundant header key.

Registering handler, decorating methods

The final thing to do is to register the handler, so in App_Start we add:

For testing, I get rid of the XmlFormatter as well.

We also add our custom queryable filter to all Action’s returning IQueryable:

Running the application

Everything is ready, so let’s roll.

First let’s get all Urls:

Let’s test OData:

Now let’s get a single Url:

Now, just to show that this doesn’t really on any strong types, let’s add a new repository with Blog objects.

Summary & source code

Hopefully someone will find the functionality described here useful. It can be obviously extended further, and optimized (perhaps someone can figure out a better way to handle DataContractSerializer).

Anyway, I had fun writing this, hope you had fun reading. Till next time!

source code on Github

Be Sociable, Share!

  • Richard Reukema

    How about changing that time stamp to DateTimeOffset, allowing for the client to convert the time into their own local time….

  • Pingback: Dew Drop – June 4, 2012 (#1,340) | Alvin Ashcraft's Morning Dew

  • chad wackerman

    Many of these fields belong in the HTTP headers not the body. Especially “last modified.” If you do it right, you get caching through CDN, Proxy, etc. for free.

    Look at how Amazon, Microsoft (Azure) and others do it. Don’t reinvent HTTP.

    • Filip W

      I think you are totally missing the point here. Headers, that’s a completely separate topic. Caching that’s another story and not the purpose of this article! Amazon or Azure has nothing to do with it. I do not present here a complete ready to production API service!

      What I am doing is merely enriching the response body with some useful information for the client. And believe me, this is not my half-baked idea, since if you call any API these days, say Twitter API or ESPN API or any other, you’d see that each of those return “Results” as a property of a response, rather than the response alone, and alongside that, other properties which contain all kinds of information about paging, execution time, results limit, service status and so on (the point of this article).

      And regarding your last-modified remark. I merely included a timestamp (which means query execution time). Last modified in the headers is one thing (and I agree it’s very needed, but that is not the point of this article), but what if you implemented (using this technique here) a last modified date *per-every item* in the results collection? This way you give your client information that allows them to recognize when every single result item was last modified and re-request that when needed, instead of relying on last modified date for the entire response, and having to re-request everything.

  • Marco

    I can’t imagine something like that isn’t part of the core release. This is so needed for every enterprise scale API.

    MS did a great job with Web API but left a lot of holes to fill manually – with things like this or not supporting OData $inlinecount.

    Thanks for a superb post.

  • Pingback: The Morning Brew - Chris Alcock » Afternoon Tea - Sunday 10th June 2012

  • Stevo

    Good job mate.

    Changes to MetadataHandler to support DataContractSerializer in a generic way:

    protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
    return base.SendAsync(request, cancellationToken).ContinueWith(
    task =>
    {
    if (ResponseIsValid(task.Result))
    {
    object responseObject;
    task.Result.TryGetContentValue(out responseObject);
    ProcessObject(responseObject as IEnumerable, task.Result, responseObject is IQueryable);
    }
    return task.Result;
    }
    );
    }

    private void ProcessObject(IEnumerable responseObject, HttpResponseMessage response, bool isIQueryable)
    {
    Type responseType = responseObject.GetType();

    if (!responseType.IsGenericType)
    throw new InvalidOperationException(“Excpecting strongly typed results”);
    else
    {
    Type itemType = responseType.GetGenericArguments()[0];
    Type metadataType = typeof(Metadata).MakeGenericType(itemType);
    Type objectContentType = typeof(ObjectContent).MakeGenericType(metadataType);

    var metadata = (IMetadata)Activator.CreateInstance(metadataType, response, isIQueryable);
    var originalSize = new string[1] as IEnumerable;

    response.Headers.TryGetValues(“originalSize”, out originalSize);
    response.Headers.Remove(“originalSize”);
    if (originalSize != null)
    {
    metadata.TotalResults= Convert.ToInt32(originalSize.FirstOrDefault());
    }

    var formatter = GlobalConfiguration.Configuration.Formatters.First(t => t.SupportedMediaTypes.Contains(new MediaTypeHeaderValue(response.Content.Headers.ContentType.MediaType)));
    var objectContent = (HttpContent)Activator.CreateInstance(objectContentType, metadata, formatter);
    response.Content = objectContent;
    }
    }

    public interface IMetadata
    {
    int ReturnedResults { get; set; }
    bool Success { get; set; }
    DateTime Timestamp { get; set; }
    int TotalResults { get; set; }
    }

    • Filip W

      oh, great stuff! will try it out!
      Thanks a million

  • http://www.timothy-giedner.com Tractor Tim

    great page mate

  • Curious Chap

    great stuff mate

  • Josh

    Thank you, this got me up and running! Unfortunately, the TotalResults variable only gets set on the original Queryable. If you add a $filter, it still returns the full count instead of the $filtered count. Is there anyway to get the count of the $filtered results, but before $skip and $take?

  • Stevo

    huh, after updating to latest MVC 4, the QueryableAttribute is gone. Any ideas?

    • Filip W

      OData is now supported through an external package, Microsoft.AspNet.WebApi.OData, available on Nuget in a preview/alpha format. Full release is coming later this fall. You can grab it from here. There is a nice overview post available here.

      • Georgios

        Right, but the ApplyResultLimit method is gone – any idea how to make this work with the new OData package?

        • Filip W

          Yes the QueryableAttribute is slightly different now. Please read this new post.

  • David Dupe

    Great blog but maybe you should get up and stand up

  • Pingback: How can I include custom metadata in an ASP.NET Web API method response?