BSON (Binary JSON) and how your Web API can be even faster

· 1481 words · 7 minutes to read

I have been reading the wishlist at Web API Codeplex repository recently, and noticed that one of the most popular requested features, is to add support for BSON (Binary JSON) media type (6th on the [list][1]).

Of course all it takes to include BSON into your Web API is to simply write a media type formatter for it, and since JSON.NET already has great BSON support, it is actually quite easy.

Now, you might be asking a question, why to do it in the first place? Isn’t JSON enough? Well, the main reason is performance, as according to JSON.NET tests, BSON would produce output that’s often smaller than JSON (up to 25%). It is also much quicker to encode and decode, as for simple types there is no parsing to/from their string representation.

Let’s do it then.

About BSON 🔗

If you are still unsure about using BSON, you can find more information about the specification [here][2], and of course at [Wikipedia][3]. The format has been popularized by MongoDB, and is a great alternative to binary/JSON/XML way of passing the data between the various systems.

The main advantage is that if you for example have a an integer, it doesn’t get converted to a string and back to integer when you are serializing/deserializing your POCO.

Creating the formatter 🔗

So what we need is a new media type formatter. If you are unfamiliar with the concept, take a look at my previous articles: about [media type formatters][4] and about [content negotiation][5].

To create a media type formatter, we inherit from System.Net.Http.Formatting.MediaTypeFormatter.

We will have to override the methods shown below. Please note, this is relevant for Web API RC, because in RTM these signatures change slightly (i.e. HttpContent instead of HttpContentHeaders is passed).

public class BsonMediaTypeFormatter : MediaTypeFormatter  
{  
public override bool CanReadType(Type type) {}  
public override bool CanWriteType(Type type) {}  
public override Task

<object>
  ReadFromStreamAsync(Type type, Stream stream, HttpContentHeaders contentHeaders, IFormatterLogger formatterLogger) {}<br /> public override Task WriteToStreamAsync(Type type, object value, Stream stream, HttpContentHeaders contentHeaders, TransportContext transportContext) {}<br /> }<br /> ```</p> 
  
  <p>
    In this example we will follow the conventions of the Web API source code, so in addition to the necessary overrides, we expose some public members of the formatter the same way the default Web API JSON.NET formatter does.
  </p>
  
  <p>
    Let's start with a constructor though.<br /> ```csharp
<br /> private JsonSerializerSettings _jsonSerializerSettings;<br /> public BsonMediaTypeFormatter()<br /> {<br /> SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/bson"));<br /> _jsonSerializerSettings = CreateDefaultSerializerSettings();<br /> }<br /> ```
  </p>
  
  <p>
    We start off by telling the formatter to support "application/bson" media type. We also initialize our private JSON.NET serializer settings for later use.<br /> ```csharp
<br /> public JsonSerializerSettings CreateDefaultSerializerSettings()<br /> {<br /> return new JsonSerializerSettings()<br /> {<br /> MissingMemberHandling = MissingMemberHandling.Ignore,<br /> TypeNameHandling = TypeNameHandling.None<br /> };<br /> }
  </p>
  
  <p>
    public JsonSerializerSettings SerializerSettings<br /> {<br /> get { return _jsonSerializerSettings; }<br /> set<br /> {<br /> if (value == null)<br /> {<br /> throw new ArgumentNullException("Value is null");<br /> }
  </p>
  
  <p>
    _jsonSerializerSettings = value;<br /> }<br /> }<br /> ```
  </p>
  
  <p>
    The default serialization settings are the same as those of the default Web API JSON.NET formatter. Additionally, you can set your own if you wish, through the use of the public property.
  </p>
  
  <p>
    Now let's handle the <i>CanReadType</i> and <i>CanWriteType</i> methods.<br /> ```csharp
<br /> public override bool CanReadType(Type type)<br /> {<br /> if (type == null) throw new ArgumentNullException("type is null");<br /> return true;<br /> }
  </p>
  
  <p>
    public override bool CanWriteType(Type type)<br /> {<br /> if (type == null) throw new ArgumentNullException("Type is null");<br /> return true;<br /> }<br /> ```
  </p>
  
  <p>
    Since JSON.NET can serialize any CLR types, we will always return <i>true</i> for both methods, as long as a type is passed to the formatter.
  </p>
  
  <p>
    Finally, let's deal with writing to the stream (serialization) and reading from the stream (deserialization).<br /> ```csharp
<br /> public override Task WriteToStreamAsync(Type type, object value, Stream stream, HttpContentHeaders contentHeaders, TransportContext transportContext)<br /> {<br /> if (type == null) throw new ArgumentNullException("type is null");<br /> if (stream == null) throw new ArgumentNullException("Write stream is null");
  </p>
  
  <p>
    var tcs = new TaskCompletionSource
    
    <object>
      ();<br /> using (BsonWriter bsonWriter = new BsonWriter(stream) { CloseOutput = false })<br /> {<br /> JsonSerializer jsonSerializer = JsonSerializer.Create(_jsonSerializerSettings);<br /> jsonSerializer.Serialize(bsonWriter, value);<br /> bsonWriter.Flush();<br /> tcs.SetResult(null);<br /> }</p> 
      
      <p>
        return tcs.Task;<br /> }<br /> ```
      </p>
      
      <p>
        The "write" code should be very straightforward. We use JSON.NET's <i>BsonWriter</i> to serialize our object. We run the serialization synchronously (hence the use of TaskCompletionSource), the same way as Web API does it, because there is no real advantage of switching threads for these simple operations.
      </p>
      
      <p>
        Now the deserialization:<br /> ```csharp
<br /> public override Task
        
        <object>
          ReadFromStreamAsync(Type type, Stream stream, HttpContentHeaders contentHeaders, IFormatterLogger formatterLogger)<br /> {<br /> var tcs = new TaskCompletionSource
          
          <object>
            ();<br /> if (contentHeaders != null && contentHeaders.ContentLength == 0) return null;</p> 
            
            <p>
              try<br /> {<br /> BsonReader reader = new BsonReader(stream);<br /> if (typeof(IEnumerable).IsAssignableFrom(type) reader.ReadRootValueAsArray = true;
            </p>
            
            <p>
              using (reader)<br /> {<br /> var jsonSerializer = JsonSerializer.Create(_jsonSerializerSettings);<br /> var output = jsonSerializer.Deserialize(reader, type);<br /> if (formatterLogger != null)<br /> {<br /> jsonSerializer.Error += (sender, e) =><br /> {<br /> Exception exception = e.ErrorContext.Error;<br /> formatterLogger.LogError(e.ErrorContext.Path, exception.Message);<br /> e.ErrorContext.Handled = true;<br /> };<br /> }<br /> tcs.SetResult(output);<br /> }<br /> }<br /> catch (Exception e)<br /> {<br /> if (formatterLogger == null) throw;<br /> formatterLogger.LogError(String.Empty, e.Message);<br /> tcs.SetResult(GetDefaultValueForType(type));<br /> }
            </p>
            
            <p>
              return tcs.Task;<br /> }<br /> ```
            </p>
            
            <p>
              While this method is also fairly simple, two notes here. We need to use <i>IsAssignableFrom</i> to determine if the deserialized type is a collection. This is a limitation of BSON as it cannot automatically detect the root container type. If it is, we treat the entire BSON as a collection.
            </p>
            
            <p>
              The second thing worth noting is the error handling. This is actually copied from the ASP.NET Web API source, and that's exactly how the default JSON.NET formatter handles the errors. The goal is, as mark all exceptions as handled, as otherwise it may be rethrown at each recursive level and overflow the CLR stack.
            </p>
            
            <h3>
              Plugging it in
            </h3>
            
            <p>
              As usually, we need to plug it into our <i>GlobalConfiguration</i>.<br /> ```csharp
<br /> GlobalConfiguration.Configuration.Formatters.Add(new BsonMediaTypeFormatter());<br /> ```
            </p>
            
            <h3>
              Consuming BSON serialized types
            </h3>
            
            <p>
              Now let's see this in practice. I will be testing with a couple of simple Console app methods.
            </p>
            
            <p>
              In this test example I am using a simple <i>MyType</i> class which has one property only, a string <i>Name</i>.<br /> ```csharp
<br /> var client = new HttpClient();<br /> client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/bson"));<br /> Task<Stream> stream = client.GetStreamAsync("http://localhost:50006/api/values/GetAll");
            </p>
            
            <p>
              MemoryStream result = new MemoryStream();<br /> stream.Result.CopyTo(result);<br /> result.Seek(0, SeekOrigin.Begin);
            </p>
            
            <p>
              using (BsonReader reader = new BsonReader(result) { ReadRootValueAsArray = true })<br /> {<br /> var jsonSerializer = new JsonSerializer();<br /> var output = jsonSerializer.Deserialize<IEnumerable<MyType>>(reader);<br /> foreach (var item in output)<br /> {<br /> Console.WriteLine(item.Name);<br /> }<br /> }<br /> ```
            </p>
            
            <p>
              It looks like this in Fiddler (binary response):<br /> <a href="/images/2012/07/fiddler_view.png"><img src="/images/2012/07/fiddler_view.png" alt="" title="fiddler_view" width="620" height="331" class="aligncenter size-full wp-image-401" srcset="/images/2012/07/fiddler_view.png 831w, /images/2012/07/fiddler_view-300x160.png 300w" sizes="(max-width: 620px) 100vw, 620px" /></a>
            </p>
            
            <p>
              This returns the following output (binary response can be deserialized smoothly):<br /> <a href="/images/2012/07/items_console.png"><img src="/images/2012/07/items_console.png" alt="" title="items_console" width="620" height="133" class="aligncenter size-full wp-image-402" srcset="/images/2012/07/items_console.png 665w, /images/2012/07/items_console-300x65.png 300w" sizes="(max-width: 620px) 100vw, 620px" /></a>
            </p>
            
            <p>
              Now, what I can do as well, obviously, is I can post an item back using the BSON format.<br /> ```csharp
<br /> var client = new HttpClient();<br /> client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/bson"));<br /> var content = new ObjectContent(typeof(MyType),new MyType() {Name = "New Item"} ,new BsonMediaTypeFormatter());<br /> client.PostAsync("http://localhost:50006/api/values/Post", content);<br /> ```
            </p>
            
            <p>
              Now if I enumerate all of them again, I can see it has been added.<br /> <a href="/images/2012/07/items_console2.png"><img src="/images/2012/07/items_console2.png" alt="" title="items_console2" width="620" height="137" class="aligncenter size-full wp-image-403" srcset="/images/2012/07/items_console2.png 663w, /images/2012/07/items_console2-300x66.png 300w" sizes="(max-width: 620px) 100vw, 620px" /></a>
            </p>
            
            <p>
              Finally, let's see if the performance really improves with BSON. JSON.NET actually comes in with nUnit performance tests. If you run those you'd see that BSON gives the smallest size, and usually outperforms JSON and is right on par or faster than DCS.
            </p>
            
            <p>
              <a href="/images/2012/07/performance.png"><img src="/images/2012/07/performance.png" alt="" title="performance" width="460" height="453" class="aligncenter size-full wp-image-408" srcset="/images/2012/07/performance.png 460w, /images/2012/07/performance-300x295.png 300w" sizes="(max-width: 460px) 100vw, 460px" /></a>
            </p>
            
            <h3>
              Summary and source code
            </h3>
            
            <p>
              As you can see, you can easily add BSON support for your Web API, and benefit from its performance. Of course it is not suitable for sending down to the browser, but whereever your client has the capabilities of deserializing BSON (i.e. any .NET client which has access to JSON.NET), then you should definitely consider providing BSON media type from your API.
            </p>
            
            <p>
              Source code, as always, included.
            </p>
            
            <p>
              - <a href="https://gist.github.com/3160050">source code (gist)</a>
            </p>

 [1]: http://aspnetwebstack.codeplex.com/workitem/list/basic
 [2]: http://bsonspec.org/
 [3]: http://en.wikipedia.org/wiki/BSON
 [4]: /2012/04/rss-atom-mediatypeformatter-for-asp-net-webapi/
 [5]: /2012/07/everything-you-want-to-know-about-asp-net-web-api-content-negotation/

About


Hi! I'm Filip W., a cloud architect from Zürich 🇨🇭. I like Toronto Maple Leafs 🇨🇦, Rancid and quantum computing. Oh, and I love the Lowlands 🏴󠁧󠁢󠁳󠁣󠁴󠁿.

You can find me on Github and on Mastodon.

My Introduction to Quantum Computing with Q# and QDK book
Microsoft MVP