A guide to asynchronous file uploads in ASP.NET Web API RTM

Strath

August 29th, 2012

A guide to asynchronous file uploads in ASP.NET Web API RTM

Because files neeed to be uplaoded sometimes

It’s been 2 weeks since Web API has been released, and in a post highlighting the changes, I mentioned I will write an updated tutorial on file upload.

File upload is quite an important topic for Web API endpoitns or for API-driven applications, and sure enough there are some nice changes to the MultiPartFormDataStreamProvider, which, contrary to its versions in the Beta and RC allow easy and flexible model for dealing with uploaded data. On the other hand, these changes between Beta-RC-RTM, mean many of the upload-related posts found in the Web API community (this blog included) no longer work properly.

Let’s have a look at how you could now upload files to your ASP.NET Web API.

Sample controller

First, a small disclaimer, all the code here is .NET 4.0 specific – you could easily make it 4.5 by replacing the Tasks with async and ContinueWith with await. However, since most people still run 4.0, I thought it might be more apporpriate.

To be honest, in order to just upload the files to Web API, you need roughly 10 lines of code:

In a gist, that’s all you need. Granted, this code is not very useful yet, but it will grab the files from the request and save them in a specified location. We will continue morphing the code into a better solution as we go on with this post.

There are several problems with it now:
1) The files are saved using awkward names such as “BodyPart_3ded3dfb-40be-3153-b589-3401f93e90af” and so on.

2) The path where we are saving is hardcoded, and that’s especially problematic on a shared host where you might not even know what’s your folder path

3) There is a big chance, that we want to return something to the client, like a list of uploaded files and their related info (some might argue that’s not very REST-like, but let’s do it for the sake of an example). This was quite a problematic thing to achieve in RC/beta, so let’s explore how much easier it has become.

Correct filenames

The reason why body parts get saved on the disk with such random names is not an error or an accident. It has been designed like to that as a security measure – so that malicious uploaders wouldn’t compromise your system by uploading evil files disguised with friendly names.

Previously, one of the hacks I used, was to let the files save themselves with those randomly generated names, and then use File.Move to rename them. However, this is not the most efficient solution solution in the world by any stretch of the imagination.

However, now, you can easily derive from the default MultiPartFormDataStreamProvider and provide your own naming mechanism.

Let’s have a look at such simple example:

In this case, we implement our own naming convention, and tell the framework to pull out the name of the file from the Content-Disposition part of the request. this, in turn, will be the original name of the file as provided by the client.

This is the most basic example of all, but I hope you can imagine that you could do a lot more here – for exmaple implement your naming conventions and achieve file name patterns suitable for your application such as “UserA_avatar1.jpg” and so on.

You use it the same way as you would use the out-of-the-box MultipartFormDataStreamProvider:

Saving relatively to the web app path

As mentioned, using a hardcoded path c:/uploads/ is far from ideal, especially in shared hosting.

You can easily save the files relatively to the web app though, by taking advantage of the fact that web hosted ASP.NET Web API runs on top of ASP.NET and reaching to the dreaded HttpContext:

This will upload files to the folder relative to your web app rooy (note: in this simple example the “uploads” fodler must exist, otherwise you will get a 500 exception!).

Returning some meaningful info

Finally, you may want to return some information about the uploaded files to the client. If that’s the case, one way to do it is to use a helper class (which was used already in the older post):

This will be our DTO that will transport file information to the client. LEt’s modify our controller now.

So now, instead of void we return a List<FileDesc> which can provide the client information about each of the uploaded files: its name, path and size.

Of course in this example we are returning the same name the client uploaded (so not too useful) but you can imagine in real life scenario if you had some custom naming going on it would be useful to communicate it to the client. Moreover, the path property will simply contain a link (url).

This is especially easy now in RTM, because MultipartFileStreamProvider now exposes a FileData collection which allows you to reach into the request and pull out information about local file names (as they have been saved), which we could then use to get even more information using FileInfo from System.IO.

Now if you run this, you will see things immediately get nice and informative.

And of course, correctly uploaded in the server’s file structure.

Calling it from the client

There are numerous way to call this controller from a client application to upload files. Ultimately, whereever you are capable of composing a MIME Multipart content HTTP message, you are able to call upload files to Web API.

I will show here 3 quick examples:

1. JavaScript
I have already discussed it before in great details, but you could drag-and-drop files to the browser. Then these files could be picked up arranged into a FormData object and sent via XHR:

2. HTML

This simple HTML form will allow you to select a file using the traditional browser dialog and send it to the Web API controller. You could also use the HTML5 multiple attribute to allow for multi-file selections.

In both cases, the upload works just fine.

3. .NET application
You might also have a need to upload files from a .NET application – i.e. your WPF app or a Windows service. In that case (assuming you already have a list of the filepaths in the files variable):

The end result is the same as previously, a multipart message with files attached as stream data is sent to the controller. You can find more info in this post.

Summary & Source

I hope this post will help you a little bit with dealing with file uploads scenarios when using Web API endpoints. As usually I am including the source project, and if you have questions or comments – you know where to find me!

source on github

Be Sociable, Share!

  • darren

    So what about uploading the content to SQL Server?

    From what I’ve seen to use the web api with file uploads I will have to 1.) Upload to folder on web server, 2.) read the file into stream in-order to write it to a database.

    Unless I’m missing something…

    • Filip W

      In such case you should use MultipartMemoryStreamProvider instead of MultipartFormDataStreamProvider.
      Pseudo code:

      var streamProvider = new MultipartMemoryStreamProvider();
      var multipart = Request.Content.ReadAsMultipartAsync(streamProvider).ContinueWith(t =>
      {
      foreach (var item in streamProvider.Contents){
      //do stuff
      }
      }

  • Pingback: The Morning Brew - Chris Alcock » The Morning Brew #1179

  • Pingback: Dew Drop – August 31, 2012 (#1,392) | Alvin Ashcraft's Morning Dew

  • http://techshorts.ddpruitt.net darren

    I tried but MultipartFormDataStreamProvider constructor is protected. I pulled the code for the class from CodePlex and there is nothing there.

    True, it implements the abstract class MultipartStreamProvider but the one method override is GetStream, which only returns an empty MemoryStream.

    And I have searched for a couple of days now and have found no examples of using the Web Api to save a file to a database.

    At this point I still think the best approach is to save the file locally then upload to file to the database. Otherwise I will have to take the MultipartFormDataStreamProvider and tweak it to do the MultipartMemoryStreamProvider.

    For the record your post is great and it is the only one I found that fully explained the current MultipartFormDataStreamProvider, so Thank You! You saved me much frustration.

    • simmo

      You may need to set the stream’s Position to 0. Sometimes when you get a stream it has its read/write offset set to the end of the stream so you’ll end up getting nothing when you try to read from it.
      Just a thought. I haven’t actually tried it for your scenario.

  • http://blog.nathan-taylor.net Nathan

    This was a really informative post, thanks!

  • Pingback: TWC9: WAMS! Windows Store End to End, Reflection, VS2012 ALM VM and more - Windows Azure Blog

  • key1

    That’s a nice post.

  • http://tatabanya.helendoron.hu tatabánya angol

    “CustomMultipartFormDataStreamProvider”

    huh, live long intellisense :)

  • sando

    After ContinueWith() the task remains infinitely on “WaitingForActivation” status and hangs the process. Though I see that the file has been written in the “Uploads” folder.

    I find the same issue being discussed here: http://forums.asp.net/p/1836953/5126753.aspx/1?Re+ReadAsMultipartAsync+Breaks+with+Release

    Is it that RTM has this bug RC doesnt?

  • Pete Sykes

    Thanks for this post – it’s been a lifesaver. A couple of points though – for an empty file (“NoName”) the replacement of escaped quotes needs to take place *before* the check for IsNullOrWhiteSpace.

    My version of GetLocalFileName in CustomMultipartFormDataStreamProvider:

    public override string GetLocalFileName(System.Net.Http.Headers.HttpContentHeaders headers) {
    var suppliedName = headers.ContentDisposition.FileName.Replace(“\”", string.Empty); // this is here to deal with escaped quotation marks
    return !string.IsNullOrWhiteSpace(suppliedName) ? suppliedName : “NoName”;
    }

    • Filip W

      yes, good catch, thanks.

      Additionally, if you want to validate the name or extension of the file *prior* to it being read into the stream you need to override the GetStream method instead. GetLocalFileName executes after reading the file into a stream.

      filip

  • BK

    First off, Thanks for the great post.

    I am running into faulted Task problem during subsequent requests.

    Here’s the scenario: I have a sample page that posts a file along with other parameters to the Web API Post method indicated in your post. It works fine the first time. But, all subsequent requests end up the task in a faulted state. I get “Unexpected end of MIME multipart stream. MIME multipart message is not complete.” Any ideas why?

    • Filip W

      The reason here is probably, and that’s something I have to assume without more details, is that you have something else aside from the controller reading the body. Since request’s body can only be read once (it’s a stream after all), a second time you try to read the body it would fail.

      More info here http://aspnetwebstack.codeplex.com/workitem/166.

      And example workaround:
      Stream reqStream = this.Request.Content.ReadAsStreamAsync().Result;
      if(reqStream.CanSeek)
      {
      reqStream.Position =0;
      }

      • Karl Smith

        Am I just putting this in the beginning of the controller’s action? I’m using MVC 4 and it is still giving me the same error.

        • bloodgroove

          Any progress on this? I’m fighting the same headache.

          • http://www.strathweb.com/ Filip W

            Progress on what?

  • BK
  • BK

    Thanks for your response Filip! Where do I stick that code in? I tried to stick that in before this, but it didn’t work:

    var task = Request.Content.ReadAsMultipartAsync(provider).ContinueWith(t =>

  • TonyTons

    Hello; I need to know the best way to run a file upload from an entity model generated class; all i want to store is the image’s url after successful upload, working on a webapi project, but i need the client to handle normal uploads still.

  • John

    Thanks for the great post. Each time I run the simple example from a form post (target /api/fileupload, I get a 500 error. Strange thing is the file uploads correctly, so the exception must be happening after the file is posted. Anyone else getting this?

  • justin

    This doesn’t work at all for me. In ContinueWith() the task object comes back as faulted and therefore no file’s are written to disk and it complains that the multi form mime type ended unexpectedly. Further research into it leads me to believe this was a bug fixed back in June related to the stream provider expecting /CR /LF at the end of the http post which isn’t required in the HTTP spec.

  • justin

    But yeah, long story short, I got the RTM release shipped with Visual Studio 2012 and I am having heck of a time figuring out how to get the latest bits (bc currently I am running into dependency hell and visual studio whining about configuration versions not matching etc…)

    • Filip W

      Make sure you don’t return void from the method, but Task<HttpResponseMessage>

  • Ian Welsh

    Great series of posts on WebApi, Thanks.

    I’d like to change the signature of your post method to send a complex type that holds data relating to the uploaded file eg. public Task Post(Info fileMetaData)

    Is this possible? I’ve been banging my head against the wall with this.

  • jasenko

    hello,

    I’m having hard time with retrieving file name. I was following your guide but on FileData i’m getting error that CustomMultipartFormDataStreamProvider doesn’t containt definition (missing reference). What am i doing wrong?

  • jasenko

    hello,

    i’m having hard time with retrieving file name. I was following guide but on FileData i’m getting error “CustomMultipartFormDataStreamProvider doesn’t contain definition…(missing reference)”. when i open your project everything works fine. What am i missing?

    Thank you

  • andre van der plas

    Hi Filip,

    I am trying out the first example code (the gist one).
    I am posting an image (jpeg of 311 Kb) using Fiddler, and although the post doesn’t throw any exceptions, the image is only partly there. Looks like not all the image data is written to the file system.

    I found some other code that has ‘async Task’ in the controller’s signature. With that signature the entire image does get uploaded to the file system.

    Any clues on this one??

    –André

  • don

    Thank you. This is exactly what I needed. Thank you for your work.

    • http://www.strathweb.com/ Filip W

      Thanks!

  • Pingback: Multipart form POST using ASP.Net Web API | BlogoSfera

  • http://www.mikeroosa.com Mike Roosa

    This code is working great on my local pc, but when I deploy to a server running IIS6 and ASP.NET 4.0, I’m getting a 500 internal server error. Any ideas what that could be?

  • B. Clay Shannon

    Thanks for this article! You’re missing a “}” above the “else” in the first snippet.

  • B. Clay Shannon

    The .NET client upload code would be great except that MultipartFormDataContent does not seem to be available for my purposes (Compact Framework) http://msdn.microsoft.com/en-dus/library/system.net.http.multipartformdatacontent(v=vs.110).aspx

  • Pingback: Web API File Upload, Single or Multiple files | Software Engineering