Unable to post data with WWWForm

I’m working on an application for a client which connects to a ruby on rails website.

They currently have a working webform which allows you to supply a name in a text field, and upload a png file from your hard drive.

The data echoed back from a test script is as follows

-----------------------------1551734562961374366152009334
Content-Disposition: form-data; name="name"

HelloWorld
-----------------------------1551734562961374366152009334
Content-Disposition: form-data; name="save_file"

newXML
-----------------------------1551734562961374366152009334
Content-Disposition: form-data; name="thumbnail"; filename="Snapshot.png"
Content-Type: image/png

--BINARY PNG DATA GOES HERE--
-----------------------------1551734562961374366152009334
Content-Disposition: form-data; name="commit"

Save
-----------------------------1551734562961374366152009334--

The output from the following code snippet

WWWForm webPost = new WWWForm();
webPost.AddField( "name", "Test" );
webPost.AddField( "save_file", "newXML" );
webPost.AddField( "commit", "Save" );
webPost.AddBinaryData( "thumbnail", tex.EncodeToPNG() );

Debug.Log( "Sending.... ");

WWW result = new WWW( "TARGET_URL_GOES_HERE", webPost );
yield return result;		

if( result.error != null )
{
	Debug.Log( "WWW Form Error : " + result.error );
}
else
{
	Debug.Log( "Upload Sent : \n" + result.bytes );
}

Yields the following response

--qoDROVw1U7VgaXtuvfBnkeMhIt58Jz4XKMaGZXt3
Content-Type: text/plain; charset="utf-8"
Content-disposition: form-data; name="name"

Test
--qoDROVw1U7VgaXtuvfBnkeMhIt58Jz4XKMaGZXt3
Content-Type: text/plain; charset="utf-8"
Content-disposition: form-data; name="save_file"

newXML
--qoDROVw1U7VgaXtuvfBnkeMhIt58Jz4XKMaGZXt3
Content-Type: text/plain; charset="utf-8"
Content-disposition: form-data; name="commit"

Save
--qoDROVw1U7VgaXtuvfBnkeMhIt58Jz4XKMaGZXt3
Content-Type: image/png
Content-disposition: form-data; name="thumbnail"; filename="thumbnail.png"

--PNG DATA GOES HERE--
--qoDROVw1U7VgaXtuvfBnkeMhIt58Jz4XKMaGZXt3--

When we redirect the POST operations to the actual script, instead of the CGI echo script, their post data is accepted, and the WWWForm’s data is rejected.

My questions are as follows:

  1. Does it matter that “Content-Type:” precedes “Content-disposition” when Unity sends a request?
  2. Is it a problem that “Content-Type:” is included for the ‘text/plain’ fields?

I don’t think this is a bug, since unity’s WWWForm output is (as best as I can tell) well formed, and that the bug likely exists within the webapp’s CGI parsing. But I wanted to get some feedback from the Unity devs and/or community at large. Maybe you have encountered problems like this before?

UPDATE:
Just in case it is a malformed request that Unity is generating, here is the response from the server when I execute the above code snippet.

Returned data<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>400 Bad Request</TITLE>
</HEAD><BODY>
<H1>Bad Request</H1>
Your browser sent a request that this server could not understand.


Premature end of script headers: ~myuserhome/public_html/appdemo/dispatch.cgi




Additionally, a 500 Internal Server Error
error was encountered while trying to use an ErrorDocument to handle the request.
<HR>
<ADDRESS>Apache/1.3.37 Server at [url]www.mysitehere.com[/url] Port 80</ADDRESS>
</BODY></HTML>

We encountered thus problem as well but I’m not sure which one is at fault here, Unity or Ruby, because I was using a Perl script to accept the POST during initial testing and it worked fine. It was only when we moved over to using Ruby did we have this issue. It appears to happen in cgi.rb when the code is checking the multipart boundary and there is a spurious EOL and EOF error. Since Perl correctly handles the exact same POST data I’d tend to think Ruby is at fault or Perl is more tolerate of the data. I don’t know the exact multipart spec to know if Unity’s request is actually malformed or not. We are probably going to wind up just modifying cgi.rb to accept what Unity is sending.

OK it looks like the problem is that Unity is inserting one extra blank line after the HTTP headers which Ruby is choking on. So the solution seems to be to either modify cgi.rb or change our script to strip out that line from STDIN before creating Ruby’s CG object.

OK this should fix the problem. It probably won’t help the original poster, judging by the date, but for any other unfortunate souls, here it is, call this before creating the CGI object:

require ‘stringio’

s = “”
$stdin.each_line do |l|
if(l == “\r\n”)
break
else
s += l;
end
end
s += $stdin.read
$stdin = StringIO.new(s)

Well that stdin munging doesn’t work in Rails because you can’t get to the stdin soon enough. Also Ruby still doesn’t quite like the POST format even with the initial newline taken out. We would up just sending the POST to a Perl script which then just repackaged the POST and forwarded it along to Rails. Perl is much more forgiving about malformed POST’s so it’ll read whatever crap Unity is outputting and then you can just repost the form data in Perl all nicely formed to Ruby.

We could change the line endings in the WWW implementation, but I’m not convinced that is the correct behavior. According to wikipedia, all headers need to be followed by , which is what we are doing here.

if you run this code in the editor or standalone (which uses curl for processing the headers), does that work for you? What are the line endings reported by that?

CRLF is fine ( see the official format specs: HTTP/1.1: HTTP Message ), including an empty line at the end. Thats the clear specified way of “ending the headers”

Well you’ll just have to test the WWWForm code against Ruby if you want to get it to work with it. I’m inclined to think Ruby is one at fault here, but what are you gonna do? :wink: While munging that newline did cause the straight Ruby script to parse the POST, it still didn’t identify the multipart data entirely correctly, only when repackaging the request with Perl did we get it to work, which was our solution to the problem.

I had a huge problems when calling the (.asmx) web service methods using the WWW class. I solved it, and I think it is important, so I’d like to share it.

When looking at expected SOAP request (in method specification):

POST /TrilleniumWebService/ProductService.asmx HTTP/1.1
Host: danko-pc
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: “http://dankokozar.com/Test
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:soap=“http://schemas.xmlsoap.org/soap/envelope/”>
soap:Body


string



</soap:Body>
</soap:Envelope>

… I concluded that these are the headers I have to add:

Host: danko-pc
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: “http://dankokozar.com/Test

So, I added them to Headers hashtable “by hand”:

Hashtable headers = new Hashtable();
headers.Add("Host", "danko-pc");
headers.Add("SOAPAction", "http://dankokozar.com/Test");
headers.Add("Content-type", "text/xml; charset=utf-8");
headers.Add("Content-Length", byteArray.Length);

The manifestations of my problems were:

  • when calling the web method from Unity IDE, some percentage of calls didn’t get to the server (something like 50% percent)
  • when testing the app from Firefox, I was getting the “400 Bad request” error)
  • when testing from IE, everything worked OK

Now, I believed Firebug all the time when inspecting the headers. But when I checked it with Fiddler, I saw some headers are duplicated

When removing the duplicated ones from my Hashtable, everything worked.

So, my conclusions:

  • Firebug is lying you. It doesn’t give you a hint that you have some duplicated headers, but spits out “400 Bad request” error instead.
  • IE is ignoring duplicated headers.
  • Some headers are overriding other headers (for instance the “Content-Length” header). Some get duplicated (like “Host” header).

Overal conclusion: Be very carefull with headers: use Fiddler to debug your SOAP requests / web service calls.

There are screenshots of Firebug’s and Fiddler’s log in the attach (login to forum to see it).

Hope this helped to someone :slight_smile:

Danko