Nginx proxy upload file timeout

Posted on Friday, September 13, 2013


Recently I did a big update of my local network in my home.  I run a few servers on an ESXi 5.0 box I have set up behind a simple netgear router.  

Before redoing my network I would simply port forward certain ports to certain machines.  This worked well when I had one web server, one bind server, one (insert type) server….

But now I have a few different web server doing different things and it becomes annoying to not be able to user port 80 for each one and have the traffic route based on the URL  (my Netgear router is not smart enough to route traffic in this manner).

So I have all my web traffic (port 80) going to a single server that runs nginx.  This nginx server hosts my static pages and forwards a few URLs to other servers (acting as a proxy).




The Problem


The problem I am having is timing out on uploading files.  I have an alfresco server running in my house that the nginx forwards to based on its URL.  It works just fine except when I upload a large file in my test this morning it failed when I tried to upload a 10MiB video.  And I was uploading it from within my house on a Gigabit network that my laptop was wired into, so it should be plenty fast!  But it still failed.

Here is the portion of my nginx.conf file that forwards to the alfresco system (the URLs have been changed to protect the innocent).



  server {
    listen       80;
    listen       8080;
    server_name alf.example.com;
    access_log   /www/log/access.alf.example.com.log  main_fmt;
    error_log    /www/log/error.alf.example.com.log;
    location / {
      proxy_pass  http://192.168.0.180:80;
    }
  }



This is port forwarding any information sent to alf.example.com on port 80 or port 8080 are sent to the server at 192.168.0.180 port 80.


Looking around the web I found this post http://forum.nginx.org/read.php?2,4290 [1] that discussed this problem and gave a few suggestions.

I added two lines to my configuration.  I added client_max_body_size and proxy_read_timeout


  server {
    listen       80;
    listen       8080;
    server_name alf.example.com;
    client_max_body_size 200m;
    proxy_read_timeout 600s;
    access_log   /www/log/access.alf.example.com.log  main_fmt;
    error_log    /www/log/error.alf.example.com.log;
    location / {
      proxy_pass  http://192.168.0.180:80;
    }
  }



I set the client_max_body_size to 200 MiB and the proxy_read_timeout to 600s.  I restarted the nginx service


> sudo /etc/init.d/nginx restart


And that solved my problem uploading 10miB files are no problem now! 


For those curious what my main_fmt log format is, here it is.


  log_format  main_fmt '$remote_addr - $remote_user [$time_local]  $status '
    '"$request" $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';









References
[1]  nginx proxy "504 Gateway Time-Out"
       chrisnx
       Visited 9/2013




2 comments:

  1. I am glad the guide was helpful to you.

    ReplyDelete
  2. Hey patrick, I am in a crazy situation similar to yours but nothing seems to fix my issue Can you help ?

    Architecture:
    User<-->(nginx1.6)<-->(tomcat7.x)<-->DB

    Good news:
    1. Any sized file gets uploaded fine when I throw HTTPD 2.2.4 in front of tomcat.
    2. If i try to upload any sized file directly to tomact, it all juts works

    Bad news:
    Want to replace HTTP by nginx and it give me error on GUI that says -
    > perhaps your browser does not send files correctly OR (not possible, same error on opera, chrome, firefox)
    > session expired OR (not possible, session is valid)
    > there was a server error (only option left)

    Another observation:
    Same result (fail) with or without https.

    What backend uploader we are using?
    gwt file uploader

    Environment:
    Centos 6.x (2.6.32-358.11.1.el6.x86_64)

    Tomcat:
    7.x

    What did i try so far in last few days ? Why are we "almost" certain its nginx configuration issue?
    1. httpd (Apache/2.4.4), another web server works perfect (just for upload) but we dont want to do it for obvious benefits of nginx
    2. if we remove web server altogether, upload works flawlessly.
    3. played with nearly all timeouts on nginx without luck
    4. tried nginx v1.4.1, v1.5.1 without luck (same result)
    5. multiple browsers same result – firefox/chrome windows, firefox/chrome linux
    6. tcpdump and wireshark show data movements for first 60 seconds.
    7. nginx log looks pretty innocent but its not revealing anything meaningful

    ReplyDelete