Set Up NGINX Reverse Proxy for WebSocket Application in Elastic Beanstalk

Preface

In general, WebSocket uses another port against a normal HTTP application port. It’s possible to use multiple ports on Elastic Beanstalk (EB) via ebextensions [1].

In my test environment, I have a Node.js application with 2 listening ports, one for a normal web application, another for WebSocket. In order to separate these 2 ports, I use ebextensions [1] to overwrite NGINX configuration in Beanstalk’s EC2. The way to retrieve the default configuration files is to log into EC2 console and find out the files.

According to the proxy server configuration document [2], I did some tests and the configuration below works in my environment no matter you choose Amazon Linux 2 (current default version) as your operating system or Amazon Linux AMI (the fisrt version, has been deprecated).

NGINX configuration for EB (Amazon Linux 2)

My Node.js runs on port 8080, and WebSocket runs on port 40510. In order to overwrite default, I add 2 new files:

file: ${source_root_folder}/.platform/nginx/nginx.conf

user                    nginx;
error_log               /var/log/nginx/error.log warn;
pid                     /var/run/nginx.pid;
worker_processes        auto;
worker_rlimit_nofile    32650;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    include       conf.d/*.conf;

    map $http_upgrade $connection_upgrade {
        default     "upgrade";
    }

    server {
        listen        80;
        access_log    /var/log/nginx/access.log main;

        client_header_timeout 60;
        client_body_timeout   60;
        keepalive_timeout     60;
        gzip                  off;
        gzip_comp_level       4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        # Include the Elastic Beanstalk generated locations
        include conf.d/elasticbeanstalk/*.conf;
    }
}

file: ${source_root_folder}/.platform/nginx/conf.d/001.conf

upstream nodejs {
    server 127.0.0.1:8080;
    keepalive 256;
}
upstream ws {
    server 127.0.0.1:40510;
    keepalive 256;
}

server {
    listen 80 default_server;

    access_log  /var/log/nginx/access.log  main;

    location /websocket {
        proxy_pass  <http://ws>;
        proxy_http_version 1.1;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_cache_bypass $http_upgrade;
    }
    location / {
        proxy_pass  <http://nodejs>;
        proxy_http_version 1.1;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    gzip on;
    gzip_comp_level 4;
    gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
}

After this modification, you can create an application source bundle [3] and upload it to the Elastic Beanstalk.


NGINX configuration for EB (Amazon Linux AMI)

If you are using Amazon Linux AMI (the first version, has been deprecated) as your operating system, I recommend you to migrate to new version (Amazon Linux 2) [4] as soon as possible. Although, here are the configurations:

My Node.js runs on port 8081, and WebSocket runs on port 40510. In order to overwrite default configuration, I add a new file in ${source_root_folder}/.ebextensions/proxy.config

files:
  /etc/nginx/conf.d/proxy.conf:
    mode: "000644"
    owner: root
    group: root
    content: |
      upstream nodejs {
          server 127.0.0.1:8081;
          keepalive 256;
      }
      upstream ws {
          server 127.0.0.1:40510;
          keepalive 256;
      }

      server {
          listen 8080;

          access_log  /var/log/nginx/access.log  main;

          location /websocket {
              proxy_pass  <http://ws>;
              proxy_http_version 1.1;
              proxy_set_header        Host            $host;
              proxy_set_header        X-Real-IP       $remote_addr;
              proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection 'upgrade';
              proxy_cache_bypass $http_upgrade;
          }
          location / {
              proxy_pass  <http://nodejs>;
              proxy_http_version 1.1;
              proxy_set_header        Host            $host;
              proxy_set_header        X-Real-IP       $remote_addr;
              proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
          }

          gzip on;
          gzip_comp_level 4;
          gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
      }

  /opt/elasticbeanstalk/hooks/configdeploy/post/99_kill_default_nginx.sh:
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash -xe
      rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
      service nginx stop
      service nginx start

container_commands:
  removeconfig:
    command: "rm -f /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf"

After this modification, you can create an application source bundle [3] and upload it to the Elastic Beanstalk.

References

[1] Advanced environment customization with configuration files (.ebextensions) https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html

[2] Configuring the proxy server https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/nodejs-platform-proxy.html

[3] Create an application source bundle https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/applications-sourcebundle.html

[4] Migrating your Elastic Beanstalk Linux application to Amazon Linux 2 https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.migration-al.html?icmpid=docs_elasticbeanstalk_console


comments powered by Disqus