NOTE: For this post, you should be familiar with the basic concepts of WebSockets and ELB.

Recently, a project required us to implement WebSockets. This posed quite a challenge, because:

• AWS ELB doesn’t support WebSockets on HTTP/HTTPS (Layer 7).
• Switching ELB protocols to TCP/SSL does the trick, but we do not receive X-Forwarded-For header from ELB anymore, i.e. client IP information is not obtained.

The solution we developed addresses both points above and covers the following points:

• WebSockets should work on ELB with TCP/SSL protocols enabled, while client IP information should be retained.
• All the TCP traffic should be redirected to SSL, allowing only secure communication to take place.

Fixing #1: Enable Proxy Protocol on ELB

AWS introduced support for proxy protocol on ELB. After enabling proxy protocol on ELB the X-Forwarded-For can be received even at a TCP/SSL level. Once proxy protocol is enabled, ELB prepends a human-readable header to the request header, which contains connection specific information. For nginx, running on instances behind ELB, to understand this additional header configuration change is essentially needed. We recommend performing the configuration change prior to enabling proxy protocol on ELB, otherwise all requests will return a 400 - Bad Request error.

This post was quite helpful in having required nginx configuration:

Using Proxy Protocol With Nginx

You should be very careful about following configuration changes:

• Define custom log format: Add this log format before including any virtual host configuration.

  log_format elb_log '$proxy_protocol_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent "$http_referer" ' '"$http_user_agent"';
  

• Define default virtual host:

  server {
    listen       80 proxy_protocol;
    server_name _; #Default virtual host
    }

You might have already noticed that the default virtual host listens to proxy_protocol. A rule of thumb for virtual host definitions is to have one default virtual host listening to proxy_protocol. The first virtual host that listens to proxy_protocol returns 400 - Bad Request to all requests received; all subsequent virtual hosts work properly. We’ve filed a bug with nginx for this issue.

• Virtual hosts for application:

  #Virtual host accepting TCP traffic
  
    server {
    listen       80 proxy_protocol;
    server_name mysite.com;
    access_log  logs/mysite.access.log  elb_log;
    location / {
    root   html;
    rewrite ^(.*)$ https://$server_name$1 permanent;
    index  index.html index.htm;
    }
    }
  
    #Virtual host accepting SSL traffic
  
    server {
    listen       81 proxy_protocol;
    server_name mysite.com;
    access_log  logs/mysite.access.log  elb_log;
    ……
    }

The first virtual host accepts TCP traffic and all requests are redirected to SSL. Both virtual hosts listen to proxy_protocol and a custom log format is associated with corresponding access log files.

Currently, this feature can only be enabled through API’s and AWS CLI. Considering the ease of use, we’ll enable proxy protocol on ELB from AWS CLI:

• Create ELB Policy for proxy protocol:

aws elb create-load-balancer-policy --load-balancer-name my-loadbalancer --policy-name EnableProxyProtocol --policy-type-name ProxyProtocolPolicyType --policy-attributes AttributeName=ProxyProtocol,AttributeValue=True

• Set ELB Policy for Backend Servers:For Port 80:

  aws elb set-load-balancer-policies-for-backend-server --load-balancer-name my-loadbalancer --instance-port 80 --policy-names EnableProxyProtocol
  

  For Port 81:

  aws elb set-load-balancer-policies-for-backend-server --load-balancer-name my-loadbalancer --instance-port 81 --policy-names EnableProxyProtocol
  

In above commands, place your load balancer name instead of my-loadbalancer.

Fixing #2: Redirect TCP traffic to SSL

The nginx configuration done in step one, above, takes care of this issue partially. The ELB listeners need to be changed, this can be done through AWS Management Console. Change your settings to the following:

The ELB should be all set to support WebSockets!

Like what you read? Join our community to get more technical information, chances to win prizes, and more: built.io/community