Return to site

Scalable Websockets with API Gateway & Serverless magic

Websocket + API Gateway + Lambda + DynamoDB = scaleable awesomeness!

· Serverless
A preview of the live map on

[ final product ]

We needed to create a live map of search requests to Not wanting to bloat the API codebase, and wanting to iterate fast - a Serverless app was the natural choice. It turned out to be easier than expect, and users love the result.

[ How the new Serverless app connects to the existing API application ]

The API pushes a message for each request* into the Serverless app using an SQS queue, this allows easy throttling, fault tolerance and scaling, by decoupling the producer from the consumer.

*As theCatAPI gets millions of requests a day, I've throttled it to push one at random every 6 seconds.

API Gateway's new Websocket feature!

AWS’s API Gateway has recently added Websocket support, as has the framework. Together this makes a compelling option to build an almost infinitely scalable system to broadcast out messages, with very little development time.

Using the framework I created this stack by describing it in a serverless.yml file. Then finished it off by assigning a custom domain & SSL cert in the AWS console.

Here’s an example of the one i used: serverless.yml
  1. Clients connect via Websocket to API Gateway & given a unique connectionId

  2. $connect event triggers a Lambda function which saves connectionId to DynamoDB

  3. If a client disconnects, $disconnect event triggers a Lambda function which deletes connectionId

  4. A new message (Image & location data) from the SQS Queue triggers a Lambda function….

  5. ... which gets all the connectionIds from DynamoDB…

  6. and sends each the message via ‘AWS.ApiGatewayManagementApi.postToConnection()’ using it’s connectionId

  7. Clients receive message via websocket and show Image on map to user.

Key points:

  • Setup API Gateway as Websocket

  • Assign a Lambda function for $connect, $disconnect, $default

  • Unique connectionId assigned for each client by API Gateway, picked up in Lambda function via ‘event.requestContext.connectionId’

  • Save to DynamoDB so the Websocket connectionId is available to any other Lambda function wishing to send them messages.

  • Use AWS.ApiGatewayManagementApi to make callback to each client. As of Apr 2019 it is not in the AWS SDK automatically available in the Lambda runtime, so you have to include it in your bundle, or add it as a Layer.

  • If AWS.ApiGatewayManagementApi.postToConnection() returns an error with statusCode 410 it means the client has disconnected, so delete the connectionId from the DB

Limitations & things to be aware of:

  • Despite my best efforts, i was unable to add a path to the url which could be used to filter by topic. This seems to be a restriction. I got around this by having one subdomain for each topic (less than ideal).

  • Timeouts, max (2 hours) inactive(10 mins)

  • A Client changing IP twice causes a disconnect

All Posts

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!