Unexpected Errors in Lambda Functions

Previously, we looked at how to debug errors in our Lambda function code. In this chapter let’s look at how to debug some unexpected errors. Starting with the case of a Lambda function timing out.

Debugging Lambda Timeouts

Our Lambda functions often make API requests to interact with other services. In our notes app, we talk to DynamoDB to store and fetch data; and we also talk to Stripe to process payments. When we make an API request, there is the chance the HTTP connection times out or the remote service takes too long to respond. We are going to look at how to detect and debug the issue. The default timeout for Lambda functions are 6 seconds. So let’s simulate a timeout using setTimeout.

Replace our services/notes/get.js with the following:

import handler from "./libs/handler-lib";
import dynamoDb from "./libs/dynamodb-lib";

export const main = handler(async (event, context) => {
  const params = {
    TableName: process.env.tableName,
    // 'Key' defines the partition key and sort key of the item to be retrieved
    // - 'userId': Identity Pool identity id of the authenticated user
    // - 'noteId': path parameter
    Key: {
      userId: event.requestContext.identity.cognitoIdentityId,
      noteId: event.pathParameters.id
    }
  };

  const result = await dynamoDb.get(params);
  if ( ! result.Item) {
    throw new Error("Item not found.");
  }

  // Set a timeout
  await new Promise(resolve => setTimeout(resolve, 10000));

  // Return the retrieved item
  return result.Item;
});

Let’s commit this code.

$ git add .
$ git commit -m "Adding a timeout"
$ git push

Head over to your Seed dashboard, select the prod stage in the pipeline and deploy the debug branch.

On your notes app, try and select a note. You will notice the page tries to load for a couple of seconds, and then fails with an error alert.

You’ll get an error alert in Sentry. And if you head over to the Issues tab in Seed you’ll notice a new error — Lambda Timeout Error.

If you click on the new error, you’ll notice that the request took 6006.18ms. And since the Lambda timeout is 6 seconds by default. This means that the function timed out.

To drill into this issue further, add a console.log in your Lambda function. This messages will show in the request log and it’ll give you a sense of where the timeout is taking place.

Next let’s look at what happens when our Lambda function runs out of memory.

Debugging Out of Memory Errors

By default, a Lambda function has 1024MB of memory. You can assing any amount of memory between 128MB and 3008MB in 64MB increments. So in our code, let’s try and allocate more memory till it runs out of memory.

Replace your get.js with:

import handler from "./libs/handler-lib";
import dynamoDb from "./libs/dynamodb-lib";

function allocMem() {
  let bigList = Array(4096000).fill(1);
  return bigList.concat(allocMem());
}

export const main = handler(async (event, context) => {
  const params = {
    TableName: process.env.tableName,
    // 'Key' defines the partition key and sort key of the item to be retrieved
    // - 'userId': Identity Pool identity id of the authenticated user
    // - 'noteId': path parameter
    Key: {
      userId: event.requestContext.identity.cognitoIdentityId,
      noteId: event.pathParameters.id
    }
  };

  const result = await dynamoDb.get(params);
  if ( ! result.Item) {
    throw new Error("Item not found.");
  }

  allocMem();

  // Return the retrieved item
  return result.Item;
});

Now we’ll set our Lambda function to use the lowest memory allowed and make sure it has time to allocate the memory.

Replace the get function block in your services/notes/serverless.yml.

  get:
    # Defines an HTTP API endpoint that calls the main function in get.js
    # - path: url path is /notes/{id}
    # - method: GET request
    handler: get.main
    memorySize: 128
    timeout: 10
    events:
      - http:
          path: notes/{id}
          method: get
          cors: true
          authorizer: aws_iam

Let’s commit this.

$ git add .
$ git commit -m "Adding a memory error"
$ git push

Head over to your Seed dashboard and deploy it. Then, in your notes app, try and load a note. It should fail with an error alert.

Just as before, you’ll see the error in Sentry. And head over to new issue in Seed.

Note the request took all of 128MB of memory. Click to expand the request.

You’ll see exited with error: signal: killed Runtime.ExitError. This is printed out by Lambda runtime indicating the runtime was killed. This means that you should give your function more memory or that your code is leaking memory.

Next, we’ll look at how to debug errors that happen outside your Lambda function handler code.