Minimal Setup for AWS API Gateway with Java

Go to the Table of Contents for the Java API Gateway

Although I've been using clouds for the past decade or so, I do not (yet) consider myself an expert with all things serverless.  This year,  I have done some work with AWS API Gateway and Lambdas, but everything I've done has been with node.js.

The node.js environment in Lambda is really quite friendly: you can (for small enough packages) view and even edit your code online.  But it seems fairly obvious that the same approach will not work for Java since "runtime" Java consists of bytecodes and jar files.

The background for this investigation is that I want to take one of my mainline servers, currently running on provisioned AWS boxes with high availability proxies and get it running in a serverless environment.

Of course, because it's a real system, it's a little complicated.  So I need to make sure that I can do all of the following things:

• Actually get something up and running with API Gateway, Lambda and Java
• Obtain all the information sent across from the client (method, headers, path information, payloads, etc)
• Integrate with my existing code which uses custom servlet handlers
• Handle WebSocket traffic (a showstopper a couple of years ago, but available since last December)
• Integrate with complex backend components (specifically Couchbase on a different configured server)

Given that this is quite a few separate things, I'm going to split my treatment up into a series of blog posts (there is a contents page linked from the top of every post).  All the code is in one place - git@github.com:gmmapowell/ignorance-may-be-strength.git - under the directory aws-java-gateway.  Each post has its own git tag: for this post you will want to check out the tag API_GATEWAY_MINIMAL.

Before we Begin

If you want to do more than follow along, you'll obviously need an AWS account and you'll need to have it appropriately configured along with suitable configuration on your machine.  I think most of that is beyond the scope of this article.

You will definitely need to install the AWS CLI.  For MacOS, you can follow these instructions.

The operations we are going to perform need to take place in an AWS region.  You need to specify this to the scripts I have written in the REGION environment variable.  To move the code from your machine to AWS Lambda, we use an S3 bucket.  You need to provide the name of a bucket you have permission to write to in the BUCKET environment variable.

Getting Something Working

The first thing we want to do is to get an AWS API Gateway up and running.  I've done this bit before and - if you know me at all - you will know that there's no way I'm going to do it through the UI other than to experiment.  So we're going to start by using CloudFormation to build an API Gateway.  If you've not used CloudFormation before, this is a good time to start.  I should probably write a separate blog about that, but I haven't yet.

You can invoke CloudFormation in a number of ways, but for simplicity I'm just going to invoke it from the command line.  Everything we're going to do here is wrapped up in a couple of scripts (createGateway.sh and dropGateway.sh) and they reference a CloudFormation description (gateway-cf.json).  You can specify CloudFormation in either JSON or YAML; YAML is more terse, but I tend to find it's harder to read.  Moreover, it is less "machine-friendly": it's harder to parse and generate (at least with languages I tend to use).

You would think that it wouldn't be that hard to get a simple gateway up and running, and it's probably not, but by the time you have all the plumbing in place, you'll find you've created:

• An execution role for the lambda function we're going to want to execute
• The lambda function itself
• A "permission" that allows api gateway to invoke the lambda
• The gateway itself
• A resource on the gateway
• A method on the resource
• A "deployment" for the gateway (i.e. where it shows up in the real world)

Running the createGateway.sh script creates all these resources which you can confirm in the UI if you'd like.

Buried deep in there is the fact that we are pulling a zip file from S3 in the hopes that it will have some useful Java resources in it.  If you look, you'll see that zip file is built earlier on in the script with just junk in it.  For now, what is in the zip file is unimportant: it just matters that it is a zip file.

One thing I do find very useful to do in the UI is testing.  It may well be possible to do the same level of testing with the same quality of output from the command line, but if so, I don't know how.  Even if it is, the ability to see and scroll through the output makes the UI worthwhile.

To test the gateway in the UI, start on the CloudFormation page in the console. Select the Resources and identify the gateway (called SimpleGateway).  Click on the named link and you will go to the API Gateway page.  Click on through to the appropriate gateway and you can see the resources.  Select the GET method on the left hand side and press "TEST".  Scroll down and press "TEST" at the bottom.  On the right hand side, you'll see a testing pane appear.  At this point, I'd expect you to see an error something like this:

Lambda execution failed with status 200 due to customer function error: Class not found: blog.ignorance.apigateway.Handler

If noted above,  we created a zip file from the scripts directory - there is no Java in sight!  It's not surprising AWS can't find a handler.

Which is really the point of this blog post - and what we have been building up to.  We need to create some Java, put it in the right place and see if it works.

Creating a Java Handler for a Lambda

There are a number of different ways of writing Java Lambda Handlers and the exact technique you choose depends on what you are trying to do.  The options are outlined in the AWS Java Documentation, but I have to say that I didn't myself come away very clear on how any of it worked - hence these experiments here.

But after considerable research, and finding madhead's repository, I realized that with a Lambda Proxy integration, the POST packet from API Gateway gives you a serious quantity of information that you can pull into a POJO.  This uses the technique that AWS describes as Leveraging Predefined Interfaces for Creating Handler (Java).

So, let's begin to write some code.

First, we need a class.  If you look back carefully at what we created, you will see that the handler for the lambda was defined to be blog.ignorance.apigateway.Handler.  The plethora of options in how to define a Handler seems complicated, but for the case I want to pursue, this appears to be the name of a class that implements the com.amazonaws.services.lambda.runtime.RequestHandler interface.

Let's create that class, and provide a minimal implementation of the required Request and Response objects and call our work here "done".

public class Handler implements RequestHandler<IgnorantRequest, IgnorantResponse> {
  @Override
  public IgnorantResponse handleRequest(IgnorantRequest arg0, Context arg1) {
    return new IgnorantResponse();
  }
}

We can now build this code and update the lambda with the appropriate zip file using the scripts/package.sh script.  This script does four things:

• Downloads all the necessary libraries from maven
• Compiles and assembles a lambda zip file
• Uploads the file to S3
• Calls aws lambda update-function-code to notify AWS to refresh the lambda code from the S3 bucket 

Note that this final step is very important - without it, your lambda will keep on using the same code it has always had.

With all the code in place, it should be possible to test the handler again with hopefully delightful results:

Thu Dec 05 10:09:10 UTC 2019 : Successfully completed execution
Thu Dec 05 10:09:10 UTC 2019 : Method completed with status: 200

It doesn't do anything useful yet, but in addition to testing in the UI, you should also be able to hit it from the command line or a browser using the URI that was printed during the creation of the gateway.

curl https://tctf8f9r45.execute-api.us-east-1.amazonaws.com/ignorance/hello
{"message":"Forbidden"}

Oh, well, such is life.

Cleaning Up

There is a script to clean up as well: scripts/dropGateway.sh.

Conclusion

So ends the first part of this tutorial.  I have created (and committed to github) a very simple project that builds the minimal resources to deploy an AWS API Gateway backed by a very simple Java Lambda.

What I haven't done yet (apart from being able to access it from outside the test environment) is to access any of the fields coming from AWS, do any useful work, integrate with anything else or return anything to the user.

We clearly have more work to do.

Next: Let's figure out that 403