PHP and the AWS Lambda Custom Runtime (Part -1)

Running PHP with AWS Lambda Custom Runtime API

This is an introductory article to executing PHP code within the AWS lambda environment. Although some experience with PHP, AWS, and NodeJS is expected, this is a beginner friendly tutorial.

Having said that, It’s worth mentioning that I will not be covering the intricacies of using the AWS console in this tutorial. If you are brand new to Amazon Web Services, it might be best to check back once you’re comfortable using AWS.

I know you’re here to write some code, so if you’d like to skip ahead feel free to do so. But if you take the few minuets to gain a foundation of these concepts, you’ll thank yourself later.

The goal of this tutorial is to be able to execute PHP code inside of a lambda function. AWS Lambda does not natively support PHP, but in the last year they have exposed an API surface that gives developers the ability to “bring thier own runtime”. Although ultimately, we will be writing a simple “Hello World” PHP function, you may use this tutorial as a base for running any PHP code or PHP framework such as Laravel, inside AWS Lambda.

This tutorial will consist of two major parts. In this first part (Part 1), we will compile the PHP language from source, using an EC2 instance running Amazon Linux, which is the operating system AWS Lambda uses.

After we compile the PHP binary, we’ll add composer to the party, so we can utilize external dependencies in our Lambda code, as well as make use of autoloading.

Finally, we will package up the PHP executable, Composer, Any dependencies, and a few more things in order to create what AWS calls a “layer”. These “layers”, are the bridge AWS gives us to be able to execute code inside of our custom runtime.

We will also be utilizing a “serverless component” from the serverless framework. Although not required to run PHP in a Lambda function, The serverless framework will help us abstract away some of the calls to various AWS services, such as uploading our PHP files to S3, publishing our lambda code, letting us focus more time on the parts that are important to us.

This serverless component allows you to execute a PHP function in AWS lambda via the custom runtime API. This serverless components uses techniques from https://aws.amazon.com/blogs/apn/aws-lambda-custom-runtime-for-php-a-practical-example, from the AWS blog, as well as the great work done by https://github.com/pagnihotry/PHP-Lambda-Runtime figuring out how to bootstrap the PHP runtime.

What you’ll need

An active AWS account, with the ability to create the following resources:

• AWS Cli with API user (Sample JSON role below)
• Serverless framework installed globally (via npm)
• EC2 instance (Running Amazon Linux)
• Lambda Function
• S3 Bucket
• AWS IAM user, (sample policy below)

Setup

Although during the first part of this tutorial we’ll be working almost exclusively on an EC2 instance, its best to install the required tools locally before beginning.

Ensure both the Serverless framework, and AWS SDK are installed on your local machine. The following two commands should return version information:

$ sls -v
Framework Core: 1.52.0
Plugin: 2.0.0
SDK: 2.1.1
$ aws -v
aws-cli/1.16.238
Python/2.7.10
Darwin/18.7.0
botocore/1.12.228

There are plenty of tutorials to get started if you do not get something simliar to the above output. The guide on the Serverless framework’s website covers both, and is my personal favorite:

Installing Serverless Framework:

https://serverless.com/framework/docs/providers/aws/guide/installation/

Once you have confirmed the above two binaries are configured, move on to “Getting Started”

Getting Started

The following step assumes you are familiar with creating Ec2 instances, and know how to ssh into a machine.

Start by creating a new ec2 instance, using the (2018.03.0) version of Amazon Linux (ami-00eb20669e0990cb4). Ensure the PEM bound to the instance has root ssh access, and that port 22 is open to (at least) your local machine.

After the new instance boots up, ssh into the instance.

I will be doing most of the work out the ec2’s home directory, and you are free to do the same, or use any directory that suites you. If you use a directory other than home, just ensure you keep the same directory structure, or be sure to make the needed modifications to any of the below sample commands.

Update system packages, and install necessary dependencies to compile php:

$ sudo yum update -y
$ sudo yum install autoconf bison gcc gcc-c++ libcurl-devel libxml2-devel -y
$ sudo yum upgrade -y

This might take a few minuets.

Download and compile an earlier version of open ssl in order to ensure compatibility with the AWS lambda VM.

Install OpenSSL 1.0

$ curl -sL http://www.openssl.org/source/openssl-1.0.1k.tar.gz | tar -xvz
$ cd openssl-1.0.1k
$ ./config && make && sudo make install
$ cd ~

Download the PHP 7.3 source

$ mkdir ~/php-7-bin
$ curl -sL https://github.com/php/php-src/archive/php-7.3.0.tar.gz | tar -xvz
$ cd php-src-php-7.3.0

Here we will compile the php source downloaded in the previous step, as well as pass the — with-openssl flag to ensure its compiled with the correct open ssl version. This step will take about 10 minuets, so step back, make yourself a cup of coffee, and pray to the c++ gods for no errors.

$ ./buildconf — force
$ ./configure — prefix=/home/ec2-user/php-7-bin/ — with-openssl=/usr/local/ssl — with-curl — with-zlib
$ make install

Once completed, verify the compilation was successful by attempting to output the php cli version information:

$ /home/ec2-user/php-7-bin/bin/php -v

Should output:

PHP 7.3.0 (cli) (built: Dec 11 2018 17:45:29) ( NTS )
Copyright © 1997–2018 The PHP Group
Zend Engine v3.3.0-dev, Copyright © 1998–2018 Zend Technologies

If you get the above output, congratulations! You’re ready to move onto step two.

Creating the lambda runtime layer

Now that we have php compiled in an environment that replicates the lambda runtime, we need to add a few dependencies. Lets start with composer.

If you had poked around at all to see what was downloaded in the previous step, move back into home before proceeding:

$ cd ~

Lets start by creating ourselves a new directory to work in:

$ mkdir -p ~/php-example/bin/
$ cd ~/php-example

Next, lets move the php executable in our working directory to keep things easy:

$ cp ~/php-7-bin/bin/php ./bin

Next, lets get the composer binary, and stick it in our new bin directory:

$ curl -sS https://getcomposer.org/installer | ./bin/php

If successful, the output should read something like this:

All settings correct for using Composer
Downloading…
Composer (version 1.9.1) successfully installed to: /home/ec2-user/php-example/composer.phar
Use it: php composer.phar

Next, lets work on creating our bootstrap file, which will tell lambda how to execute our code with our custom runtime.

from the root of our workspace folder (/home/ec2-user/php-example), create a boostrap file, and make it executable:

$ touch ./bootstrap && chmod +x ./bootstrap

Next, lets install guzzle so we can use it in our boostrap files handler function:

$ ./bin/php composer.phar require guzzlehttp/guzzle

You should now have the following directory structure:

$ ls -la /home/ec2-user/php-example/
| bin/
| — php
| — bootstrap
| — composer.json
| — composer.lock
| — composer.phar
| — vendor/
| — autoload.php
| — guzzlehttp

Next, let’s work on the code necessary to actually bootstrap our environment.

Using vim, or the cli editor of your choice, paste the following contents into the bootstrap file:

#!/opt/bin/php
<?php
// This invokes Composer’s autoloader so that we’ll be able to     use Guzzle and any other 3rd party libraries we need.
require __DIR__ . ‘/vendor/autoload.php’;
// This is the request processing loop. Barring unrecoverable failure, this loop runs until the environment shuts down.
do {
// Ask the runtime API for a request to handle.
$request = getNextRequest();
// Obtain the function name from the _HANDLER environment variable and ensure the function’s code is available.
$handlerFunction = array_slice(explode(‘.’, $_ENV[‘_HANDLER’]), -1)[0];
require_once $_ENV[‘LAMBDA_TASK_ROOT’] . ‘/src/’ . $handlerFunction . ‘.php’;
// Execute the desired function and obtain the response.
$response = $handlerFunction($request[‘payload’]);
// Submit the response back to the runtime API.
sendResponse($request[‘invocationId’], $response);
} while (true);

The bootstrap is the main engine that will drive our example; it can be written in any language that Lambda’s underlying Amazon Linux environment is able to run. Since this is a PHP example, both the custom runtime and the bootstrap script itself are written in PHP, and they will be executed using the PHP binary we compiled earlier for Amazon Linux.

Lambda will place the files from our various layers under /opt, so our /home/ec2-user/php-example/bin/php file will ultimately end up being /opt/bin/php. The #!/opt/bin/php shebang declaration at the top of our bootstrap will instruct the program loader to use our PHP binary to execute the remainder of the code.

Next, we’ll need to implement two functions in order for our code to actually execute. Add the following to the bootstrap file:

function getNextRequest()
{
$client = new \GuzzleHttp\Client();
$response = $client->get(‘http://' .   $_ENV[‘AWS_LAMBDA_RUNTIME_API’] . ‘/2018–06–01/runtime/invocation/next’);
return [
‘invocationId’ => $response->getHeader(‘Lambda-Runtime-Aws-Request-Id’)[0],
‘payload’ => json_decode((string) $response->getBody(), true)
];
}

And

function sendResponse($invocationId, $response) {
$client = new \GuzzleHttp\Client();
$client->post(
‘http://' . $_ENV[‘AWS_LAMBDA_RUNTIME_API’] . ‘/2018–06–01/runtime/invocation/’ . $invocationId . ‘/response’,
[‘body’ => $response]
);
}

If you’ve made it this far, give yourself a pat on the back. We are moments away from finishing our custom runtime layer, and being able to jump into some actual code!

We’ll need to upload everything we’ve created so far as a zip file to lambda, so lets create two zip files, one for our lambda, and one for our vendor dependencies:

zip -r runtime.zip bin bootstrap
zip -r vendor.zip vendor/

Finally, we’ll need a way to actually upload this zip file to AWS, and using the the cli sounds like the easiest option.

First, install the cli:

$ sudo pip install — upgrade awscli

Next, create an IAM user within the AWS web console with full lambda and S3 permissions. Download the key and secret CSV, and use them to configure the AWS cli:

$ aws configure
AWS Access Key ID [None]: ***
AWS Secret Access Key [None]: ***
Default region name [None]: us-east-1
Default output format [None]: json

*Important! — Lambda functions MUST be created in us-east-1!

With the AWS cli configured, we are now ready to publish our layers to lambda:

First, lets publish the runtime:

$ aws lambda publish-layer-version \
layer-name php-73-runtime \
zip-file fileb://runtime.zip \
region us-east-1

Upon success, you should receive a JSON object response, that looks similar to the following:

{
“Content”: {
“CodeSize”: 11586871,
“CodeSha256”: “***”,
“Location”: “***”
},
“LayerVersionArn”: “arn:aws:lambda:us-east-1:869029932727:layer:php-73-runtime:2”,
“Version”: 2,
“Description”: “”,
“CreatedDate”: “2019–11–06T03:48:24.425+0000”,
“LayerArn”: “arn:aws:lambda:us-east-1:869029932727:layer:php-73-runtime”
}

Make note of both the LayerVersionArn, and the LayerArn.

Next, lets publish the zip file containing our vendor dependencies:

$ aws lambda publish-layer-version \
layer-name php-73-vendor \
zip-file fileb://vendor.zip \
    
    region us-east-1

Upon success, you should receive a JSON object response, that looks similar to the following:

{
“Content”: {
“CodeSize”: 217920,
“CodeSha256”: “***”,
“Location”: “***”
},
“LayerVersionArn”: “arn:aws:lambda:us-east-1:869029932727:layer:php-73-vendor:1”,
“Version”: 1,
“Description”: “”,
“CreatedDate”: “2019–11–05T23:19:14.873+0000”,
“LayerArn”: “arn:aws:lambda:us-east-1:869029932727:layer:php-73-vendor”
}

Again, make note of both the LayerVersionArn and the LayerArn.

Congratulations! You’ve just compiled php from source, bootstrapped the lambda environment to run PHP, and created a lambda layer! Next, we’ll get to actually writing some PHP code!