Getting Lambda to run Python3 scripts with dependencies

I'm currently rebuilding my Dad's website with the intent of adding an order form so he can sell his wine (it's not live yet though).

I've taken the opportunity to try and get my head around VueJS which is currently holding up very well and I now have fantasies of hosting the site in an S3 bucket as there is pretty much no need for a server.

The only two bits of server-side functionality required are processing the payments and subscribing to the newsletter. While it would be trivial to build a Flask app to take care of all of that, my dreams of a site cheaply hosted in an S3 bucket were vanishing. So I thought I'd try my hand at AWS's Lambda offering.

Basically, Lambda "lets you run code without provisioning or managing servers". So if you just need a really simple API endpoint to do one very specific thing (like sign someone up to a newsletter) Lambda is perfect and so cheap it may as well be free.

It took me a while to get my Python3 script running so I thought I'd document the 'gotchas' I found along the way.

So firstly, make sure your Handler is set correctly. It uses pretty standard Python dot paths. The default is lambda_function.lambda_handler. I'm a fan of defaults because they are predictable.

So next we want to make a file in a directory on it's own called, you guessed it, lambda_function.py and inside that file you'll need a function called lambda_handler.

The next tricky bit is dependencies. Basically they all need to be beside your lambda_function.py file. This is a two step process.

Firstly, install your dependencies (in my case, I needed stripe installed):

$ pip3 install --install-option="--prefix=/absolute/path/to/folder/with/script" --upgrade stripe

That will create a bin and lib folder in your directory next to your script.

Now we want to drill down into the lib folder until we get to the site-packages director and move its entire contents to the directory with your script in it.

Update (I actually read the git docs, the above is unnecessarily complicated.)

I you just want to install them directly into the directory you're currently in use the following:

$ pip install -t . --upgrade stripe

That folder should now look a bit like:

...
/__init__.py
/lambda_function.py
/certifi
/certifi-2017.7.27.1-py3.6.egg-info
/stripe
/stripe-1.62.0-py3.6.egg-info
/urllib3
/urllib3-1.22-py3.6.egg-info
...

I'm not sure if the __init__.py is required or not but I put it in there anyway.

The final step is to zip up the files so that when they are extracted they aren't in their own directory (which is the default behaviour on macOS at least). I delete the bin and lib folders first but they probably do no harm:

$ cd path/to/lambda_function
$ zip -r path/to/zip_location.zip .

BY zipping it up this way it ensures that when the files are extracted the file path will match the python path defined as your Handler earlier. I'm sure it's possible to pop all the files in an src directory and change the Handler to src.lambda_function.lambda_handler or something like that but I think this way is neater.

For more documentation I recommend AWS's Getting Started which will flesh out plenty I haven't fleshed out here.

Show Comments