If you are building a serverless application on AWS Lambda and Node.js and keep your code in a monorepo you might find this approach to managing and distributing dependencies useful.
Yarn
First, a note on Node package management - this approach relies on using Yarn as a package manager, rather than NPM. The reason for using Yarn is to take advantage of a particular feature: Yarn workspaces.
Yarn Workspaces
Yarn workspaces simplify working with monorepos. A "workspace" is usually a top-level directory in a monorepo. Each workspace defines its own dependencies via a package.json file. Workspaces can then be declared in a root package.json.
Given a directory structure like this:
/cloud-infrastructure
/lambda-a
/lambda-b
/util-logger
package.json
yarn.lock
The monorepo root package.json would look like this:
{
"name": "our-monorepo",
"private": true,
"version": "0.0.0",
"scripts": {
"build": "yarn workspaces run build"
},
"workspaces": [
"util-logger",
"cloud-infrastructure",
"lambda-a",
"lambda-b"
]
}
When you run yarn install, dependencies for all workspaces in a monorepo are hoisted to a shared node_modules folder at the root of the project. This strategy avoids duplication of dependencies and speeds up installations. Workspaces can also depend on other workspaces, making it trivial to share common utilities across your monorepo, rather than having to run something like yarn link.
The order of the workspaces array in your root package.json becomes important if you have workspaces that depend on each other. For instance, in the above example, util-logger is used by lambda-a and needs to be built first so it can be resolved when building lambda-a.
Another useful feature of Yarn workspaces is the ability to run scripts on all workspaces via a single command. In the same example above, we can see a root-level build script - executable via yarn build - that will recursively run the build script in each workspace, in the order defined in the workspaces array.
AWS Lambda Layers
Lambda layers are a really cool feature of AWS Lambda and can enable some powerful abstraction techniques. Among other things, Lambda layers can be used to distribute shared dependencies to your Lambda functions. Dependencies do not need to be shipped with the Lambda deployment package, meaning leaner Lambdas, reduced build times and reduced invocation durations.
This is where they can work super well in a Yarn workspaces monorepo.
Workspaces X Layers
Yarn workspaces and Lambda layers can be combined to provide a seamless dependency workflow across local and remote environments. Locally, we can use Yarn to install dependencies across workspaces and execute our code and run our tests. We can also use Yarn workspaces in our remote CI environments to keep our build and test commands concise; simply executing yarn build and yarn test across the entire application codebase.
Dependency management becomes centralised and standardised.
Another huge benefit is that Lambda code written in TypeScript (or using JavaScript features not supported by Node.js 12.x via Babel) can be compiled without needing a module bundler, like Rollup or Webpack. Each workspace's build process is as simple as running tsc.
Example code
A working example of combining Yarn workspaces with Lambda layers can be found on GitHub. This example implements some of the features discussed in this post plus a few extras!
- Install all monorepo dependencies locally and in CI with a single command:
yarn install
- Build all workspaces locally and in CI with a single command:
yarn build
- Test all workspaces locally and in CI with a single command:
yarn test
- Install only production dependencies (local and 3rd party) to the Lambda Layer:
yarn layer (the magic happens in this script ✨)
- Lambda layer imports
/opt/nodejs/ can be mapped to local directories when - working with TypeScript or Jest
- Keep CI and CD pipelines concise and self-explanatory
- Bonus: Compose Lambdas and layers using the AWS CDK
