Running Software in Production
January 2, 2021β’13 words
Anything can go wrong once. Nothing should be allowed to go wrong twice.
L
A digital garden; finding chaos in the systems that surround us
January 2, 2021β’13 words
Anything can go wrong once. Nothing should be allowed to go wrong twice.
January 2, 2021β’14 words
Choose two.
July 2, 2020β’34 words
June 1, 2020β’60 words
By the end of this month (June 2020) I will have been pushing code to GitHub for 8 years. I came across this neat tool for charting the history of your GitHub contributions.
Here's how my 8 years looks:
This tells a number of stories: most obviously, for the first year I didn't make any contributions π
June 1, 2020β’643 words
If you are building a serverless application on AWS Lambda and Node.js and keep your code in a monorepo you might find this approach to managing and distributing dependencies useful.
First, a note on Node package management - this approach relies on using Yarn as a package manager, rather than NPM. The reason for using Yarn is to take advantage of a particular feature: Yarn workspaces.
Yarn workspaces simplify working with monorepos. A "workspace" is usually a top-level directory in a monorepo. Each workspace defines its own dependencies via a package.json
file. Workspaces can then be declared in a root package.json
.
Given a directory structure like this:
/cloud-infrastructure
/lambda-a
/lambda-b
/util-logger
package.json
yarn.lock
The monorepo root package.json
would look like this:
{
"name": "our-monorepo",
"private": true,
"version": "0.0.0",
"scripts": {
"build": "yarn workspaces run build"
},
"workspaces": [
"util-logger",
"cloud-infrastructure",
"lambda-a",
"lambda-b"
]
}
When you run yarn install
, dependencies for all workspaces in a monorepo are hoisted to a shared node_modules
folder at the root of the project. This strategy avoids duplication of dependencies and speeds up installations. Workspaces can also depend on other workspaces, making it trivial to share common utilities across your monorepo, rather than having to run something like yarn link
.
The order of the workspaces
array in your root package.json
becomes important if you have workspaces that depend on each other. For instance, in the above example, util-logger
is used by lambda-a
and needs to be built first so it can be resolved when building lambda-a
.
Another useful feature of Yarn workspaces is the ability to run scripts on all workspaces via a single command. In the same example above, we can see a root-level build script - executable via yarn build
- that will recursively run the build
script in each workspace, in the order defined in the workspaces
array.
Lambda layers are a really cool feature of AWS Lambda and can enable some powerful abstraction techniques. Among other things, Lambda layers can be used to distribute shared dependencies to your Lambda functions. Dependencies do not need to be shipped with the Lambda deployment package, meaning leaner Lambdas, reduced build times and reduced invocation durations.
This is where they can work super well in a Yarn workspaces monorepo.
Yarn workspaces and Lambda layers can be combined to provide a seamless dependency workflow across local and remote environments. Locally, we can use Yarn to install dependencies across workspaces and execute our code and run our tests. We can also use Yarn workspaces in our remote CI environments to keep our build and test commands concise; simply executing yarn build
and yarn test
across the entire application codebase.
Dependency management becomes centralised and standardised.
Another huge benefit is that Lambda code written in TypeScript (or using JavaScript features not supported by Node.js 12.x via Babel) can be compiled without needing a module bundler, like Rollup or Webpack. Each workspace's build process is as simple as running tsc
.
A working example of combining Yarn workspaces with Lambda layers can be found on GitHub. This example implements some of the features discussed in this post plus a few extras!
yarn install
yarn build
yarn test
yarn layer
(the magic happens in this script β¨)/opt/nodejs/
can be mapped to local directories when - working with TypeScript or JestMay 15, 2020β’34 words
May 4, 2020β’47 words
January 15, 2020β’4 words
Hello, world! π π