Once we decided to adopt serverless and started with early versions of Purple Stack, there was a consensus on using the mono repo organized code structure and Lerna was definitely a clear choice.

Although it's a powerful tool, we faced unanticipated requirements, such as missing systems to check the execution order or execute-only for what had changed since the last run.

Unfortunately, there were no solutions for the same issues, so we had to smarten it with our lerna-smart-run addon.😎

GitHub - purple-technology/lerna-smart-run: Lerna add-on for only running npm script for packages changed since previous tag

Lerna add-on for only running npm script for packages changed since previous tag - GitHub - purple-technology/lerna-smart-run: Lerna add-on for only running npm script for packages changed since p...

GitHubpurple-technology

Don't deploy it again please!

One of our main challenges in Purple Stack was slow CI/CD process. As our team is growing, we need to optimize the pipelines to maximize the efficiency and running deployment for every service, even if no code changes are made, slows down our process a lot.

Hence, we've implemented a feature that creates a git tag for every successful deployment. So every deployment looks for changes since the last tag (including local dependencies) and deploys only services that have changed.

You need to add --tagOnSuccess option:

$ smartRun <script> --tagOnSuccess

Execution order

The other issue was related to packaging order and sequential execution. Typically we need to deploy storage resources before we deploy the api and frontend.

To resolve this, we implemented the --runFirst or --runLast command options which we can use depending on the ordering needs.

$ smartRun <script> --runFirst="first*" --runFirst="second" --runLast="last"

Example 🚀

The best example of usage can is the preview branch model in CI, which contains all the features mentioned above, and it's something you will come across in practice.

For better visualization, the following flow chart illustrates what you can achieve with this addon.

Let's following application structure:

• ./api/package.json - {"name": "api"}
• ./backend/resources/primary/package.json - {"name": "@be-prioritized/primary"}
• ./backend/resources/secondary/package.json - {"name": "@be-prioritized/secondary"}
• ./backend/services/a/package.json - {"name": "@be/a"}
• ./backend/services/b/package.json - {"name": "@be/b"}
• ./frontend/package.json - {"name": "frontend"}

Firstly, we need to define the Lerna packages in lerna.json file. You can run lerna init command or create the following file manually.

./lerna.json

{
	"packages": ["api", "backend/resources/*", "backend/services/*", "frontend"],
	"version": "1.0.0"
}

Then, we define all required run scripts in all defined packages we need to run.

./api/package.json

{
    "name": "api",
    "scripts": {
        "deploy": ... // Your deploy script here
    }
}

As the final step, add global run script with the defined order in the root dir:

./package.json

{
	"scripts": {
		"deploy": "smartRun deploy --runFirst='@be-prioritized/*' --runFirst='@be/*' --runFirst='api' --runLast='frontend' --tagOnSuccess",
		"delete-tag": "smartRun --deleteTagOnSuccess"
	},
	"devDependencies": {
		"@purple/lerna-smart-run": "^0.3.1",
		"lerna": "^3.22.1",
	}
}

As you can see we defined --runFirst='@be-prioritized/*' and --runFirst='@be/*' - it means that you can use glob patterns for choosing multiple packages as a group so deployment order expressed by package.json file path will be:

1. ./backend/resources/primary/package.json
2. ./backend/resources/secondary/package.json
3. ./backend/services/a/package.json
4. ./backend/services/b/package.json
5. ./api/package.json
6. ./frontend/package.json

There is also an option --deleteTagOnSuccess which simply looks for the most recent tag generated by this package and deletes it if smart run execution was successful. It can be very useful for preview deployments. After you merge features or delete your feature branch and still want to keep your history clean.

Implementation in CI can look like in the following Circle CI example:

version: 2.1
references:
  feature_branch_filter: &feature_branch_filter
    branches:
      only:
        - /feature\/.*/
        - /hotfix\/.*/
        - /fix\/.*/
        - /h\/.*/
        - /f\/.*/
jobs:
  deploy:
    steps:
      - deploy:
          name: Deploy
          command: npm run deploy

  remove:
    steps:
      - deploy:
          name: Remove
          command: npm run remove && npm run delete-tag
workflows:
  version: 2
  deploy_feature:
    jobs:
      - approve_deploy:
          type: approval
          filters: *feature_branch_filter
      - deploy:
          requires:
            - approve_deploy
          filters: *feature_branch_filter
  remove_feature:
    jobs:
      - approve_remove:
          type: approval
          filters: *feature_branch_filter
      - remove:
          requires:
            - approve_remove
          filters: *feature_branch_filter

Conclusion

As we created this tool for our Purple Stack needs, it's the best example of seeing the addon working in practice, and we hope that it helps you with similar problems that you are tackling.

Feel free to contribute and don't forget to give us a star below!🙂