Build and Deploy Code AWS CI/CD -Codedeploy, Codebuild and Codepipeline

Recently, I had worked on an interesting assignment where I had setup an AWS Codepipeline (CI/CD Solutions) with multiple Github repositories for a mission critical production system which handles nearly 1 million hits a month.

So, they process eCommerce transactions like parking, lounge, hotel and cab booking for airports across the world.

What is the need for continuous integration/deployment (CI/CD)?

Continuous integrations and continuous deployment (CI/CD) are one of the crucial pieces in code automation in today’s world and AWS Codepipeline simplifies the process of uploading a new piece of software and also allows companies to improve their code deployment process.

Let’s understand the need of using a CI/CD pipeline. Every developer of a company can submit their code to code repository like (Github, Bitbucket or AWS Codecommit) from where the code can be directly pulled to the production/staging servers in this process the code can be built and deployed automatically to the servers. The deployment and integration happen automatically, and you do not have to access the servers manually.

You also might be wondering the reason why the server’s direct access to servers is not preferred? The main goal is to prevent accidents and human errors. If all the interactions of the servers are automated, your code is always safe, and you also get inputs for improvisation.

Our Challenge — Bringing code from multiple Github repositories and pushing them to single AWS Auto scaling System

Under standard scenarios, you would get a piece of code, and you would use code deploy and pipeline to integrate it with the AWS server. In this case we we had the code coming from three code repositories, and it had to be brought together and pushed to a single autoscaling system with blue/green deployments.

Our Resolution/Success

The best solution, in this case, is the use of AWS pipeline, which is a service that helps you to integrate multiple AWS code services like AWS Codebuild and AWS Codedeploy and our source was Github.

PS: For setting up AWS Codedeploy, You’ll need to setup a AWS Codedeploy/EC2 role. You can follow the tutorial here for setting up the role.

Step 1: Setting up AWS Code Deploy

In step, we would setup a basic code pipeline architecture which will be tweaked later to include multiple Github repositories and AWS Codebuild.

You’ll need to setup AWS CodeDeploy first which will be required for setting up AWS Codepipeline. So, we had a mission critical production system which can’t have any downtime. So, I had went for Blue/green deployment and an Autoscaling group.

Step 2: Basic Pipeline Architecture with AWS Codepipeline

When you setup AWS Codepipeline, it would ask you to add single source. You can select your first Github repository from here and follow the next step which would show you the Github Build. You can skip the Github Build stage for now and move over to the next step of selecting a deploy stage.

Important: You would need an appspec.yml file in your Github repo. This file contains various instructions like location in the servers where the code will be pushed, scripts to run post or pre run for AWS Codepipeline.

Step 3: Bringing multiple Github sources to AWS Codepipeline

Adding two new sources to AWS codepipeline. So, in order to add this just edit the pipeline and add another Github repository as the source.

Similarly, you can add another stage to the Codepipeline and this stage would be our Build stage. Here in the build stage, you’ll select AWS CodeBuild as the service and your output artifacts of Github (source) stage would be input artifacts of AWS CodeBuild. Now, your single output artifact of codebuild would be input artifact for AWS Codedeploy (which would be our next stage).

Important: You would need an buildspec.yml file in your main Github repo. This file contains various instructions like location where your sources would be combined and any build commands that needs to be run for AWS CodeBuild.

The final step was using AWS’s service called AWS code deploy, which integrated the combined code and pushed it to the servers. The integration with the server is automated, and we practically completed the whole process without any accident/problem. Code Build and Code Deploy, along with CI/CD, is a significant milestone in improving the user experience for customers by Amazon!

Voila! Your pipeline is ready to pull code from multiple Github repos and push them to EC2 autoscaling system.

Final Thoughts

You would find many options for building robust CI/CD solutions we’ve found AWS CodeBuild and CodePipeline to be rich in functionality and exceedingly cost effective.

If you want complete control of your continuous integration and delivery environment, you run apps and infrastructure in AWS and you’d like to limit the number of third party providers accessing your data.

I’m an AWS Certified Architect. You can reach me on gaurav.ideabox@outlook.com