DevOps is an organizational model that combines development and operations teams to enable quicker application development and more straightforward maintenance of current installations. DevOps promotes shorter, more controlled iterations by assisting firms in strengthening relationships between Dev, Ops, and other firm stakeholders through the adoption of best practices, automation, and new tools. This project aims at creating a solution for a company that uses DevOps practices to improve its software lifecycle. They want a solution that will perform orchestration from one cloud to another using this DevOps Pipeline.
BUSINESS NEED
This solution aims at a use case where a company uses AWS and Azure as its cloud providers. This company uses Azure DevOps to manage its software lifecycle. They do not want to use Jenkins or Team City to build or orchestrate their infrastructure; instead, they want to leverage Terraform to provision their infrastructure using automation.
SOLUTION
To implement this use case, we need to create an infrastructure as code using Terraform, where we will provision multiple AWS services (EC2, S3, Security Group). We will be automating the infrastructure development setup for AWS using the Azure DevOps Pipeline. We will also use S3 buckets to store Terraform state information, and DynamoDB tables will provide lock capability while performing operations. Our code repository will be GitHub, and we will link it with the Azure DevOps Pipeline.
CI/CD Continuous Integration & Deployment with DevOps
Using Azure DevOps, we create various jobs, which will be initiated as soon as we make a release. Each job will define multiple stages of Terraform states such as "init," "planning," "applying," and "destroying." After these jobs are run, another job will deploy the services inside AWS. After the project is complete, another release will delete the resources from AWS.
Security Architecture
From a security perspective, the data coming from the VCS (GitHub) will be encrypted by Azure. Azure allows us to integrate different VCSs into its DevOps platform. We must sign in using our GitHub credentials to establish a secure connection.
When the code is executed in Azure, the deployment will be carried out on AWS. To access the AWS account, the AWS toolkit allows Azure DevOps to sign in to an AWS account with programmatic access (Access Key & Secret Key)
Solution Architecture
The DevOps team will access the Terraform code from GitHub and create jobs inside the release pipeline.
backend.tf
terraform {
backend "s3" {
bucket = "mydev-tf-state-bucket"
key = "main"
region = "us-east-2"
dynamodb_table = "my-dynamodb-table"
}
}
Jenkinsfile.tf
pipeline {
agent any
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage ("terraform init") {
steps {
sh ("terraform init -reconfigure")
}
}
stage ("plan") {
steps {
sh ('terraform plan')
}
}
stage (" Action") {
steps {
echo "Terraform action is --> ${action}"
sh ('terraform ${action} --auto-approve')
}
}
}
}
main.tf
provider "aws" {
region = var.aws_region
}
resource "aws_vpc" "main" {
cidr_block = "172.16.0.0/16"
instance_tenancy = "default"
tags = {
Name = "main"
}
}
#Create security group with firewall rules
resource "aws_security_group" "jenkins-sg-2022" {
name = var.security_group
description = "security group for Ec2 instance"
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# outbound from jenkis server
egress {
from_port = 0
to_port = 65535
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
tags= {
Name = var.security_group
}
}
resource "aws_instance" "myFirstInstance" {
ami = var.ami_id
key_name = var.key_name
instance_type = var.instance_type
vpc_security_group_ids = [aws_security_group.jenkins-sg-2022.id]
tags= {
Name = var.tag_name
}
}
# Create Elastic IP address
resource "aws_eip" "myFirstInstance" {
vpc = true
instance = aws_instance.myFirstInstance.id
tags= {
Name = "my_elastic_ip"
}
}
s3.tf
resource "aws_s3_bucket" "my-s3-bucket" {
bucket_prefix = var.bucket_prefix
acl = var.acl
versioning {
enabled = var.versioning
}
tags = var.tags
}
sonar.tf
resource "aws_instance" "mySonarInstance" {
ami = "ami-0b9064170e32bde34"
key_name = var.key_name
instance_type = "t2.micro"
vpc_security_group_ids = [aws_security_group.sonar-sg-2022.id]
tags= {
Name = "sonar_instance"
}
}
resource "aws_security_group" "sonar-sg-2022" {
name = "security_sonar_group_2022"
description = "security group for Sonar"
ingress {
from_port = 9000
to_port = 9000
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# outbound from Sonar server
egress {
from_port = 0
to_port = 65535
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
tags= {
Name = "security_sonar"
}
}
# Create Elastic IP address for Sonar instance
resource "aws_eip" "mySonarInstance" {
vpc = true
instance = aws_instance.mySonarInstance.id
tags= {
Name = "sonar_elastic_ip"
}
}
variables.tf
variable "aws_region" {
description = "The AWS region to create things in."
default = "us-east-2"
}
variable "key_name" {
description = " SSH keys to connect to ec2 instance"
default = "myJune2021Key"
}
variable "instance_type" {
description = "instance type for ec2"
default = "t2.micro"
}
variable "security_group" {
description = "Name of security group"
default = "jenkins-sgroup-dec-2021"
}
variable "tag_name" {
description = "Tag Name of for Ec2 instance"
default = "my-ec2-instance"
}
variable "ami_id" {
description = "AMI for Ubuntu Ec2 instance"
default = "ami-020db2c14939a8efb"
}
variable "versioning" {
type = bool
description = "(Optional) A state of versioning."
default = true
}
variable "acl" {
type = string
description = " Defaults to private "
default = "private"
}
variable "bucket_prefix" {
type = string
description = "(required since we are not using 'bucket') Creates a unique bucket name beginning with the specified prefix"
default = "my-s3bucket-"
}
variable "tags" {
type = map
description = "(Optional) A mapping of tags to assign to the bucket."
default = {
environment = "DEV"
terraform = "true"
}
}
As the Terraform code is executed, the Terraform state will be stored inside an S3 bucket that is already created; it uses a LockID in DynamoDB. The provisioning of resources is carried out on the AWS account.
Install Terraform Task on Azure DevOps Pipeline
Terraform INIT stage on Azure DevOps Pipeline
Terraform Plan stage on Azure DevOps Pipeline
Terraform Apply stage on Azure DevOps Pipeline
After this release pipeline is launched, a Windows server is assigned, and all the tasks will be executed. It will initialize the job, download artifacts, install Terraform, run Terraform init, plan, apply, and finish the job.
After all, this has been completed, and the resources will be created on your AWS Console.