Merikanto

一簫一劍平生意，負盡狂名十五年

AWS - 11 Cloud Pipeline Components

Veröffentlicht am in
Symbole im Artikel gezählt: 5.6k Wörter Lesezeit ≈ 7 min

In the eleventh post of the AWS series, we’re going to talk about 5 services services that are components of the CI / CD pipeline in the cloud:

  • CodeCommit
  • CodeBuild
  • CodeDeploy
  • CodePipeline
  • CodeStar

CodeCommit

  • Source-control service, host Git-based repositories (AWS’s version of GitHub)
  • CodeCommit stores repos in S3 & DynamoDB

Git to CodeCommit (HTTPS / SSH)

  • Clone
  • Mirror
  • Migrate branches

Monitoring

  • IAM to control & monitor who can have access
  • CT & CW, SNS

CodeBuild

1. Continuous Integration (CI) service

  • Compile sources code
  • Run tests
  • Produce software packages

2. Concepts

  • Build project: defines how CodeBuild will run a build. It includes information such as where to get the source code, which build environment to use, the build commands to run, and where to store the build output
  • Build specification (buildspec.yml): Choose the commands to run at each phase of the build and other settings

3. Features

  • Stores artifacts in S3
  • Integrate CodeBuild into existing CI/CD pipelines using its source integrations, build commands, or Jenkins integration
  • CodeBuild connect with CodeCommit / GitHub / DockerHub / ECR to pull source code for builds
  • Access past build results through the console, CW, or the API. The results include outcome (success or failure), build duration, output artifact location, and log location
  • Automate release process by using CodePipeline to test your code and run your builds with CodeBuild

4. Caching

  • S3
  • Local: store cache locally on a build host. Cache modes:
    • Source
    • Docker layer
    • Custom

5. Monitoring & Security

  • Specify key stored in KMS to encrypt artifacts
  • Security and separation at the infrastructure and execution levels
  • Use CW to watch the builds. Monitor builds at 2 levels:
    • Project level: These metrics are for all builds in the specified project only
    • AWS account level: These metrics are for all builds in one account

CodeDeploy

Deployment service. Automates software deployments to a variety of compute services such as EC2, Lambda, and on-premises servers

CodeDeploy deploys the app on the compute platform (EC2, ECS, Lambda, on-premise servers)


Appspec.yml (Application specification): Manage each deployment as a series of lifecycle event hooks


Deployment group contains individually tagged instances, EC2 instances in AS groups, or both

  • Lambda (appspec.yml): defines a set of CodeDeploy configurations for future deployments of an AWS Lambda function

  • ECS: Deployment group specifies ECS service, LB, test listener (optional), and two target groups. It also specifies when to reroute traffic to the replacement task set, and when to terminate the original task set and ECS application after a successful deployment


Two Types of deployment

  • In-place (Only EC2 / On-premise)
    • Instances in the deployment group are updated with the latest application revision
    • Current application is stopped, new version is installed & started
  • Blue / Green (EC2, ECS, Lambda)
    • Traffic is rerouted from one set of instances to another by deregistering the original instances from a LB and registering a replacement set of instances that typically has the latest application revision already installed
    • CodeDeploy shift traffic from blue (old) to green (new) according to specifications

Blue / Green for ECS, EC2, Lambda

  • ECS: Traffic shifts from your ECS service’s original task set to a replacement task set all at once
  • EC2
    • CodeDeploy agent must be installed on each instance
    • More than one EC2 instances, with EC2 tags / EC2 AS group, and correct IAM instance profile
    • During replacement:
      • Use EC2 AS group you specify as a template for the replacement environment
      • Specify the replacement instance using EC2 instance tags, EC2 AS group names, or both

  • Lambda (3种) - Control how traffic is shifted

    • Canary: Shift traffic by percentage
    • Linear: Shift traffic equally
    • All at once: All traffic is shifted from the original Lambda function to the updated Lambda version

  • Pros

    • Apps can be installed and tested in the new replacement environment, and deployed to production simply by re-routing traffic
    • EC2 / On-premise, rollback to previous version is fast & reliable (just route traffic back)


EC2 Deployment Configuration (CodeDeployDefault)

e.g. CodeDeployDefault.AllAtOnce

  • .AllAtOnce
    • In-place: downtime
    • Blue / Green: route all traffic at once
  • .HalfAtATime
    • In-place: Replace only a half
    • Blue / Green: Route half
  • .OneAtATime (default)
    • In-place: deploy 1 instance per time
    • Blue / Green: Routes traffic to 1 instance in the replacement environment per time. Succeeds if traffic is rerouted to all replacements

Lambda Deployment Configuration (CodeDeployDefault.Lambda)

e.g. CodeDeployDefault.LambdaCanary10PercentXMinutes

  • Canary10PercentXMinutes: Shift 10% traffic first, shift 90% X min later
  • Canary10PercentEveryXMinutes: Shift 10% traffic every X minutes
  • AllAtOnce: Shift all at once

Deployment Revision

  • Lambda: Revision stored in S3
  • ECS: Specifies ECS task definition for deployment, container name, port mapping (route traffic), Lambda functions (optional)
  • EC2 / On-premise: Source content & application specification file, revision stored in S3

Deployment Lifecycle

  • ApplicationStop
  • DownloadBundle
  • BeforeInstall
  • Install
  • AfterInstall
  • ApplicationStart
  • ValidateService

Features

  • Prevent downtime
    • Rolling updates
    • Deployment health tracking
  • Model: file and command-based install
  • CodeDeploy integrates with AS (scale EC2 capacity based on traffic, new instance place behind ELB)
  • On-premise server: Connect to AWS public endpoints

Deployment Notes

  • Lambda & EC2: CodeDeploy implements rollbacks by redeploying
  • ECS: rollbacks by re-routing traffic from replacement task set to original task set
  • CodeDeploy monitors the instance health in the deployment group. 2 health status:
    • Revision health (application revision currently installed) - current / old / unknown
    • Instance health (if deployment is successful) - healthy / unhealthy

Monitoring

  • Need to at least monitor: Events & status of deployment and instance
  • Tools:
    • CW Alarms, Events & Logs
    • CT, SNS
    • CodeDeploy console

CodePipeline

Continuous Delivery (CD) service

  • Automate release pipelines
  • Integrate with 3rd party services (GitHub, Jenkins)

Concepts

  • Pipeline: Defines your release process workflow, and describes how a new code change progresses through your release process
  • A pipeline has a series of stages (build, test, deploy)

CodeStar

  • Quickly develop, build & test apps on AWS
  • Used with CodeCommit, CodeBuild, CodeDeploy, CodePipeline for robust CI / CD toolchain
  • Provide project template to start with
  • CodeStar Roles: Owner, Contributor, Viewer