Tag: circleci

  • Continuous Integration & Delivery (CI/CD) for Kubernetes Using CircleCI & Helm

    Introduction

    Kubernetes is getting adopted rapidly across the software industry and is becoming the most preferred option for deploying and managing containerized applications. Once we have a fully functional Kubernetes cluster we need to have an automated process to deploy our applications on it. In this blog post, we will create a fully automated “commit to deploy” pipeline for Kubernetes. We will use CircleCI & helm for it.

    What is CircleCI?

    CircleCI is a fully managed saas offering which allows us to build, test or deploy our code on every check in. For getting started with circle we need to log into their web console with our GitHub or bitbucket credentials then add a project for the repository we want to build and then add the CircleCI config file to our repository. The CircleCI config file is a yaml file which lists the steps we want to execute on every time code is pushed to that repository.

    Some salient features of CircleCI is:

    1. Little or no operational overhead as the infrastructure is managed completely by CircleCI.
    2. User authentication is done via GitHub or bitbucket so user management is quite simple.
    3. It automatically notifies the build status on the github/bitbucket email ids of the users who are following the project on CircleCI.
    4. The UI is quite simple and gives a holistic view of builds.
    5. Can be integrated with Slack, hipchat, jira, etc.

    What is Helm?

    Helm is chart manager where chart refers to package of Kubernetes resources. Helm allows us to bundle related Kubernetes objects into charts and treat them as a single unit of deployment referred to as release.  For example, you have an application app1 which you want to run on Kubernetes. For this app1 you create multiple Kubernetes resources like deployment, service, ingress, horizontal pod scaler, etc. Now while deploying the application you need to create all the Kubernetes resources separately by applying their manifest files. What helm does is it allows us to group all those files into one chart (Helm chart) and then we just need to deploy the chart. This also makes deleting and upgrading the resources quite simple.

    Some other benefits of Helm is:

    1. It makes the deployment highly configurable. Thus just by changing the parameters, we can use the same chart for deploying on multiple environments like stag/prod or multiple cloud providers.
    2. We can rollback to a previous release with a single helm command.
    3. It makes managing and sharing Kubernetes specific application much simpler.

    Note: Helm is composed of two components one is helm client and the other one is tiller server. Tiller is the component which runs inside the cluster as deployment and serves the requests made by helm client. Tiller has potential security vulnerabilities thus we will use tillerless helm in our pipeline which runs tiller only when we need it.

    Building the Pipeline

    Overview:

    We will create the pipeline for a Golang application. The pipeline will first build the binary, create a docker image from it, push the image to ECR, then deploy it on the Kubernetes cluster using its helm chart.

    We will use a simple app which just exposes a `hello` endpoint and returns the hello world message:

    package main
    
    import (
    	"encoding/json"
    	"net/http"
    	"log"
    	"github.com/gorilla/mux"
    )
    
    type Message struct {
    	Msg string
    }
    
    func helloWorldJSON(w http.ResponseWriter, r *http.Request) {
    	m := Message{"Hello World"}
    	response, _ := json.Marshal(m)
    	w.Header().Set("Content-Type", "application/json")
    	w.WriteHeader(http.StatusOK)
    	w.Write(response)
    }
    func main() {
    	r := mux.NewRouter()
    	r.HandleFunc("/hello", helloWorldJSON).Methods("GET")
    	if err := http.ListenAndServe(":8080", r); err != nil {
    		log.Fatal(err)
    	}
    }

    We will create a docker image for hello app using the following Dockerfile:

    FROM centos/systemd
    
    MAINTAINER "Akash Gautam" <akash.gautam@velotio.com>
    
    COPY hello-app  /
    
    ENTRYPOINT ["/hello-app"]

    Creating Helm Chart:

    Now we need to create the helm chart for hello app.

    First, we create the Kubernetes manifest files. We will create a deployment and a service file:

    apiVersion: apps/v1beta1
    kind: Deployment
    metadata:
      name: helloapp
    spec:
      replicas: 1
      strategy:
      type: RollingUpdate
      rollingUpdate:
        maxSurge: 1
        maxUnavailable: 1
      template:
        metadata:
          labels:
            app: helloapp
            env: {{ .Values.labels.env }}
            cluster: {{ .Values.labels.cluster }}
        spec:
          containers:
          - name: helloapp
            image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
            imagePullPolicy: {{ .Values.image.imagePullPolicy }}
            readinessProbe:
              httpGet:
                path: /hello
                port: 8080
                initialDelaySeconds: 5
                periodSeconds: 5
                successThreshold: 1

    apiVersion: v1
    kind: Service
    metadata:
      name: helloapp
    spec:
      type: {{ .Values.service.type }}
      ports:
      - name: helloapp
        port: {{ .Values.service.port }}
        protocol: TCP
        targetPort: {{ .Values.service.targetPort }}
      selector:
        app: helloapp

    In the above file, you must have noticed that we have used .Values object. All the values that we specify in the values.yaml file in our helm chart can be accessed using the .Values object inside the template.

    Let’s create the helm chart now:

    helm create helloapp

    Above command will create a chart helm chart folder structure for us.

    helloapp/
    |
    |- .helmignore # Contains patterns to ignore when packaging Helm charts.
    |
    |- Chart.yaml # Information about your chart
    |
    |- values.yaml # The default values for your templates
    |
    |- charts/ # Charts that this chart depends on
    |
    |- templates/ # The template files

    We can remove the charts/ folder inside our helloapp chart as our chart won’t have any sub-charts. Now we need to move our Kubernetes manifest files to the template folder and update our values.yaml and Chart.yaml

    Our values.yaml looks like:

    image:
      tag: 0.0.1
      repository: 123456789870.dkr.ecr.us-east-1.amazonaws.com/helloapp
      imagePullPolicy: Always
    
    labels:
      env: "staging"
      cluster: "eks-cluster-blog"
    
    service:
      port: 80
      targetPort: 8080
      type: LoadBalancer

    This allows us to make our deployment more configurable. For example, here we have set our service type as LoadBalancer in values.yaml but if we want to change it to nodePort we just need to set is as NodePort while installing the chart (–set service.type=NodePort). Similarly, we have set the image pull policy as Always which is fine for development/staging environment but when we deploy to production we may want to set is as ifNotPresent. In our chart, we need to identify the parameters/values which may change from one environment to another and make them configurable. This allows us to be flexible with our deployment and reuse the same chart

    Finally, we need to update Chart.yaml file. This file mostly contains metadata about the chart like the name, version, maintainer, etc, where name & version are two mandatory fields for Chart.yaml.

    version: 1.0.0
    appVersion: 0.0.1
    name: helloapp
    description: Helm chart for helloapp
    source:
      - https://github.com/akash-gautam/helloapp

    Now our Helm chart is ready we can start with the pipeline. We need to create a folder named .circleci in the root folder of our repository and create a file named config.yml in it. In our config.yml we have defined two jobs one is build&pushImage and deploy.

    Configure the pipeline:

    build&pushImage:
        working_directory: /go/src/hello-app (1)
        docker:
          - image: circleci/golang:1.10 (2)
        steps:
          - checkout (3)
          - run: (4)
              name: build the binary
              command: go build -o hello-app
          - setup_remote_docker: (5)
              docker_layer_caching: true
          - run: (6)
              name: Set the tag for the image, we will concatenate the app verson and circle build number with a `-` char in between
              command:  echo 'export TAG=$(cat VERSION)-$CIRCLE_BUILD_NUM' >> $BASH_ENV
          - run: (7)
              name: Build the docker image
              command: docker build . -t ${CIRCLE_PROJECT_REPONAME}:$TAG
          - run: (8)
              name: Install AWS cli
              command: export TZ=Europe/Minsk && sudo ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > sudo  /etc/timezone && sudo apt-get update && sudo apt-get install -y awscli
          - run: (9)
              name: Login to ECR
              command: $(aws ecr get-login --region $AWS_REGION | sed -e 's/-e none//g')
          - run: (10)
              name: Tag the image with ECR repo name 
              command: docker tag ${CIRCLE_PROJECT_REPONAME}:$TAG ${HELLOAPP_ECR_REPO}:$TAG    
          - run: (11)
              name: Push the image the ECR repo
              command: docker push ${HELLOAPP_ECR_REPO}:$TAG

    1. We set the working directory for our job, we are setting it on the gopath so that we don’t need to do anything additional.
    2. We set the docker image inside which we want the job to run, as our app is built using golang we are using the image which already has golang installed in it.
    3. This step checks out our repository in the working directory
    4. In this step, we build the binary
    5. Here we setup docker with the help of  setup_remote_docker  key provided by CircleCI.
    6. In this step we create the tag we will be using while building the image, we use the app version available in the VERSION file and append the $CIRCLE_BUILD_NUM value to it, separated by a dash (`-`).
    7. Here we build the image and tag.
    8. Installing AWS CLI to interact with the ECR later.
    9. Here we log into ECR
    10. We tag the image build in step 7 with the ECR repository name.
    11. Finally, we push the image to ECR.

    Now we will deploy our helm charts. For this, we have a separate job deploy.

    deploy:
        docker: (1)
            - image: circleci/golang:1.10
        steps: (2)
          - checkout
          - run: (3)
              name: Install AWS cli
              command: export TZ=Europe/Minsk && sudo ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > sudo  /etc/timezone && sudo apt-get update && sudo apt-get install -y awscli
          - run: (4)
              name: Set the tag for the image, we will concatenate the app verson and circle build number with a `-` char in between
              command:  echo 'export TAG=$(cat VERSION)-$CIRCLE_PREVIOUS_BUILD_NUM' >> $BASH_ENV
          - run: (5)
              name: Install and confgure kubectl
              command: sudo curl -L https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl && sudo chmod +x /usr/local/bin/kubectl  
          - run: (6)
              name: Install and confgure kubectl aws-iam-authenticator
              command: curl -o aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-07-26/bin/linux/amd64/aws-iam-authenticator && sudo chmod +x ./aws-iam-authenticator && sudo cp ./aws-iam-authenticator /bin/aws-iam-authenticator
           - run: (7)
              name: Install latest awscli version
              command: sudo apt install unzip && curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && unzip awscli-bundle.zip &&./awscli-bundle/install -b ~/bin/aws
          - run: (8)
              name: Get the kubeconfig file 
              command: export KUBECONFIG=$HOME/.kube/kubeconfig && /home/circleci/bin/aws eks --region $AWS_REGION update-kubeconfig --name $EKS_CLUSTER_NAME
          - run: (9)
              name: Install and configuire helm
              command: sudo curl -L https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-linux-amd64.tar.gz | tar xz && sudo mv linux-amd64/helm /bin/helm && sudo rm -rf linux-amd64
          - run: (10)
              name: Initialize helm
              command:  helm init --client-only --kubeconfig=$HOME/.kube/kubeconfig
          - run: (11)
              name: Install tiller plugin
              command: helm plugin install https://github.com/rimusz/helm-tiller --kubeconfig=$HOME/.kube/kubeconfig        
          - run: (12)
              name: Release helloapp using helm chart 
              command: bash scripts/release-helloapp.sh $TAG

    1. Set the docker image inside which we want to execute the job.
    2. Check out the code using `checkout` key
    3. Install AWS CLI.
    4. Setting the value of tag just like we did in case of build&pushImage job. Note that here we are using CIRCLE_PREVIOUS_BUILD_NUM variable which gives us the build number of build&pushImage job and ensures that the tag values are the same.
    5. Download kubectl and making it executable.
    6. Installing aws-iam-authenticator this is required because my k8s cluster is on EKS.
    7. Here we install the latest version of AWS CLI, EKS is a relatively newer service from AWS and older versions of AWS CLI doesn’t have it.
    8. Here we fetch the kubeconfig file. This step will vary depending upon where the k8s cluster has been set up. As my cluster is on EKS am getting the kubeconfig file via. AWS CLI similarly if your cluster in on GKE then you need to configure gcloud and use the command  `gcloud container clusters get-credentials <cluster-name> –zone=<zone-name>`. We can also have the kubeconfig file on some other secure storage system and fetch it from there.</zone-name></cluster-name>
    9. Download Helm and make it executable
    10. Initializing helm, note that we are initializing helm in client only mode so that it doesn’t start the tiller server.
    11. Download the tillerless helm plugin
    12. Execute the release-helloapp.sh shell script and pass it TAG value from step 4.

    In the release-helloapp.sh script we first start tiller, after this, we check if the release is already present or not if it is present then we upgrade otherwise we make a new release. Here we override the value of tag for the image present in the chart by setting it to the tag of the newly built image, finally, we stop the tiller server.

    #!/bin/bash
    TAG=$1
    echo "start tiller"
    export KUBECONFIG=$HOME/.kube/kubeconfig
    helm tiller start-ci
    export HELM_HOST=127.0.0.1:44134
    result=$(eval helm ls | grep helloapp) 
    if [ $? -ne "0" ]; then 
       helm install --timeout 180 --name helloapp --set image.tag=$TAG charts/helloapp
    else 
       helm upgrade --timeout 180 helloapp --set image.tag=$TAG charts/helloapp
    fi
    echo "stop tiller"
    helm tiller stop 

    The complete CircleCI config.yml file looks like:

    version: 2
    
    jobs:
      build&pushImage:
        working_directory: /go/src/hello-app
        docker:
          - image: circleci/golang:1.10
        steps:
          - checkout
          - run:
              name: build the binary
              command: go build -o hello-app
          - setup_remote_docker:
              docker_layer_caching: true
          - run:
              name: Set the tag for the image, we will concatenate the app verson and circle build number with a `-` char in between
              command:  echo 'export TAG=$(cat VERSION)-$CIRCLE_BUILD_NUM' >> $BASH_ENV
          - run:
              name: Build the docker image
              command: docker build . -t ${CIRCLE_PROJECT_REPONAME}:$TAG
          - run:
              name: Install AWS cli
              command: export TZ=Europe/Minsk && sudo ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > sudo  /etc/timezone && sudo apt-get update && sudo apt-get install -y awscli
          - run:
              name: Login to ECR
              command: $(aws ecr get-login --region $AWS_REGION | sed -e 's/-e none//g')
          - run: 
              name: Tag the image with ECR repo name 
              command: docker tag ${CIRCLE_PROJECT_REPONAME}:$TAG ${HELLOAPP_ECR_REPO}:$TAG    
          - run: 
              name: Push the image the ECR repo
              command: docker push ${HELLOAPP_ECR_REPO}:$TAG
      deploy:
        docker:
            - image: circleci/golang:1.10
        steps:
          - attach_workspace:
              at: /tmp/workspace
          - checkout
          - run:
              name: Install AWS cli
              command: export TZ=Europe/Minsk && sudo ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > sudo  /etc/timezone && sudo apt-get update && sudo apt-get install -y awscli
          - run:
              name: Set the tag for the image, we will concatenate the app verson and circle build number with a `-` char in between
              command:  echo 'export TAG=$(cat VERSION)-$CIRCLE_PREVIOUS_BUILD_NUM' >> $BASH_ENV
          - run:
              name: Install and confgure kubectl
              command: sudo curl -L https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl && sudo chmod +x /usr/local/bin/kubectl  
          - run:
              name: Install and confgure kubectl aws-iam-authenticator
              command: curl -o aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-07-26/bin/linux/amd64/aws-iam-authenticator && sudo chmod +x ./aws-iam-authenticator && sudo cp ./aws-iam-authenticator /bin/aws-iam-authenticator
           - run:
              name: Install latest awscli version
              command: sudo apt install unzip && curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && unzip awscli-bundle.zip &&./awscli-bundle/install -b ~/bin/aws
          - run:
              name: Get the kubeconfig file 
              command: export KUBECONFIG=$HOME/.kube/kubeconfig && /home/circleci/bin/aws eks --region $AWS_REGION update-kubeconfig --name $EKS_CLUSTER_NAME
          - run:
              name: Install and configuire helm
              command: sudo curl -L https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-linux-amd64.tar.gz | tar xz && sudo mv linux-amd64/helm /bin/helm && sudo rm -rf linux-amd64
          - run:
              name: Initialize helm
              command:  helm init --client-only --kubeconfig=$HOME/.kube/kubeconfig
          - run:
              name: Install tiller plugin
              command: helm plugin install https://github.com/rimusz/helm-tiller --kubeconfig=$HOME/.kube/kubeconfig        
          - run:
              name: Release helloapp using helm chart 
              command: bash scripts/release-helloapp.sh $TAG
    workflows:
      version: 2
      primary:
        jobs:
          - build&pushImage
          - deploy:
              requires:
                - build&pushImage

    At the end of the file, we see the workflows, workflows control the order in which the jobs specified in the file are executed and establishes dependencies and conditions for the job. For example, we may want our deploy job trigger only after my build job is complete so we added a dependency between them. Similarly, we may want to exclude the jobs from running on some particular branch then we can specify those type of conditions as well.

    We have used a few environment variables in our pipeline configuration some of them were created by us and some were made available by CircleCI. We created AWS_REGION, HELLOAPP_ECR_REPO, EKS_CLUSTER_NAME, AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY variables. These variables are set via. CircleCI web console by going to the projects settings. Other variables that we have used are made available by CircleCI as a part of its environment setup process. Complete list of environment variables set by CircleCI can be found here.

    Verify the working of the pipeline:

    Once everything is set up properly then our application will get deployed on the k8s cluster and should be available for access. Get the external IP of the helloapp service and make a curl request to the hello endpoint

    $ curl http://a31e25e7553af11e994620aebe144c51-242977608.us-west-2.elb.amazonaws.com/hello && printf "n"
    
    {"Msg":"Hello World"}

    Now update the code and change the message “Hello World” to “Hello World Returns” and push your code. It will take a few minutes for the pipeline to complete execution and once it is complete make the curl request again to see the changes getting reflected.

    $ curl http://a31e25e7553af11e994620aebe144c51-242977608.us-west-2.elb.amazonaws.com/hello && printf "n"
    
    {"Msg":"Hello World Returns"}

    Also, verify that a new tag is also created for the helloapp docker image on ECR.

    Conclusion

    In this blog post, we explored how we can set up a CI/CD pipeline for kubernetes and got basic exposure to CircleCI and Helm. Although helm is not absolutely necessary for building a pipeline, it has lots of benefits and is widely used across the industry. We can extend the pipeline to consider the cases where we have multiple environments like dev, staging & production and make the pipeline deploy the application to any of them depending upon some conditions. We can also add more jobs like integration tests. All the codes used in the blog post are available here.

    Related Reads:

    1. Continuous Deployment with Azure Kubernetes Service, Azure Container Registry & Jenkins
    2. Know Everything About Spinnaker & How to Deploy Using Kubernetes Engine
  • Set Up Simple S3 Deployment Workflow with Github Actions and CircleCI

    In this article, we’ll implement a continuous delivery (referred to as CD going forward) workflow using the Serverless framework for our demo React SPA application using Serverless Finch.

    Deploying single-page applications to AWS S3 is a common use case. Manual deployment and bucket configuration can be tedious and unreliable. By using Serverless and CD platforms, we can simplify this commonly faced CD challenge.

    In almost every project we have worked on, we have built a general-purpose continuous integration (referred to as CI through the rest of this article) setup as part of our basic setups. The CI requirements might range from simple test workflows to cluster deployments.

    In this article, we’ll be focusing on a simple deployment workflow using Github Actions and CircleCI. Github Actions brought CI/CD to a wider community by simplifying the setup for CI pipelines. 

    Prerequisites

    This article assumes you have a basic understanding of CICD and AWS services such as IAM and S3. The sample application uses a basic Create React Application for the deployment demo. But knowing React.js is not required. You can implement the same flow for any other SPA or bare-bones application.

    Why Github Actions?

    There have always been great tools and CI platforms, such as AWS CodePipeline, Jenkins, Travis CI, CircleCI, etc. What makes Github Actions so compelling is that it’s built inside Github. Many organizations use Github for source control, and they often have to spend time configuring repositories with CI tools. On top of that, starting with Github Actions is free.

    As Github Actions is built inside the Github ecosystem, it’s a piece of cake to get CI pipelines up and running. Github Actions also allow you to build your own actions. However, there are some limitations because the CI platform is quite new compared to others.

    Why CircleCI?

    CircleCI has been in the market for almost a decade providing CICD solutions. One of many reasons to choose CircleCI is its pricing. CircleCI offers free credits each month without any upfront payments or payment details. It also offers a wide-ranging repository of plugins called Orbs. You can even build your own orbs, which are easy. It also offers simple and reliable workflow building tools. You can check other features as well.

    Let’s Get Started

    To introduce the application, we’ll create a simple React application with master-detail flow added to it. We’ll be using React’s official CRA tool to create our project, which creates the boilerplate for us.

    Installing Dependencies

    Let’s install the create-react-app as a global package. We’ll be calling our demo project “Serverless S3”. Now, we will create our react app with the following:

    yarn global add create-react-app
    create-react-app serverless-s3

    Now that we’ve created the frontend application, we can start building something cool with it. If we run the application with yarn start, we should be able to see the default CRA welcome page:

    Source: React

    To implement our master-detail flow of Github repositories, we’ll need to add some navigation to our app. Also, to keep it short, we’ll be using Github’s official SDK package. So, let’s use the react-router for the same.

    yarn add react-router-dom @octakit/core

    Our demo application will consist of two routes: 

    1. A list of all public repos of an organization
    2. The details of the repository after clicking a repo item from the list 

    We’ll be using the Octokit client to fetch the data from Github’s open endpoints. This won’t need any authentication with Github.

    Adding Application Components

    Alright, now that we have our dependencies installed, we can add the routes to our App.js, which is the entry point for our React app.

    import { BrowserRouter as Router, Switch, Route } from 'react-router-dom';
     
    import RepoList from './RepoList';
    import RepoDetails from './RepoDetails';
     
    import './App.css';
     
    function App() {
      return (
       <Router>
         <div className="App">
           <Switch>
             <Route path="/repo/:owner/:repo" component={RepoDetails} />
             <Route path="/" component={RepoList} />
           </Switch>
         </div>
       </Router>
     );
    }
     
    export default App;

    Let’s initialize our Octokit client, which will help us make calls to Github’s open endpoints to get data.

    import { Octokit } from '@octokit/core';
     
    export const octokit = new Octokit({});

    You can even make calls to authorized resources with the Octokit client. Octokit client supports both GraphQL and REST API. You can learn more about the client through the official documentation.

    Let’s add the RepoList.js component to the application, which will fetch the list of repositories of a given organization and display hyperlinks to the details page.

    import React, { useEffect, useState } from 'react';
    import { Link } from 'react-router-dom';
    import { octokit } from './client';
     
    function RepoList() {
     const [repos, setRepos] = useState([]);
     useEffect(() => {
       octokit
         .request('GET /orgs/:org/repos', {
           org: 'octokit',
         })
         .then((data) => setRepos(data.data));
     }, []);
     
     return (
       <div className="repo-list-container">
         <h1>Repositories</h1>
         <ul>
           {repos.map((repo) => (
             <li key={repo.id} className="repo-list-item">
               <Link to={`/repo/${repo.owner.login}/${repo.name}`}>{repo.full_name}</Link>
             </li>
           ))}
         </ul>
       </div>
     );
    }
     
    export default RepoList;

    Now that we have our list of repositories ready, we can now allow users to see some of their general details. Let’s create our details component called RepoDetails:

    import { useEffect, useState } from 'react';
    import { useParams } from 'react-router-dom';
    import { octokit } from './client';
    function RepoDetails() {
      const [repo, setRepo] = useState();
      const { repo: repoName, owner } = useParams();
      useEffect(() => {
        octokit
          .request('GET /repos/{owner}/{repo}', {
            owner,
            repo: repoName,
          })
          .then((data) => setRepo(data.data));
      }, [repoName, owner]);
      if (!repo) {
        return <b>loading...</b>;
      }
      return (
        <div className="repo-container">
          <h1>{repo.full_name}</h1>
          <p>Description: {repo.description}</p>
          <ul>
            <li><b>Forks:</b> {repo.forks}</li>
            <li><b>Subscribers:</b> {repo.subscribers_count}</li>
            <li><b>Watchers:</b> {repo.watchers}</li>
            <li><b>License:</b> {repo.license.name}</li>
          </ul>
        </div>
      );
    }
    export default RepoDetails;

    Setting up Serverless

    With this done, we have our repositories master-detail flow ready. Assuming we have an AWS account setup, we can start adding the Serverless config to our project. Let’s start with the CD setup. As we said before, we’ll be using the Serverless framework to achieve our deployment workflow. Let’s add it.

    We’ll also install the Serverless plugin called serverless-finch, which allows us to configure and deploy to S3 buckets.

    yarn global add serverless
    yarn add serverless-finch --save-dev

    Now that we have our Serverless CLI installed, we init the serverless service in our project by running the following command to create a hello-world serverless service:

    serverless create -t hello-world

    This will create a configuration yaml file and a handler lambda function. We don’t need the handler, so we can delete handler.js. Our serverless.yml should look like this:

    service: serverless-s3
    frameworkVersion: '2'
     
    # The `provider` block defines where your service will be deployed
    provider:
     name: aws
     runtime: nodejs12.x
     
    functions:
     helloWorld:
       handler: handler.hello-world
         events:
         - http:
             path: helloWorld
             method: get
             cors: true

    The serverless.yml file contains configurations for a lambda function called hello-world. We can remove the functions block completely. After doing that, let’s register our Serverless Finch plugin:

    service: serverless-s3
    frameworkVersion: '2'
     
    provider:
     name: aws
     runtime: nodejs12.x
     
    plugins:
     - serverless-finch

    Alright, now that our plugin is ready to be used, we can add details about our S3 buckets so it can deploy to it. Let’s add this block, which tells Serverless to use the serverless-s3-galileo bucket to deploy our code from the build directory. Make sure you use a different bucket name, as S3 bucket names are unique globally.

    custom:
     client:
       bucketName: serverles-s3-galileo
       distributionFolder: build
       indexDocument: index.html
       errorDocument: index.html

    That is it! We’re ready to deploy our app on our bucket. Haven’t created a bucket yet? No problem—serverless-finch will automatically create it. The last thing we need to add is bucket-policy so our app can be accessed publicly. Let’s create our bucket policy.

    Note: The indexDocument is the entry point for our web application, which is index.html in this case. We also need to add the same to errorDocument so our React routing works well in S3 hosting.

    {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Principal": {
                   "AWS": "*"
               },
               "Action": "s3:GetObject",
               "Resource": "arn:aws:s3:::serverles-s3-galileo/*"
           }
       ]
    }

    As the default access to S3 assets is private, we need to set up a bucket policy for our deployment bucket. The policy gives read-only access to the public for our app so we can browse the deployed assets in the browser. You can learn more about bucket policies. Let’s update our Serverless config to use our policy. This is how our serverless.yml should look:

    service: serverless-s3
    frameworkVersion: '2'
     
    provider:
     name: aws
     runtime: nodejs12.x
     
    plugins:
     - serverless-finch
     
    custom:
     client:
       bucketName: serverles-s3-galileo
       distributionFolder: build
       indexDocument: index.html
       errorDocument: index.html
       bucketPolicyFile: config/bucket-policy.json

    Creating Github Actions Workflow

    Assuming you’ve created your repo and pushed the code to it, we can start setting up our first workflow using Github Actions. As we’re using AWS for our Serverless deployments to S3, we need to provide the details of our IAM role. The env block allows us to insert custom env variables into the CI build. In this case, we need the AWS access key and secret access key to deploy build files to the S3 bucket. 

    Github allows us to store secret values that can be used in the CI environment of Github Actions. You can easily set up these secrets for your repositories. This is how they should look when configured:

    Now, we can move ahead and add a Github Action workflow. Let’s create a workflow file at the .github/deploy.yml location and add the following to it.

    name: Serverless S3 Deploy
    on:
     push:
       branches: [ master ]
     pull_request:
       branches: [ master ]

    Alright, so the Github Actions config above tells Github to trigger this workflow whenever someone pushes to the master branch or creates a PR against it.

    As of now, our action config is incomplete and does nothing. Let’s add our first and only job to the workflow:

    name: Serverless S3
     
    on:
     push:
       branches: [ master ]
     pull_request:
       branches: [ master ]
     
    jobs:
     build:
       runs-on: ubuntu-latest
       strategy:
         matrix:
           node-version: [10.x]
       steps:
       - uses: actions/checkout@v2

    Let’s try to digest the config above:

    runs-on:  ubuntu-latest

    The runs-on statement specifies which executor will be running the job. In this case, it’s the latest release of Linux Ubuntu variant.

    Strategy: 

         Matrix:

            node-version: [10.x]

    The strategy defines the environment we want to run our job on. This is usually useful when we want to run tests on multiple machines. In our case, we don’t want that. So, we’ll be using a single node environment with version 10.x

       steps:

       – uses: actions/checkout@v2

    In the configuration’s steps block, we can define various tasks to be sequentially performed within a job. actions/checkout@v2 does the work of checking out branches for us. This step is required so we can do further work on our source code.

    This bare minimum setup is required for running a job in our Github workflows. After this, we will need to set up the environment and deploy our application. So, let’s add the rest of the steps to it.

    name: Serverless S3
     
    on:
     push:
       branches: [ master ]
     pull_request:
       branches: [ master ]
     
    jobs:
     build:
       runs-on: ubuntu-latest
       strategy:
         matrix:
           node-version: [10.x]
       steps:
       - uses: actions/checkout@v2
       - name: Use Node.js ${{ matrix.node-version }}
         uses: actions/setup-node@v1
         with:
           node-version: ${{ matrix.node-version }}
       - run: yarn install
       - run: yarn build
       - name: serverless deploy s3
         uses: serverless/github-action@master
         with:
           args: client deploy --no-confirm
         env:
           AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

    These actions need to be executed to deploy our frontend assets to our S3 buckets. As we read through the steps, we’re doing the following things in sequence:

    1. Check out the current branch code

    2. Setting up our node.js environment

    3. Installing our dependencies with yarn install

    4. Building our production build with yarn build

    5. Deploying our build to S3 with serverless deploy –no-confirm

    • The uses block defines which custom action we’re using
    • The args block allows us to pass arguments to the actions
    • The –no-confirm flag is needed so Serverless Finch does not ask us for confirmations while deploying to S3 buckets. 
    • The args allows us to tell action to run it with specific arguments
    • env allows us to pass custom environment variables to an action

    Alright, so now we have the CD workflow setup to deploy our app. We can make a commit and push to the master branch. This should trigger our workflow. You can see your workflow running in the Actions section of your repository like this:

    You can check the output of the serverless deploy step and browse the S3 website URL. It should now show our application running.

    Creating CircleCI Workflow

    To start building a repository, we need to authorize it with our Github account. You can do that by signing up for CircleCI and following the steps here.

    As we did, add the IAM role secret credentials to our actions workflow. We can set up env variables for our workflows in CircleCI. This is how they should look once configured in the project settings:

    Just like the Github Actions workflow, we can create workflows in CircleCI. CircleCI also allows us to use third-party custom plugins. We can use the available plugins called Orbs in our deployment workflows in CircleCI.

    We’ll need the official CircleCI distributions of the aws-cli, serverless-framework, and node.js orbs for our deploy workflow. Let’s create our first job for our workflow:

    version: 2.1
     
    orbs:
     aws-cli: circleci/aws-cli@1.0.0
     serverless: circleci/serverless-framework@1.0.1
     node: circleci/node@4.1.0
     
    jobs:
     deploy:
       executor: serverless/default

    The executor here is a prebuilt image, which allows us to run. 

    Just like we defined steps for our jobs in Github Actions, we can add for CircleCI. Here we’re using commands made available from the node orb to install dependencies, build projects, and set up Serverless with AWS. Just like we set up the secrets for Github Actions, we need to define our AWS credentials under the CircleCI environment variables.

    version: 2.1
     
    orbs:
     aws-cli: circleci/aws-cli@1.0.0
     serverless: circleci/serverless-framework@1.0.1
     node: circleci/node@4.1.0
     
    jobs:
     deploy:
       executor: serverless/default
       steps:
         - checkout
         - node/install-yarn
         - run:
             name: install
             command: yarn install
         - run:
             name: build
             command: yarn build
         - aws-cli/setup
         - serverless/setup:
             app-name: serverless-s3
             org-name: velotio
         - run:
             name: deploy
             command: serverless client deploy --no-confirm
    workflows:
     deploy:
       jobs:
         - deploy:
             filters:
               branches:
                 only:
                   - master

    The workflows section in the above yml file indicates that we want to trigger the deploy workflow whenever our master branch gets updated. Just like we mentioned the steps for the Github Actions deploy job, we did the same for CircleCI jobs.

    1. Check out the code
    2. Install yarn package manager with node/install-yarn 
    3. Install dependencies with yarn install
    4. Build the project with yarn build
    5. Setup AWS and Serverless CLI
    6. Deploy to s3 with serverless client deploy –no-confirm

    The workflow block in the config above tells CircleCI to run the deploy job. The filters block for the deploy job above tells us that we want to run the job only when the master branch gets updated. 

    Once we’re done with the above setup, we can make a test commit and check whether our workflow is running.

    Conclusion

    We can easily integrate build/deployment workflows with simple configurations offered through Github Actions. If we don’t primarily use GitHub as version control, we can opt for CircleCI for our workflows.

    Related Articles

    1. Automating Serverless Framework Deployment using Watchdog
    2. To Go Serverless Or Not Is The Question

    You can find the referenced code at this repo.