Jenkins Pipeline + Nginx + Spring Boot(Maven) 활용한 무중단 배포

Jenkins Pipeline + Nginx + Spring Boot(Maven) 무중단 배포

 

요새는 CI/CD 가 잘되어 있어서 jetbrains, aws, atlassian, jenkins 등.. 선택지가 많으나 비용이 적게드는 최소한의 방법으로 구성을 했던 내용을 복습하기 위해 해당 포스팅을 작성하게 된다.

 

bitbucket webhook 을 통하여 소스 pr 후 merge 시에 trigger 가 발생하게 되는 구성으로 되어있으며 (해당 과정은 생략) 이후 jenkins pipeline 을 통하여 빌드하고 각 배포할 서버에 전송 후 구동 및 스위칭 스크립트를 작성하는 프로세스로 진행할 것이다.

 

배포할 서버에 빌드파일을 전송할 JenkinsFile 작성 (Maven)

[Jenkinsfile]

pipeline {
    agent any

    tools {
        maven "maven-3.8.1"
    }

    environment {
        MAVEN_HOME = "tool maven-3.8.1"
    }

    stages {
        stage('build') {
            steps {
                script {
                    sh "mvn -version"
                    sh "mvn clean package -DskipTests"
                }
            }
        }

        stage('deploy1') {
            steps {
                script {
                    if (env.BRANCH_NAME == 'develop') {
                        echo ">>>>>>>>>>>>>>> Deploy DevelopWeb1 Server"
                        sh "scp -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ./target/bkjeon-service-*.jar ec2-user@{target ip}:/home/ec2-user/app/"

                        echo ">>>>>>>>>>>>>>> Restart DevelopWeb1 Server"
                        sh 'ssh -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ec2-user@{target ip} "nohup sh /home/ec2-user/app_deploy.sh > /dev/null 2>&1 &"'
                    } else if (env.BRANCH_NAME == 'master') {
                        echo ">>>>>>>>>>>>>>> Deploy LiveWeb1 Server"
                        sh "scp -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ./target/bkjeon-service-*.jar ec2-user@{target ip}:/home/ec2-user/app/"

                        echo ">>>>>>>>>>>>>>> Restart LiveWeb1 Server"
                        sh 'ssh -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ec2-user@{target ip} "nohup sh /home/ec2-user/app_deploy.sh > /dev/null 2>&1 &"'
                    }
                }
            }
        }

        stage('deploy2') {
            steps {
                script {
                    if (env.BRANCH_NAME == 'develop') {
                        echo ">>>>>>>>>>>>>>> Deploy DevelopWeb2 Server"
                        sh "scp -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ./target/bkjeon-service-*.jar ec2-user@{target ip}:/home/ec2-user/app/"

                        echo ">>>>>>>>>>>>>>> Restart DevelopWeb2 Server"
                        sh 'ssh -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ec2-user@{target ip} "nohup sh /home/ec2-user/app_deploy.sh > /dev/null 2>&1 &"'
                    } else if (env.BRANCH_NAME == 'master') {
                        echo ">>>>>>>>>>>>>>> Deploy LiveWeb2 Server"
                        sh "scp -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ./target/bkjeon-service-*.jar ec2-user@{target ip}:/home/ec2-user/app/"

                        echo ">>>>>>>>>>>>>>> Restart LiveWeb2 Server"
                        sh 'ssh -i /var/lib/jenkins/bkjeon.pem -o StrictHostKeyChecking=no ec2-user@{target ip} "nohup sh /home/ec2-user/app_deploy.sh > /dev/null 2>&1 &"'
                    }
                }
            }
        }
    }
}

 

nginx 리버스 프록시 변경을 위한 profile 을 확인하는 API 및 yml 추가

[DeployController.java]

...

import lombok.RequiredArgsConstructor;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.Arrays;

@RestController
@RequiredArgsConstructor
@RequestMapping("/api")
public class DeployController {

    private final Environment env;

    @GetMapping("/profile")
    public String getProfile () {
        return Arrays.stream(env.getActiveProfiles())
            .findFirst()
            .orElse("");
    }

}

[application-develop1.yml]

server:
  port: 8081
spring:
  config:
    activate:
      on-profile: develop1

  ...

[application-develop2.yml]

server:
  port: 8082
spring:
  config:
    activate:
      on-profile: develop2

  ...

 

Nginx 설정 

nginx.conf 에 service-url.inc 파일을 연동시키고 해당 service-url.inc 파일 내용을 배포 스크립트 내부에서 nginx 리버스 프록시 정보를 변경하고 reload 를 하기위해 필요한 내용들을 생성 및 수정 한다. (우리가 운영중인 웹 서버의 포트는 9091 로가정)

[service-url.inc]

set $service_url http://127.0.0.1:8081;

[/etc/nginx/nginx.conf]

...

server {
    ...

    # 주석처리
    # include /etc/nginx/default.d/*.conf;

    include /etc/nginx/service-url.inc;

    location / {
            proxy_pass $service_url;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
    }

}

 

배포 및 리버스프록시 정보 스위치 스크립트 작성

배포할 jar 파일명은 Ex) bkjeon-service-202410111244.jar

[app_deploy.sh]

#!/bin/bash
SCRIPT_PATH=/home/ec2-user
BASE_PATH=/home/ec2-user/app
DEPLOY_PATH=$BASE_PATH/

echo "> >>>>>> Current Application Started Profile Check"
CURRENT_PROFILE=$(curl -s http://localhost:9091/api/profile)
echo "> $CURRENT_PROFILE"

echo ">>>>>> Application Switching.."
if [ $CURRENT_PROFILE == develop1 ]
then
  IDLE_PROFILE=develop2
  IDLE_PORT=8082
elif [ $CURRENT_PROFILE == develop2 ]
then
  IDLE_PROFILE=develop1
  IDLE_PORT=8081
else
  echo "> Not Match Profile. Profile: $CURRENT_PROFILE"
  echo "> develop1 set IDLE_PROFILE: develop1"
  IDLE_PROFILE=develop1
  IDLE_PORT=8081
fi

JAR_NAME=$(ls -tr $BASE_PATH/ | grep jar | tail -n 1)
echo "> JAR Name: $JAR_NAME, PROFILE: $IDLE_PROFILE"

IDLE_APPLICATION=$IDLE_PROFILE-bkjeon-service.jar
IDLE_APPLICATION_PATH=$DEPLOY_PATH$IDLE_APPLICATION
ln -Tfs $DEPLOY_PATH$JAR_NAME $IDLE_APPLICATION_PATH

echo "> Application PID Check"
IDLE_PID=$(pgrep -f $IDLE_APPLICATION)

if [ -z $IDLE_PID ]
then
  echo "> Not Idle PID."
else
  echo "> kill -15 $IDLE_PID"
  kill -15 $IDLE_PID
  sleep 5
fi

echo ">>>>>> New Application Deploy"
echo $IDLE_PROFILE $IDLE_APPLICATION_PATH
nohup java -jar -server -Xms512m -Xmx512m -Dspring.profiles.active=$IDLE_PROFILE $IDLE_APPLICATION_PATH 2>&1 &

echo ">>>>>> health check start"
echo "PORT: $IDLE_PORT"
echo "curl -s http://localhost:$IDLE_PORT/api/health"
sleep 20

for retry_count in {1..10}
do
  response=$(curl -s http://localhost:$IDLE_PORT/api/health)
  up_count=$(echo $response | grep 'UP' | wc -l)

  if [ $up_count -ge 1 ]
  then
      echo "> Health check Success"
      break
  else
      echo "> Health check Error or status is not UP"
      echo "> Health check: ${response}"
  fi

  if [ $retry_count -eq 10 ]
  then
    echo "> Health check Fail. "
    echo "> Exit deployment without connectiong to nginx"
    exit 1
  fi

  echo "> Health check connection fail. retry..."
  sleep 30
done

#chmod +x /home/ec2-user/app_switch.sh
echo "> Switching .."
sh $SCRIPT_PATH/app_switch.sh

echo ">>>>>> before files delete"
cd $BASE_PATH
ls -t | grep bkjeon-service- | tail -n +4 | xargs rm -f

[app_switch.sh]

#!/bin/bash
# chmod +x ~/app/app_switch.sh
echo "> Check the currently running port"
CURRENT_PROFILE=$(curl -s http://localhost:9091/api/profile)

if [ $CURRENT_PROFILE == develop1 ]
then
  IDLE_PORT=8082
elif [ $CURRENT_PROFILE == develop2 ]
then
  IDLE_PORT=8081
else
  echo "> There are no matching profiles. Profile: $CURRENT_PROFILE"
  echo "> 8081 Setting"
  IDLE_PORT=8081
fi

echo "> Switch Target Port: $IDLE_PORT"
echo "> Port Switch"
echo "set \$service_url http://127.0.0.1:${IDLE_PORT};" |sudo tee /etc/nginx/service-url.inc

PROXY_PORT=$(curl -s http://localhost:9091/api/profile)
echo "> Nginx Current Proxy Port: $PROXY_PORT"

echo "> Nginx Reload"
sudo service nginx reload

 

 

참고

https://jojoldu.tistory.com/