Đăng vào

Deploy Laravel 11 trên Kubernetes

Trong bài viết này mình sẽ giới thiệu một ví dụ triển khai ứng dụng sử dụng Laravel framework trên Kubernetes. Phần lớn nội dung mình thực hiện theo chuỗi bải viết: Laravel in Kubernetes từ chris-vermeulen.com

Giới thiệu

Laravel framework

Laravel là một framework phát triển ứng dụng web mã nguồn mở được viết bằng ngôn ngữ PHP. Được phát triển bởi Taylor Otwell và ra mắt lần đầu vào năm 2011, Laravel đã nhanh chóng trở thành một trong những lựa chọn hàng đầu cho việc xây dựng các ứng dụng web hiện đại.

Dưới đây là một số điểm quan trọng về Laravel: trong bài viết gốc Laravel in Kubernetes từ chris-vermeulen.com, tác giả dùng database MySQL, còn mình sử dụng PostgreSQL nên câu lệnh lúc khởi tại project sẽ có dạng:

# Mac OS
curl -s "https://laravel.build/laravel-in-kubernetes?with=pgsql,redis" | bash
cd laravel-in-kubernetes
./vendor/bin/sail up

# Linux
curl -s https://laravel.build/laravel-in-kubernetes?with=pgsql,redis | bash
cd laravel-in-kubernetes
./vendor/bin/sail up

Dockerize ứng dụng Laravel

Dưới đây là Dockerfile được viết ở dạng Multi Stage Builds. So với bản gốc từ tác giả thì mình có chỉnh sửa một chút như cài extensions pdo, pdo_pgsql đễ hỗ trợ cho databse PostgreSQL, hay ở stage frontend mình dùng node:18. Chi tiết có thể tham khảo dưới dây:

# Create args for PHP extensions and PECL packages we need to install.
# This makes it easier if we want to install packages,
# as we have to install them in multiple places.
# This helps keep ou Dockerfiles DRY -> https://bit.ly/dry-code
# You can see a list of required extensions for Laravel here: https://laravel.com/docs/8.x/deployment#server-requirements
ARG PHP_EXTS="pdo pdo_pgsql bcmath ctype fileinfo mbstring dom pcntl"
ARG PHP_PECL_EXTS="redis"

# We need to build the Composer base to reuse packages we've installed
FROM composer:2.2 as composer_base

# We need to declare that we want to use the args in this build step
ARG PHP_EXTS
ARG PHP_PECL_EXTS

# First, create the application directory, and some auxilary directories for scripts and such
RUN mkdir -p /opt/apps/laravel-in-kubernetes /opt/apps/laravel-in-kubernetes/bin

# Next, set our working directory
WORKDIR /opt/apps/laravel-in-kubernetes

# We need to create a composer group and user, and create a home directory for it, so we keep the rest of our image safe,
# And not accidentally run malicious scripts
RUN apk add --no-cache libpq-dev
RUN addgroup -S composer \
    && adduser -S composer -G composer \
    && chown -R composer /opt/apps/laravel-in-kubernetes \
    && apk add --virtual build-dependencies --no-cache ${PHPIZE_DEPS} openssl ca-certificates libxml2-dev oniguruma-dev \
    && docker-php-ext-install -j$(nproc) ${PHP_EXTS} \
    && pecl install ${PHP_PECL_EXTS} \
    && docker-php-ext-enable ${PHP_PECL_EXTS} \
    && apk del build-dependencies

# Next we want to switch over to the composer user before running installs.
# This is very important, so any extra scripts that composer wants to run,
# don't have access to the root filesystem.
# This especially important when installing packages from unverified sources.
USER composer

# Copy in our dependency files.
# We want to leave the rest of the code base out for now,
# so Docker can build a cache of this layer,
# and only rebuild when the dependencies of our application changes.
COPY --chown=composer composer.json composer.lock ./

# Install all the dependencies without running any installation scripts.
# We skip scripts as the code base hasn't been copied in yet and script will likely fail,
# as `php artisan` available yet.
# This also helps us to cache previous runs and layers.
# As long as comoser.json and composer.lock doesn't change the install will be cached.
RUN composer install --no-dev --no-scripts --no-autoloader --prefer-dist

# Copy in our actual source code so we can run the installation scripts we need
# At this point all the PHP packages have been installed,
# and all that is left to do, is to run any installation scripts which depends on the code base
COPY --chown=composer . .

# Now that the code base and packages are all available,
# we can run the install again, and let it run any install scripts.
RUN composer install --no-dev --prefer-dist

# For the frontend, we want to get all the Laravel files,
# and run a production compile
FROM node:18 as frontend

# We need to copy in the Laravel files to make everything is available to our frontend compilation
COPY --from=composer_base /opt/apps/laravel-in-kubernetes /opt/apps/laravel-in-kubernetes

WORKDIR /opt/apps/laravel-in-kubernetes

# We want to install all the NPM packages,
# and compile the MIX bundle for production
RUN npm install && \
    npm run build

# For running things like migrations, and queue jobs,
# we need a CLI container.
# It contains all the Composer packages,
# and just the basic CLI "stuff" in order for us to run commands,
# be that queues, migrations, tinker etc.
FROM php:8.3-alpine as cli

# We need to declare that we want to use the args in this build step
ARG PHP_EXTS
ARG PHP_PECL_EXTS

WORKDIR /opt/apps/laravel-in-kubernetes

# We need to install some requirements into our image,
# used to compile our PHP extensions, as well as install all the extensions themselves.
# You can see a list of required extensions for Laravel here: https://laravel.com/docs/8.x/deployment#server-requirements
RUN apk add --no-cache libpq-dev
RUN apk add --virtual build-dependencies --no-cache ${PHPIZE_DEPS} openssl ca-certificates libxml2-dev oniguruma-dev && \
    docker-php-ext-install -j$(nproc) ${PHP_EXTS} && \
    pecl install ${PHP_PECL_EXTS} && \
    docker-php-ext-enable ${PHP_PECL_EXTS} && \
    apk del build-dependencies

# Next we have to copy in our code base from our initial build which we installed in the previous stage
COPY --from=composer_base /opt/apps/laravel-in-kubernetes /opt/apps/laravel-in-kubernetes
COPY --from=frontend /opt/apps/laravel-in-kubernetes/public /opt/apps/laravel-in-kubernetes/public

# We need a stage which contains FPM to actually run and process requests to our PHP application.
FROM php:8.3-fpm-alpine as fpm_server

# We need to declare that we want to use the args in this build step
ARG PHP_EXTS
ARG PHP_PECL_EXTS

WORKDIR /opt/apps/laravel-in-kubernetes

RUN apk add --no-cache libpq-dev
RUN apk add --virtual build-dependencies --no-cache ${PHPIZE_DEPS} openssl ca-certificates libxml2-dev oniguruma-dev && \
    docker-php-ext-install -j$(nproc) ${PHP_EXTS} && \
    pecl install ${PHP_PECL_EXTS} && \
    docker-php-ext-enable ${PHP_PECL_EXTS} && \
    apk del build-dependencies

# We have to copy in our code base from our initial build which we installed in the previous stage
COPY --from=composer_base --chown=www-data /opt/apps/laravel-in-kubernetes /opt/apps/laravel-in-kubernetes
COPY --from=frontend --chown=www-data /opt/apps/laravel-in-kubernetes/public /opt/apps/laravel-in-kubernetes/public

# We want to cache the event, routes, and views so we don't try to write them when we are in Kubernetes.
# Docker builds should be as immutable as possible, and this removes a lot of the writing of the live application.
RUN php artisan event:cache && \
    php artisan route:cache && \
    php artisan view:cache

# As FPM uses the www-data user when running our application,
# we need to make sure that we also use that user when starting up,
# so our user "owns" the application when running
USER  www-data

# We need an nginx container which can pass requests to our FPM container,
# as well as serve any static content.
FROM nginx:1.27-alpine as web_server

WORKDIR /opt/apps/laravel-in-kubernetes

# We need to add our NGINX template to the container for startup,
# and configuration.
COPY nginx.conf.template /etc/nginx/templates/default.conf.template

# Copy in ONLY the public directory of our project.
# This is where all the static assets will live, which nginx will serve for us.
COPY --from=frontend /opt/apps/laravel-in-kubernetes/public /opt/apps/laravel-in-kubernetes/public

# We need a CRON container to the Laravel Scheduler.
# We'll start with the CLI container as our base,
# as we only need to override the CMD which the container starts with to point at cron
FROM cli as cron

WORKDIR /opt/apps/laravel-in-kubernetes

# We want to create a laravel.cron file with Laravel cron settings, which we can import into crontab,
# and run crond as the primary command in the forground
RUN touch laravel.cron && \
    echo "* * * * * cd /opt/apps/laravel-in-kubernetes && php artisan schedule:run" >> laravel.cron && \
    crontab laravel.cron

CMD ["crond", "-l", "2", "-f"]

FROM cli

Để build các image thì có thể lần lượt chạy theo các command dưới đây:

# Composer Stage
docker build . --target composer_base -t my-registry/composer-base
# Frontend Stage
docker build . --target frontend -t my-registry/frontend
# CLI Container
docker build . --target cli -t my-registry/cli
# FPM Container
docker build . --target fpm_server -t my-registry/fpm-server
# Web Server
docker build . --target web_server -t my-registry/web-server
# Cron build
docker build . --target cron -t my-registry/cron

Deploy lên Kubernetes

Chuyển file .env sang ConfigMap và Secret

Sau khi chạy câu lệnh khởi tạo Laravel, bạn sẽ có một project mẫu với đầy đủ các cấu hình cần thiết được lưu ở trong file .env. Chúng ta sẽ chuyển file này vào ConfigMap những thông tin cấu hình cơ bản, đổi với những thông tin cần phải bảo giấu kín như mật khẩu chúng ta sẽ bỏ vào Secret.

Chẳng hạn như:

app-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: laravel
  namespace: laravel-app
data:
  APP_NAME: "Laravel"
  APP_ENV: "production"
  APP_DEBUG: "true"
  # Once you have an external URL for your application, you can add it here. 
  APP_URL: "laravel.mydomain.com"
  
  # Update the LOG_CHANNEL to stdout for Kubernetes
  LOG_CHANNEL: "stdout"
  LOG_LEVEL: "debug"
  DB_CONNECTION: "pgsql"
  DB_HOST: "<database-host>"
  DB_PORT: "5432"
  DB_DATABASE: "app"
  BROADCAST_DRIVER: "log"
  CACHE_DRIVER: "file"
  FILESYSTEM_DRIVER: "local"
  
  # Update the Session driver to Redis, based off part-2 of series
  SESSION_DRIVER: "redis"
  SESSION_LIFETIME: "120"
  MEMCACHED_HOST: "memcached"
  REDIS_HOST: "redis"
  REDIS_PORT: "6379"
  VITE_APP_NAME: "${APP_NAME}"

  QUEUE_CONNECTION: "redis"

Những thông tin như mật khẩu, token, chúng ta bỏ vào file

app-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: laravel
  namespace: laravel-app
type: Opaque
stringData:
  APP_KEY: "base64:eQrCXchv9wpGiOqRFaeIGPnqklzvU+A6CZYSMosh1to="
  DB_USERNAME: "db-user"
  DB_PASSWORD: "db-pass"
  REDIS_PASSWORD: ""

Laravel FPM

Deployment

fpm-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-fpm
  namespace: laravel-app
  labels:
    tier: backend
    layer: fpm
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: backend
      layer: fpm
  template:
    metadata:
      labels:
        tier: backend
        layer: fpm
    spec:
      initContainers:
        - name: migrations
          image: my-registry/cli
          command:
            - php
          args:
            - artisan
            - migrate
            - --force
          envFrom:
            - configMapRef:
                name: laravel
            - secretRef:
                name: laravel
      containers:
        - name: fpm
          image: my-registry/fpm-server
          ports:
            - containerPort: 9000
          envFrom:
            - configMapRef:
                name: laravel
            - secretRef:
                name: laravel

Service

fpm-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: laravel-fpm
  namespace: laravel-app
spec:
  selector:
    tier: backend
    layer: fpm
  ports:
    - protocol: TCP
      port: 9000
      targetPort: 9000

Queue worker

queue-worker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-queue-worker-default
  namespace: laravel-app
  labels:
    tier: backend
    layer: queue-worker
    queue: default
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: backend
      layer: queue-worker
      queue: default
  template:
    metadata:
      labels:
        tier: backend
        layer: queue-worker
        queue: default
    spec:
      containers:
        - name: queue-worker
          image: my-registry/cli
          command:
            - php
          args:
            - artisan
            - queue:work
            - --queue=default
            - --max-jobs=200
          ports:
            - containerPort: 9000
          envFrom:
            - configMapRef:
                name: laravel
            - secretRef:
                name: laravel

Redis cache

redis.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc
  namespace: laravel-app
spec:
  storageClassName: <your-storage-class>
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      # We are starting with 1GB. We can always increase it later.
      storage: 1Gi
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
  namespace: laravel-app
  labels:
    tier: backend
    layer: redis
spec:
  serviceName: redis
  selector:
    matchLabels:
      tier: backend
      layer: redis
  replicas: 1
  template:
    metadata:
      labels:
        tier: backend
        layer: redis
    spec:
      containers:
      - name: redis
        image: redis:7.4-alpine
        command: ["redis-server", "--appendonly", "yes"]
        ports:
        - containerPort: 6379
          name: web
        volumeMounts:
        - name: redis-aof
          mountPath: /data
      volumes:
        - name: redis-aof
          persistentVolumeClaim:
            claimName: redis-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: laravel-app
  labels:
    tier: backend
    layer: redis
spec:
  ports:
  - port: 6379
    protocol: TCP
  selector:
    tier: backend
    layer: redis
  type: ClusterIP

Webserver

Deployment

webserver-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-webserver
  namespace: laravel-app
  labels:
    tier: backend
    layer: webserver
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: backend
      layer: webserver
  template:
    metadata:
      labels:
        tier: backend
        layer: webserver
    spec:
      containers:
        - name: webserver
          image: my-registry/web-server
          ports:
            - containerPort: 80
          env:
            - name: FPM_HOST
              value: laravel-fpm:9000

Webserver Service

webserver-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: laravel-webserver
  namespace: laravel-app
spec:
  selector:
    tier: backend
    layer: webserver
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Gateway - HTTPRoute

Thông thường người ta sẽ sử dụng Ingress để cho phép người dùng có thể truy cập ứng dụng thông qua một domain. Người đọc có thể tham khảo trên bài viết gốc từ tác giả, hoặc từ các nguồn khác trên Internet.

Đối với mình, hiện tại đang có 1 Homelab cluster sử dụng Istio Gateway API, nên dưới đây mình sẽ giới thiệu cách cấu hình HTTPRoute để làm việc tương tự như Ingress.

Istio Gateway

Chúng ta sẽ thêm phân cấu hình cho tên miền của ứng dụng Laravel như phần tô sáng.

istio-gateway.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: istio-gateway
  namespace: istio-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  gatewayClassName: istio
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: "*.mydomain.com"
      allowedRoutes:
        namespaces:
          from: All
    - name: app-abc
      hostname: app-ABC.mydomain.com
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - name: ssl-app-ABC
      allowedRoutes:
        namespaces:
          from: All
    ...
    - name: app-laravel
      hostname: laravel.mydomain.com
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - name: ssl-laravel
      allowedRoutes:
        namespaces:
          from: All

Sau đó kubectl -n istio-ingress apply -f istio-gateway.yaml

HTTPRoute

http_route.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httproute
  namespace: laravel-app
spec:
  parentRefs:
  - group: gateway.networking.k8s.io	
    kind: Gateway	
    name: istio-gateway
    namespace: istio-ingress
    sectionName: app-laravel
  hostnames:
  - laravel.mydomain.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: laravel-webserver
      group: ''	
      kind: Service
      port: 80
      weight: 1
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: tls-redirect
  namespace: laravel-app
spec:
  parentRefs:
  - group: gateway.networking.k8s.io	
    kind: Gateway	
    name: istio-gateway
    namespace: istio-ingress
    sectionName: http
  hostnames:
  - laravel.mydomain.com
  rules:
  - filters:
    - type: RequestRedirect
      requestRedirect:
        scheme: https
        port: 443
        statusCode: 302
    matches:	
        - path:	
            type: PathPrefix	
            value: /

Database

Về database cho ứng dụng Laravel, thông thường chúng ta sẽ sử dụng một PostgreSQL server riêng tách biệt với Kubernetes cluster, và bạn chỉ cần điền các thông tin kết nối đến Database trong phần Configmap và Secrets đã trình bày ở phần đầu của bài viết.

Riêng mình, do hạn chế về thiết bị nên mình sử dụng một giải pháp cho phép thiết lập một database trên chính cụm Kubernetes đó là CloudNativePG. Sẽ có nhiều luồng ý kiến ủng hộ hoặc phản bác việc triển khai Database trên Kubernetes, tuy nhiên theo mình, không có đúng sai ở đây, chỉ có phương án nào phù hợp yêu cầu đặt ra mà thôi.

Toàn bộ source code có thể xem thêm ở đây

Phần deploy database sẽ vượt qua ngoài khuôn khổ của bài viết này, nếu có thời gian mình sẽ giới thiệu ở chủ đề khác.

Chúc thành công, Anh Nguyễn