手把手教你写一个通用的helmchart

[TOC]

成都创新互联的客户来自各行各业，为了共同目标，我们在工作上密切配合，从创业型小企业到企事业单位，感谢他们对我们的要求，感谢他们从不同领域给我们带来的挑战，让我们激情的团队有机会用头脑与智慧不断的给客户带来惊喜。专业领域包括成都网站设计、成都网站制作、电商网站开发、微信营销、系统平台开发。

1. 模板介绍

首先，放上此模板链接：

https://github.com/ygqygq2/charts/tree/master/ygqygq2/mod-chart

此chart可当作POD单image的通用模板，只需要使用sed替换下chart名，并修改下README.md和NOTES.txt就可以了。下文，我通过复制此chart成example-chart来作示范说明。

[root@master1 mod-chart]# tree
.
├── Chart.yaml  # chart版本信息文件
├── README.md  # chart说明文件
├── templates  # kubernetes资源yaml模板
│   ├── configmap.yaml  # configmap模板
│   ├── deployment-statefulset.yaml  # deployment或statefulset模板
│   ├── _helpers.tpl  # 辅助模板和 partials
│   ├── ingress.yaml  # ingress模板
│   ├── NOTES.txt  # 部署chart后输出的帮助文档
│   ├── pvc.yaml  # pvc模板
│   ├── secret.yaml  # secret模板
│   ├── service-headless.yaml  # service headless模板
│   └── service.yaml  # service模板
└── values.yaml  # 默认设置

1 directory, 12 files
[root@master1 mod-chart]# helm3 lint --strict .
1 chart(s) linted, 0 chart(s) failed

2. 新chart制作

注：
下文中文件内容我保留，只加注释。
注释中需要修改的地方 [*] 标记为必选，[-] 标识为可选。

2.1 目录准备

将模板mod-chart复制成example-chart，并作内容替换。

rsync -avz mod-chart/ example-chart/
cd example-chart/
sed -i 's@mod-chart@example-chart@g' *.*
sed -i 's@mod-chart@example-chart@g' templates/*.*

2.2 修改Chart.yaml

vim Chart.yaml

apiVersion: v1  # 当前helm api版本，不需要修改
appVersion: 1.14.2  # 此处为你应用程序的版本号 [*]
description: Chart for the nginx server  # 介绍此chart是干嘛的，按需求修改
engine: gotpl  # go模板引擎，不需要修改 [-]
name: example-chart  # 模板名，对应目录名 [*]
version: 1.0.0  # 此chart版本号 [*]
home: http://www.nginx.org  # 应用程序官网 [*]
icon: /upload/otherpic48/60313.jpg  # 应用程序logo地址 [*]
keywords:  # 关键字列表 [*]
- nginx
- http
- web
- www
- reverse proxy
maintainers:  # 维护人员列表 [*]
- email: 29ygq@sina.com
  name: Chinge Yang
sources:  # 应用程序来源 [-]
- https://github.com/bitnami/bitnami-docker-nginx

2.3 修改values.yaml

因为values.yaml设置涉及到yaml格式，yaml文件格式说明可以看这篇文章：

http://www.ruanyifeng.com/blog/2016/07/yaml.html

这里提几个常用的地方：

1. 使用2个空格作缩进；
2. 确认数字为字符类型时，使用双引号引起来；
3. 为了迎合helm3的规范，空定义最好将相关符号补上：

   string: ""
   list: []
   map: {}

没什么特殊要求，一般需要修改的地方有image、service、healthCheck、persistentVolume.mountPaths

# Default values for mod-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
global:  # 设置后覆盖后面默认的镜像仓库
  imageRegistry: ""
  imagePullSecrets: []
#     - myRegistryKeySecretName

statefulset:
  enabled: false

## String to partially override fullname template (will maintain the release name)
##
nameOverride: ""

## String to fully override fullname template
##
fullnameOverride: ""

## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .
## You can change type to `Recreate` or can uncomment `rollingUpdate` specification and adjust them to your usage.
deploymentStrategy: {}
  # rollingUpdate:
  #   maxSurge: 25%
  #   maxUnavailable: 25%
  # type: RollingUpdate

# 副本个数
replicaCount: 1

# 容器image及tag
image:
  registry: docker.io
  repository: bitnami/nginx
  tag: latest
  pullPolicy: IfNotPresent  # IfNotPresent: 有则不拉（减少流量和操作步骤），Always: 不管tag总拉（适合tag不变时更新）
  pullSecrets: []
  #  - private-registry-key

service:
  type: ClusterIP  # 一般不用修改
  ingressPort: 8080
  ports:
    http:  # 多端口暴露时，复制一段
      port: 8080  # Service port number for client-a port.
      protocol: TCP  # Service port protocol for client-a port.

## env set
## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
env: []
#  - name: DEMO_GREETING
#    value: "Hello from the environment"
#  - name: DEMO_FAREWELL
#    value: "Such a sweet sorrow"

## command set
startCommand: []
#  - "java -Xdebug -Xnoagent -Djava.compiler=NONE"
#  - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
#  - "-Djava.security.egd=file:/dev/urandom"
#  - "-jar /test.jar"
#  - "-Duser.timezone=GMT+08"

## Enable configmap and add data in configmap
config:
  enabled: false
  subPath: ""
  mountPath: /conf
  data: {}

############################# 示例 ####################################
## 以下示例，挂载文件至 /conf/app.conf
#  enabled: true
#  mountPath: /conf/app.conf  
#  subPath: app.conf    # 使用subPath时，上面mountPath路径写文件完整绝对路径
#  data:
#    app.conf: |-
#      appname = example-chart

## 以下示例，挂载多个文件至 /conf/ 下
#  enabled: true
#  mountPath: /conf    # 不使用subPath
#  data:
#    app.conf: |-
#      appname = example-chart
#    bpp.conf: |-
#      bppname
#
## 挂载多个文件至多个不同路径，需要相应修改 templates/deployment-statefulset.yaml 
############################# 示例 ####################################

## To use an additional secret, set enable to true and add data
## 用法同上，不另作说明
secret:
  enabled: false
  mountPath: /etc/secret-volume
  subPath: ""
  readOnly: true
  data: {} 

## liveness and readiness 
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
healthCheck:
  enabled: true
  type: tcp  # http/tcp
  port: http  # 健康检查的端口名或端口
  httpPath: '/'  # http时必须设置
  livenessInitialDelaySeconds: 10  # 初始延迟秒数
  livenessPeriodSeconds: 10  # 检测周期，默认值10，最小为1
  readinessInitialDelaySeconds: 10  # 初始延迟秒数
  readinessPeriodSeconds: 10   # 检测周期，默认值10，最小为1

resources: {}
  # 容器资源设置
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

## Node labels and tolerations for pod assignment
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
labels: {}
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
annotations: {}

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistentVolume:   # 是否存储持久化
  enabled: false
  ## If defined, storageClassName: 
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, azure-disk on
  ##   Azure, standard on GKE, AWS & OpenStack)
  ##
  storageClass: "-"
  accessMode: ReadWriteOnce
  annotations: {}
  #   helm.sh/resource-policy: keep
  size: 1Gi  # 大小
  existingClaim: {}  # 使用已存在的pvc
  mountPaths: []
  #  - name: data-storage
  #    mountPath: /config
  #    subPath: config  # 多个路径使用同一个pvc使用subPath，用法同上面config中示例说明
  #  - name: data-storage
  #    mountPath: /data
  #    subPath: data

ingress:  # 是否使用nginx暴露域名或端口
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: []

## Prometheus Exporter / Metrics
##
metrics:
  enabled: false
  image:
    registry: docker.io
    repository: nginx/nginx-prometheus-exporter
    tag: 0.1.0
    pullPolicy: IfNotPresent
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    pullSecrets: []
    #   - myRegistrKeySecretName
  ## Metrics exporter pod Annotation and Labels
  podAnnotations:
    # prometheus.io/scrape: "true"
    # prometheus.io/port: "9113"
    ## Metrics exporter resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
  resources: {}

## Uncomment and modify this to run a command after starting the core container.
## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
lifecycle: {}
  # preStop:
  #   exec:
  #     command: ["/bin/bash","/pre-stop.sh"]
  # postStart:
  #   exec:
  #     command: ["/bin/bash","/post-start.sh"]

## Deployment additional volumes.
deployment:
  additionalVolumes: []

## init containers
## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: {}
#  - name: fmp-volume-permission
#    image: busybox
#    imagePullPolicy: IfNotPresent
#    command: ['chown','-R', '200', '/extra-data']
#    volumeMounts:
#      - name: extra-data
#        mountPath: /extra-data

## Additional containers to be added to the core pod.
additionalContainers: {}
#  - name: my-sidecar
#    image: nginx:latest
#  - name: lemonldap-ng-controller
#    image: lemonldapng/lemonldap-ng-controller:0.2.0
#    args:
#      - /lemonldap-ng-controller
#      - --alsologtostderr
#      - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
#    env:
#      - name: POD_NAME
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.name
#      - name: POD_NAMESPACE
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.namespace
#    volumeMounts:
#    - name: copy-portal-skins
#      mountPath: /srv/var/lib/lemonldap-ng/portal/skins

2.4 修改README.md和templates/NOTES.txt

根据 values.yaml中的默认设置相应修改README.md，内容使用markdown语法，这里不作详细说明。
templates/NOTES.txt是部署chart后输出的帮助文档，里面支持go template语法。模板里已经写成了非常通用。必要情况下，适当按应用需求来修改，这样会显得部署后提示非常友好和人性化。

2.5 templates下yaml简要说明

templates目录下为kubernetes资源yaml文件模板，以资源名命名文件名，复杂些的可以加上资源功能或者模块名等。

templates/secret.yaml

{{- if .Values.secret.enabled }}  # if用法。语法代码注意形成一致缩进习惯，便于阅读
apiVersion: v1
kind: Secret
metadata:
  name: {{ template "example-chart.fullname" . }}  # 使用辅助模板时，点号可用 $ 代替，特别是在range下
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: "{{ .Release.Name }}"
    heritage: "{{ .Release.Service }}"
    {{- if .Values.labels }}
{{ toYaml .Values.labels | indent 4 }}  # 使用toYaml时，不缩进，indent接空格数
    {{- end }}
data:
{{- range $key, $value := .Values.secret.data }}  # range用法
  {{ $key }}: {{ $value | b64enc | quote }}  # secret中value注意使用base64转换
{{- end }}
{{- end }}

templates/deployment-statefulset.yaml

{{- if .Values.statefulset.enabled }}  # 判断使用deployment和statefulset资源api类型
apiVersion: apps/v1
kind: StatefulSet
{{- else }}
apiVersion: apps/v1
kind: Deployment
{{- end }}
metadata:
  name: {{ template "example-chart.fullname" . }}
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.labels }}  # 额外的标签
{{ toYaml .Values.labels | indent 4 }}
{{- end }}
{{- if .Values.annotations }}  # 自定义注释
  annotations:
{{ toYaml .Values.annotations | indent 4 }}
{{- end }}
spec:
  replicas: {{ .Values.replicaCount }}  # 副本数
  {{- if .Values.statefulset.enabled }}  # statefulset需要定义serviceName
  serviceName: {{ template "example-chart.fullname" . }}-headless
  {{- end }}
  {{- if .Values.deploymentStrategy }}
  strategy:
{{ toYaml .Values.deploymentStrategy | indent 4 }}
  {{- end }}
  selector:
    matchLabels:
      app: {{ template "example-chart.name" . }}
      release: {{ .Release.Name }}
  template:
    metadata:
      annotations:
  {{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}
{{- if .Values.metrics.podAnnotations }}
{{ toYaml .Values.metrics.podAnnotations | indent 8 }}
{{- end }}
      labels:
        app: {{ template "example-chart.name" . }}
        release: {{ .Release.Name }}
    spec:
{{- include "example-chart.imagePullSecrets" . | indent 6 }}
      {{- if .Values.initContainers }}
      initContainers:
{{ toYaml .Values.initContainers | indent 8 }}
      {{- end }}
      nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
      affinity:
{{ toYaml .Values.affinity | indent 8 }}
      tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
      containers:
{{- if .Values.metrics.enabled }}  # metrics容器可根据需求修改
        - name: metrics
          image: {{ template "metrics.image" . }}
          imagePullPolicy: {{ .Values.metrics.image.pullPolicy | quote }}
          command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status']
          ports:
          - name: metrics
            containerPort: 9113
          livenessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 15
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 5
            timeoutSeconds: 1
          resources:
{{ toYaml .Values.metrics.resources | indent 12 }}
{{- end }}
        - name: {{ .Chart.Name }}
          image: {{ template "example-chart.image" . }}
          imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
          {{- if .Values.lifecycle }}
          lifecycle:
{{ toYaml .Values.lifecycle | indent 12 }}
          {{- end }}
          {{- if .Values.startCommand }}
          command:
{{ toYaml .Values.startCommand |indent 12 }}
          {{- end }}
          env:
{{ toYaml .Values.env | indent 12 }}
          resources:
{{ toYaml .Values.resources | indent 12 }}
          ports:
            {{- range $key, $value := .Values.service.ports }}
            - name: {{ $key }}
              containerPort: {{ $value.port }}
              protocol: {{ $value.protocol }}
            {{- end }}
          {{- if .Values.healthCheck.enabled }}
          livenessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.livenessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.livenessPeriodSeconds }}
          readinessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.readinessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.readinessPeriodSeconds }}
          {{- end }}
          volumeMounts:  # 容器挂载点
            {{- if .Values.config.enabled }}
            - name: {{ template "example-chart.name" . }}-conf
              mountPath: {{ .Values.config.mountPath }}
              subPath: {{ .Values.config.subPath }}
            {{- end }}
            {{- if .Values.secret.enabled }}
            - name: {{ template "example-chart.name" . }}-secret
              mountPath: {{ .Values.secret.mountPath }}
              subPath: {{ .Values.secret.subPath }}
              readOnly: {{ .Values.secret.readOnly }}
            {{- end }}
{{- if .Values.persistentVolume.mountPaths }}
{{ toYaml .Values.persistentVolume.mountPaths | indent 12 }}
{{- end }}
        {{- if .Values.additionalContainers }}
{{ toYaml .Values.additionalContainers | indent 8 }}
        {{- end }}
      volumes:  # volume名需要和上文volumeMounts中的名字一一对应
        {{- if .Values.config.enabled }}
        - name: {{ template "example-chart.name" . }}-conf
          configMap:
            name: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.secret.enabled }}
        - name: {{ template "example-chart.name" . }}-secret
          secret:
            secretName: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.deployment.additionalVolumes }}
{{ toYaml .Values.deployment.additionalVolumes | indent 8 }}
        {{- end }}
{{- if not .Values.statefulset.enabled }}
          {{- if .Values.persistentVolume.enabled }}
        - name: data-storage
          persistentVolumeClaim:
            claimName: {{ .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }}
          {{- else }}
        - name: data-storage
          emptyDir: {}
          {{- end }}
{{- else }}
  {{- if .Values.persistentVolume.enabled }}
  volumeClaimTemplates:
  - metadata:
      name: data-storage
      labels:
        app: {{ template "example-chart.name" . }}
        chart: {{ template "example-chart.chart" . }}
        release: {{ .Release.Name }}
        heritage: {{ .Release.Service }}
    spec:
      accessModes:
        - {{ .Values.persistentVolume.accessMode | quote }}
      annotations:
      {{- range $key, $value := $.Values.persistentVolume.annotations }}
        {{ $key }}: {{ $value }}
      {{- end }}
      resources:
        requests:
          storage: {{ .Values.persistentVolume.size }}
          {{- if .Values.persistentVolume.storageClass }}
            {{- if (eq "-" .Values.persistentVolume.storageClass) }}
      storageClassName: ""
            {{- else }}
      storageClassName: "{{ .Values.persistentVolume.storageClass }}"
            {{- end }}
          {{- end }}
  {{- else }}
        - name: data-storage
          emptyDir: {}
  {{- end }}
{{- end -}}

3. 小结

以上yaml中未作详细说明，仔细看其内容都能明白大致意思。以下是对于helm chart新手的一些建议：

1. 刚接触helm chart时，多模仿stable和bianami中charts的写法，特别是values.yaml和templates目录中的一些设计，差不多都已经非常统一了。这样遇到自己有相似需求，可直接使用相应的功能块，写出来的chart也显得非常专业。
2. 使用chart时，最好使用helm命令fetch下来，大概读一遍其chart内容，这样看多了，自然就越来越熟悉，而且出错时，也便于自己排查问题。

参考资料：
[1] https://helm.sh/
[2] https://whmzsu.github.io/helm-doc-zh-cn/