Kubernetes

Open Policy Agent

Operation CWAL 2021. 5. 5. 17:35

Request Workflow

관리자나 Pod이 Kubernetes 클러스터에 접근시, API 서버 내부적으로 다음과 같은 순서를 통해 Request를 처리한다. 모든 Workflow를 통과해야만, 정상적으로 작업이 이루어진다.

 

 

1. Authentication

클라이언트에서 전달한 TLS Certificate의 Common Name 필드를 조회하여, 유효한 사용자인지 검증한다. 간단히 말하면 현재 클라이언트가 누구인지 확인하는 과정이다.

 

2. Authorization

해당 사용자에 대한 RBAC(Role, RoleBinding, etc.)을 확인하여 Request에서 요구하는 작업을 수행할 권한이 있는지 체크한다.

 

3. Admission Control

Authentication 및 Authorization을 통과한 Request의 내용을 변경하거나, 내부적인 정책에 따라 Allow/Deny하는 역할을 담당한다. K8s에서 기본적으로 제공하는 Admission Control Plugin(ex: NodeRestriction, DefaultStorageClass) 외에 별도 외부 서비스와 연계할 수 있는 Admission Control Webhook을 추가할 수 있다.

순서상 Mutating Admission Controller가 Valdating Admission Controller보다 먼저 수행된다.

 

Open Policy Agent(OPA)

범용적으로 사용 가능하며, Open source로 제공되는 Policy Engine이다. 서비스 전체에 적용되는 Manifest에 대해 통일된 구성을 강제할 수 있으며, 다음과 같은 특성을 갖는다.

  • Kubernetes와는 독립적으로 존재하며, 다양한 시스템에 사용 가능
  • 'Rego' language를 통해 policy를 쉽게 구현할 수 있음
  • JSON, YAML 대상
  • K8s의 Admission Controller로 사용할 수 있음(Gatekeeper)
    • K8s 리소스(ex: Pod, Deployment)에 대한 개념이 존재하진 않음

OPA - Gatekeeper

K8s의 Validation Admission Controller로 사용 가능한 OPA 구현체(implementation)로, Pod 생성 또는 업데이트 요청을 검증한다. 'Constraint'을 통해 Policy를 정의하고, ConstraintTemplate을 사용하여 클러스터에서 해당 Policy를 시행할 수 있다.

 

Gatekeeper 설치

우선 다음과 같이 gatekeeper.yaml 파일을 작성한다.

apiVersion: v1
kind: Namespace
metadata:
  labels:
    admission.gatekeeper.sh/ignore: no-self-managing
    control-plane: controller-manager
    gatekeeper.sh/system: "yes"
  name: gatekeeper-system
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: configs.config.gatekeeper.sh
spec:
  group: config.gatekeeper.sh
  names:
    kind: Config
    listKind: ConfigList
    plural: configs
    singular: config
  scope: Namespaced
  validation:
    openAPIV3Schema:
      description: Config is the Schema for the configs API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: ConfigSpec defines the desired state of Config
          properties:
            match:
              description: Configuration for namespace exclusion
              items:
                properties:
                  excludedNamespaces:
                    items:
                      type: string
                    type: array
                  processes:
                    items:
                      type: string
                    type: array
                type: object
              type: array
            readiness:
              description: Configuration for readiness tracker
              properties:
                statsEnabled:
                  type: boolean
              type: object
            sync:
              description: Configuration for syncing k8s objects
              properties:
                syncOnly:
                  description: If non-empty, only entries on this list will be replicated into OPA
                  items:
                    properties:
                      group:
                        type: string
                      kind:
                        type: string
                      version:
                        type: string
                    type: object
                  type: array
              type: object
            validation:
              description: Configuration for validation
              properties:
                traces:
                  description: List of requests to trace. Both "user" and "kinds" must be specified
                  items:
                    properties:
                      dump:
                        description: Also dump the state of OPA with the trace. Set to `All` to dump everything.
                        type: string
                      kind:
                        description: Only trace requests of the following GroupVersionKind
                        properties:
                          group:
                            type: string
                          kind:
                            type: string
                          version:
                            type: string
                        type: object
                      user:
                        description: Only trace requests from the specified user
                        type: string
                    type: object
                  type: array
              type: object
          type: object
        status:
          description: ConfigStatus defines the observed state of Config
          type: object
      type: object
  version: v1alpha1
  versions:
    - name: v1alpha1
      served: true
      storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: constraintpodstatuses.status.gatekeeper.sh
spec:
  group: status.gatekeeper.sh
  names:
    kind: ConstraintPodStatus
    listKind: ConstraintPodStatusList
    plural: constraintpodstatuses
    singular: constraintpodstatus
  scope: Namespaced
  validation:
    openAPIV3Schema:
      description: ConstraintPodStatus is the Schema for the constraintpodstatuses API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        status:
          description: ConstraintPodStatusStatus defines the observed state of ConstraintPodStatus
          properties:
            constraintUID:
              description: Storing the constraint UID allows us to detect drift, such as when a constraint has been recreated after its CRD was deleted out from under it, interrupting the watch
              type: string
            enforced:
              type: boolean
            errors:
              items:
                description: Error represents a single error caught while adding a constraint to OPA
                properties:
                  code:
                    type: string
                  location:
                    type: string
                  message:
                    type: string
                required:
                  - code
                  - message
                type: object
              type: array
            id:
              type: string
            observedGeneration:
              format: int64
              type: integer
            operations:
              items:
                type: string
              type: array
          type: object
      type: object
  version: v1beta1
  versions:
    - name: v1beta1
      served: true
      storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.3.0
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: constrainttemplatepodstatuses.status.gatekeeper.sh
spec:
  group: status.gatekeeper.sh
  names:
    kind: ConstraintTemplatePodStatus
    listKind: ConstraintTemplatePodStatusList
    plural: constrainttemplatepodstatuses
    singular: constrainttemplatepodstatus
  scope: Namespaced
  validation:
    openAPIV3Schema:
      description: ConstraintTemplatePodStatus is the Schema for the constrainttemplatepodstatuses API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        status:
          description: ConstraintTemplatePodStatusStatus defines the observed state of ConstraintTemplatePodStatus
          properties:
            errors:
              items:
                description: CreateCRDError represents a single error caught during parsing, compiling, etc.
                properties:
                  code:
                    type: string
                  location:
                    type: string
                  message:
                    type: string
                required:
                  - code
                  - message
                type: object
              type: array
            id:
              description: 'Important: Run "make" to regenerate code after modifying this file'
              type: string
            observedGeneration:
              format: int64
              type: integer
            operations:
              items:
                type: string
              type: array
            templateUID:
              description: UID is a type that holds unique ID values, including UUIDs.  Because we don't ONLY use UUIDs, this is an alias to string.  Being a type captures intent and helps make sure that UIDs and names do not get conflated.
              type: string
          type: object
      type: object
  version: v1beta1
  versions:
    - name: v1beta1
      served: true
      storage: true
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  creationTimestamp: null
  labels:
    controller-tools.k8s.io: "1.0"
    gatekeeper.sh/system: "yes"
  name: constrainttemplates.templates.gatekeeper.sh
spec:
  group: templates.gatekeeper.sh
  names:
    kind: ConstraintTemplate
    plural: constrainttemplates
  scope: Cluster
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            crd:
              properties:
                spec:
                  properties:
                    names:
                      properties:
                        kind:
                          type: string
                        shortNames:
                          items:
                            type: string
                          type: array
                      type: object
                    validation:
                      type: object
                  type: object
              type: object
            targets:
              items:
                properties:
                  libs:
                    items:
                      type: string
                    type: array
                  rego:
                    type: string
                  target:
                    type: string
                type: object
              type: array
          type: object
        status:
          properties:
            byPod:
              items:
                properties:
                  errors:
                    items:
                      properties:
                        code:
                          type: string
                        location:
                          type: string
                        message:
                          type: string
                      required:
                        - code
                        - message
                      type: object
                    type: array
                  id:
                    description: a unique identifier for the pod that wrote the status
                    type: string
                  observedGeneration:
                    format: int64
                    type: integer
                type: object
              type: array
            created:
              type: boolean
          type: object
  version: v1beta1
  versions:
    - name: v1beta1
      served: true
      storage: true
    - name: v1alpha1
      served: true
      storage: false
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-admin
  namespace: gatekeeper-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-manager-role
  namespace: gatekeeper-system
rules:
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-manager-role
rules:
  - apiGroups:
      - '*'
    resources:
      - '*'
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - config.gatekeeper.sh
    resources:
      - configs
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - config.gatekeeper.sh
    resources:
      - configs/status
    verbs:
      - get
      - patch
      - update
  - apiGroups:
      - constraints.gatekeeper.sh
    resources:
      - '*'
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
  - apiGroups:
      - status.gatekeeper.sh
    resources:
      - '*'
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - templates.gatekeeper.sh
    resources:
      - constrainttemplates
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - templates.gatekeeper.sh
    resources:
      - constrainttemplates/finalizers
    verbs:
      - delete
      - get
      - patch
      - update
  - apiGroups:
      - templates.gatekeeper.sh
    resources:
      - constrainttemplates/status
    verbs:
      - get
      - patch
      - update
  - apiGroups:
      - admissionregistration.k8s.io
    resourceNames:
      - gatekeeper-validating-webhook-configuration
    resources:
      - validatingwebhookconfigurations
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-manager-rolebinding
  namespace: gatekeeper-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: gatekeeper-manager-role
subjects:
  - kind: ServiceAccount
    name: gatekeeper-admin
    namespace: gatekeeper-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-manager-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gatekeeper-manager-role
subjects:
  - kind: ServiceAccount
    name: gatekeeper-admin
    namespace: gatekeeper-system
---
apiVersion: v1
kind: Secret
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-webhook-server-cert
  namespace: gatekeeper-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-webhook-service
  namespace: gatekeeper-system
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    control-plane: controller-manager
    gatekeeper.sh/operation: webhook
    gatekeeper.sh/system: "yes"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    control-plane: controller-manager
    gatekeeper.sh/operation: audit
    gatekeeper.sh/system: "yes"
  name: gatekeeper-audit
  namespace: gatekeeper-system
spec:
  replicas: 1
  selector:
    matchLabels:
      control-plane: audit-controller
      gatekeeper.sh/operation: audit
      gatekeeper.sh/system: "yes"
  template:
    metadata:
      annotations:
        container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
      labels:
        control-plane: audit-controller
        gatekeeper.sh/operation: audit
        gatekeeper.sh/system: "yes"
    spec:
      containers:
        - args:
            - --operation=audit
            - --operation=status
            - --logtostderr
          command:
            - /manager
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          image: openpolicyagent/gatekeeper:469f747
          imagePullPolicy: Always
          livenessProbe:
            httpGet:
              path: /healthz
              port: 9090
          name: manager
          ports:
            - containerPort: 8888
              name: metrics
              protocol: TCP
            - containerPort: 9090
              name: healthz
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /readyz
              port: 9090
          resources:
            limits:
              cpu: 1000m
              memory: 512Mi
            requests:
              cpu: 100m
              memory: 256Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - all
            runAsGroup: 999
            runAsNonRoot: true
            runAsUser: 1000
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: gatekeeper-admin
      terminationGracePeriodSeconds: 60
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    control-plane: controller-manager
    gatekeeper.sh/operation: webhook
    gatekeeper.sh/system: "yes"
  name: gatekeeper-controller-manager
  namespace: gatekeeper-system
spec:
  replicas: 1
  selector:
    matchLabels:
      control-plane: controller-manager
      gatekeeper.sh/operation: webhook
      gatekeeper.sh/system: "yes"
  template:
    metadata:
      annotations:
        container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
      labels:
        control-plane: controller-manager
        gatekeeper.sh/operation: webhook
        gatekeeper.sh/system: "yes"
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: gatekeeper.sh/operation
                      operator: In
                      values:
                        - webhook
                topologyKey: kubernetes.io/hostname
              weight: 100
      containers:
        - args:
            - --port=8443
            - --logtostderr
            - --exempt-namespace=gatekeeper-system
            - --operation=webhook
          command:
            - /manager
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          image: openpolicyagent/gatekeeper:469f747
          imagePullPolicy: Always
          livenessProbe:
            httpGet:
              path: /healthz
              port: 9090
          name: manager
          ports:
            - containerPort: 8443
              name: webhook-server
              protocol: TCP
            - containerPort: 8888
              name: metrics
              protocol: TCP
            - containerPort: 9090
              name: healthz
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /readyz
              port: 9090
          resources:
            limits:
              cpu: 1000m
              memory: 512Mi
            requests:
              cpu: 100m
              memory: 256Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - all
            runAsGroup: 999
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
            - mountPath: /certs
              name: cert
              readOnly: true
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: gatekeeper-admin
      terminationGracePeriodSeconds: 60
      volumes:
        - name: cert
          secret:
            defaultMode: 420
            secretName: gatekeeper-webhook-server-cert
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  creationTimestamp: null
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-validating-webhook-configuration
webhooks:
  - clientConfig:
      caBundle: Cg==
      service:
        name: gatekeeper-webhook-service
        namespace: gatekeeper-system
        path: /v1/admit
    failurePolicy: Ignore
    name: validation.gatekeeper.sh
    namespaceSelector:
      matchExpressions:
        - key: admission.gatekeeper.sh/ignore
          operator: DoesNotExist
    rules:
      - apiGroups:
          - '*'
        apiVersions:
          - '*'
        operations:
          - CREATE
          - UPDATE
        resources:
          - '*'
    sideEffects: None
    timeoutSeconds: 5
  - clientConfig:
      caBundle: Cg==
      service:
        name: gatekeeper-webhook-service
        namespace: gatekeeper-system
        path: /v1/admitlabel
    failurePolicy: Fail
    name: check-ignore-label.gatekeeper.sh
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - '*'
        operations:
          - CREATE
          - UPDATE
        resources:
          - namespaces
    sideEffects: None
    timeoutSeconds: 5

Namespace 'gatekeeper-system'이 생성되며, 그 안에 Validating Admission Controller를 구현한 많은 리소스가 생성된다.

 

Gatekeeper - Policy 정의 및 적용

우선 모든 Pod의 생성을 제한하는 Policy를 다음과 같이 정의할 수 있다.

 

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8salwaysdeny
spec:
  crd:
    spec:
      names:
        kind: K8sAlwaysDeny
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            message:
              type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8salwaysdeny
        violation[{"msg": msg}] {
          1 > 0
          msg := input.parameters.message
        }

<k8s-always-deny-template.yaml>

 

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAlwaysDeny
metadata:
  name: pod-always-deny
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    message: "ACCESS DENIED!"

<pod-always-deny.yaml>

 

 

k8s-always-deny-template.yaml은 Constraint로 사용할 CRD(Custom Resource Definition)를 명시하고, 어떻게 검증할지에 대한 동작과 Parameter를 정의해야 한다. rego 필드 내용을 확인하면,  '1 > 0' 이라는 항상 true인 조건을 주고 있으며, 무조건 violation이 발생하므로 해당 Request는 내용과 상관없이 실패한다.

pod-always-deny.yaml은 Gatekeeper의 Constraint를 정의한 파일로, 정책을 적용할 Resource와 APIGroup을 명시하고 Parameter에 대한 Value를 지정한다.

 

이제 위 파일들을 차례대로 클러스터에 적용한 뒤, Pod을 생성해보자.

kubectl apply -f k8s-always-deny-template.yaml
kubectl apply -f pod-always-deny.yaml\
kubectl run nginx --image=nginx

다음과 같이, 미리 정의한 message 내용이 출력되면서 Pod 생성에 실패하는 것을 확인할 수 있다.

 

아래 명령어를 실행하면, 어떤 Request가 해당 Constraint을 통과 또는 실패했는지 확인할 수 있다.

kubectl describe K8sAlwaysDeny pod-always-deny

 

 

이번엔 보다 실용적인 Policy를 추가해보자. 위에서 작업한 ConstraintTemplate과 Constraint는 제거한다.

kubectl delete -f pod-always-deny.yaml
kubectl delete -f k8s-always-deny-template.yaml

 

리소스의 metadata.labels 필드를 확인하여 특정 Label이 존재하는지 확인하는 Policy를 적용하려고 한다. 우선 다음과 같이 ConstraintTemplate을 추가한다.

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            labels:
              type: array
              items: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

<k8srequiredlabels_template.yaml>

 

그리고 아래와 같이 Constraint을 정의한다. 전자는 Namespace에 foo라는 label의 존재 여부, 후자는 Pod에 hello라는 label의 존재 여부를 확인하는 Policy이다.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-label-foo
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["foo"]

<ns-must-have-label-foo.yaml>

 

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: pod-must-have-label-hello
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    labels: ["hello"]

<pod-must-have-label-hello.yaml>

 

다음 명령어로 현재 클러스터에 해당 Policy를 적용하자.

kubectl apply -f k8srequiredlabels_template.yaml
kubectl apply -f ns-must-have-label-foo.yaml
kubectl apply -f pod-must-have-label-hello.yaml

 

Policy가 제대로 반영되었는지 확인하기 위해 아래와 같은 Pod을 추가한다. 'hello' label이 없기 때문에, 실패해야 한다.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx

 

deny 동작은 문제없음을 확인하였으므로, 이번엔 'hello' label을 추가해보고 다시 시도해보자. 이번엔 성공해야 한다.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    hello: world
spec:
  containers:
  - image: nginx
    name: nginx

Pod에 대한 Policy는 우리가 원하는대로 적용된 것을 알 수 있다. 마찬가지로 Namespace에 대해 테스트해보자.

kubectl create ns my-namespace

위 명령어는 'foo' label이 없는 Namespace를 생성하려고 하기 때문에, 다음과 같은 메시지를 출력하며 실패한다

 

반면, 다음과 같이 'foo' label을 정의한 Namespace를 생성하면 성공하는 것을 알 수 있다.

apiVersion: v1
kind: Namespace
metadata:
  name: my-namespace
  labels:
    foo: bar

 

마치며...

Gatekeeper를 잘 활용할 수 있다면, K8s 리소스에 대한 일관적인 정책을 적용할 수 있기 때문에 관리의 효율성과 보안 향상에 많은 도움이 된다. 다만, Rego라는 Policy Language에 대한 이해가 일정 수준 필요하므로, 관련한 추가 학습이 수반될 것으로 보인다.

참고

'Kubernetes' 카테고리의 다른 글

PodDisruptionBudget을 활용한 Application 보호  (0) 2021.09.22
ETCD Backup&Restore  (0) 2021.05.29
ETCD Encryption  (0) 2021.04.21
Imperative Vs. Declarative  (0) 2021.04.08
Container Runtime과 Docker  (0) 2021.03.09