Deploy OCI artifacts and Helm charts the GitOps way with Config Sync

One principle of GitOps is to have the desired state declarations as Versioned and Immutable, where Git repositories play an important role as the source of truth. But can you have an alternative to a Git repository for storing and deploying your Kubernetes manifests via GitOps? What if you could package your Kubernetes manifests into a container image instead? What if you can reuse the same authentication and authorization mechanism as your container images?

To answer the above questions, an understanding of OCI registries and OCI artifacts is needed. Simply put, OCI registries are the registries that are typically used for container images but can be expanded to store other types of data (aka OCI artifacts) such as Helm charts, Kubernetes manifests, Kustomize overlays, scripts, etc.

Using OCI Registries and OCI Artifacts provides you with the following advantages:

Less tools to operate: Single artifact registry can store expanded data types apart from container images.
In-built release archival system: OCI registries give users two sets of mutable and immutable URLs which are tags and content-addressable ones.
Flourishing ecosystem: Standardized and supported by dozen of providers which helps users take advantage of new features and tools developed by large Kubernetes community

Given these benefits, and in addition to the support of files stored in Git repositories, we are thrilled to announce two new formats supported by Config Sync 1.13 to deploy OCI artifacts:

Config Sync is an open source tool that provides GitOps continuous delivery for Kubernetes clusters.

The Open Container Initiative (OCI) is an open governance structure for the express purpose of creating open industry standards around container formats and runtimes. OCI artifacts give you the power of storing and distributing different types of data such as Kubernetes manifests, Helm Charts, and Kustomize overlays, in addition to container images via OCI registries.

Throughout this blog, you will see how you can leverage the two new formats (OCI artifacts and Helm charts) supported by Config Sync, by using:

oras and helm to package and push OCI artifacts
Artifact registry as OCI registry to store the OCI artifacts
GKE cluster to host the OCI artifacts synced
Config Sync installed in that GKE cluster to sync the OCI artifacts

Initial setup

First, you need to have a common setup for the two scenarios by configuring and securing the access from the GKE cluster with Config Sync to the Artifact Registry repository.

Initialize the Google Cloud project you will use throughout this blog:

code_block: [StructValue([(u’code’, u’PROJECT=SET_YOUR_PROJECT_ID_HERErngcloud config set project $PROJECT’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca253a6c10>)])]

Create a GKE cluster with Workload Identity registered in a fleet to enable Config Management:

code_block: [StructValue([(u’code’, u’CLUSTER_NAME=oci-artifacts-clusterrnREGION=us-east4rngcloud services enable container.googleapis.comrngcloud container clusters create ${CLUSTER_NAME} \rn –workload-pool=${PROJECT}.svc.id.goog \rn –region ${REGION}rngcloud services enable gkehub.googleapis.comrngcloud container fleet memberships register ${CLUSTER_NAME} \rn –gke-cluster ${REGION}/${CLUSTER_NAME} \rn –enable-workload-identityrngcloud beta container fleet config-management enable’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca263d4e10>)])]

Install Config Sync in the GKE cluster:

code_block: [StructValue([(u’code’, u’cat <<EOF > acm-config.yamlrnapplySpecVersion: 1rnspec:rn configSync:rn enabled: truernEOFrngcloud beta container fleet config-management apply \rn –membership ${CLUSTER_NAME} \rn –config acm-config.yaml’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca263d4cd0>)])]

Create an Artifact Registry repository to host OCI artifacts (--repository-format docker):

code_block: [StructValue([(u’code’, u’CONTAINER_REGISTRY_NAME=oci-artifactsrngcloud services enable artifactregistry.googleapis.comrngcloud artifacts repositories create ${CONTAINER_REGISTRY_NAME} \rn –location ${REGION} \rn –repository-format docker’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca15b519d0>)])]

Create a dedicated Google Cloud Service Account with the fine granular access to that Artifact Registry repository with the roles/artifactregistry.reader role:

code_block: [StructValue([(u’code’, u’GSA_NAME=oci-artifacts-readerrngcloud iam service-accounts create ${GSA_NAME} \rn –display-name ${GSA_NAME}rngcloud artifacts repositories add-iam-policy-binding ${CONTAINER_REGISTRY_NAME} \rn –location ${REGION} \rn –member “serviceAccount:${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com” \rn –role roles/artifactregistry.reader’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca15b51090>)])]

Allow Config Sync to synchronize resources for a specific RootSync:

code_block: [StructValue([(u’code’, u’ROOT_SYNC_NAME=root-sync-ocirngcloud iam service-accounts add-iam-policy-binding \rn –role roles/iam.workloadIdentityUser \rn –member “serviceAccount:${PROJECT}.svc.id.goog[config-management-system/root-reconciler-${ROOT_SYNC_NAME}]” \rn ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca26ce6cd0>)])]

code_block: [StructValue([(u’code’, u’gcloud auth configure-docker ${REGION}-docker.pkg.dev’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca26ce61d0>)])]

Build and sync an OCI artifact

Now that you have completed your setup, let’s illustrate our first scenario where you want to sync a Namespace resource as an OCI image.

Create a Namespace resource definition:

code_block: [StructValue([(u’code’, u’cat <<EOF> test-namespace.yamlrnapiVersion: v1rnkind: Namespacernmetadata:rn name: testrnEOF’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca26ce6350>)])]

Create an archive of that file:

code_block: [StructValue([(u’code’, u’tar -cf test-namespace.tar test-namespace.yaml’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca26ce6990>)])]

Push that artifact to Artifact Registry. In this tutorial, we use oras, but there are other tools that you can use like crane.

code_block: [StructValue([(u’code’, u’oras push \rn ${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}/my-namespace-artifact:v1 \rn test-namespace.tar’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca26ce6290>)])]

Set up Config Sync to deploy this artifact from Artifact Registry:

code_block: [StructValue([(u’code’, u’cat << EOF | kubectl apply -f -rnapiVersion: configsync.gke.io/v1beta1rnkind: RootSyncrnmetadata:rn name: ${ROOT_SYNC_NAME}rn namespace: config-management-systemrnspec:rn sourceFormat: unstructuredrn sourceType: ocirn oci:rn image: ${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}/my-namespace-artifact:v1rn dir: .rn auth: gcpserviceaccountrn gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.comrnEOF’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca14f1eb10>)])]

Check the status of the sync with the nomos tool:

code_block: [StructValue([(u’code’, u’nomos status –contexts $(k config current-context)’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca14f1ef50>)])]

Verify that the Namespace test is synced:

code_block: [StructValue([(u’code’, u’kubectl get ns test’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca14f1e990>)])]

And voilà! You just synced a Namespace resource as an OCI artifact with Config Sync.

Build and sync a Helm chart

Now, let’s see how you could deploy a Helm chart hosted in a private Artifact Registry.

Create a simple Helm chart:

code_block: [StructValue([(u’code’, u’helm create test-chart’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca25ff7f50>)])]

Package the Helm chart:

code_block: [StructValue([(u’code’, u’helm package test-chart –version 0.1.0′), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca2434b150>)])]

Push the chart to Artifact Registry:

code_block: [StructValue([(u’code’, u’helm push \rn test-chart-0.1.0.tgz \rn oci://${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca1580b890>)])]

Set up Config Sync to deploy this Helm chart from Artifact Registry:

code_block: [StructValue([(u’code’, u’cat << EOF | kubectl apply -f -rnapiVersion: configsync.gke.io/v1beta1rnkind: RootSyncrnmetadata:rn name: ${ROOT_SYNC_NAME}rn namespace: config-management-systemrnspec:rn sourceFormat: unstructuredrn sourceType: helmrn helm:rn repo: oci://${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}rn chart: test-chartrn version: 0.1.0rn releaseName: test-chartrn namespace: defaultrn auth: gcpserviceaccountrn gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.comrnEOF’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca1580b290>)])]

Check the status of the sync with the nomos tool:

code_block: [StructValue([(u’code’, u’nomos status –contexts $(k config current-context)’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca1580bc10>)])]

Verify that the resources in the Namespace test-chart are synced:

code_block: [StructValue([(u’code’, u’kubectl get all -n default’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3eca25bd62d0>)])]

And voilà! You just synced an Helm chart with Config Sync.

Towards more scalability and security

In this blog, you synced both an OCI artifact and an Helm chart with Config Sync.

OCI registries and OCI artifacts are new kids on the block that can also work alongside with the Git option depending on your needs and use-cases. One of such patterns could be Git

still acting as the source of truth for the declarative configs in addition to the well established developer workflow it provides: pull request, code review, branch strategy, etc.

The continuous integration pipelines, triggered by pull requests or merges, will run tests against the declarative configs to eventually push the OCI artifacts in an OCI registry.

Finally, the continuous reconciliation of GitOps will take it from here and will reconcile between the desired state, now stored in an OCI registry, with the actual state, running in Kubernetes. Your Kubernetes manifests as OCI artifacts are now just seen like any container images for your Kubernetes clusters as they are pulled from OCI registries. This continuous reconciliation from OCI registries, not interacting with Git, has a lot of benefits in terms of scalability, performance and security as you will be able to configure very fine grained access to your OCI artifacts.

To get started, check out the two Sync OCI artifacts from Artifact Registry and the Sync Helm charts from OCI registries features today.

You could also find this other tutorial showing how you can package and push an Helm chart to GitHub Container Registry with GitHub actions, and then how you can deploy this Helm chart with Config Sync.

Attending KubeCon + CloudNativeCon North America 2022 in October? Come check out our session Build and Deploy Cloud Native (OCI) Artifacts, the GitOps Way during the GitOpsCon North America 2022 co-located event on October, 25th. Hope to see you there!

Config Sync is open sourced. We are open to contributions and bug fixes if you want to get involved in the development of Config Sync. You can also use the repository to track ongoing work, or build from source to try out bleeding-edge functionalities.