Introduction
This document outlines on how we can use crossplane in integration with argo cd to create resources in cloud. This document will not dive deep into the what and how crossplane works or the argocd. Here, I am going to outline the steps the would help create resources using infrastructure as code with crossplane
Pre requisites
Crossplane using kube api, hence we we would need a kubernetes cluster. Follow tis document if you want to setup a local kubernetes cluster Setup Kubernetes cluster with Lightweight K3S
Argo CD installation
First lets setup or agro cd for gitops implementation of our infrastructure creation. the following steps are similar to what was outline in the argocd offical documentation. For more information, visti Getting started with Argocd
First, lets create a namespace,
1
kubectl create namespace argocd
install the required CRDs
1
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Also, install the argocd cli using brew which will help in obtaining the initial login password. Now, in order test the deployment of the UI, we can do a port forward to and access the application locally at http://localhost:8080
1
kubectl port-forward svc/argocd-server -n argocd 8080:443
once, we are sure that this application is setup properly and is working, we will move with ahead and create an ingress for the traefik so, we can access the argocd dashboard at a custom domain with self signed certificates from Lets Encrypt.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: argocd
namespace: argocd
annotations:
kubernetes.io/ingress.class: <traefik_instance_name>
spec:
entryPoints:
- websecure
routes:
- match: Host(`<custom_domain_based_on_traefik_configuration>`)
kind: Rule
services:
- name: argocd-server
port: 443
- match: Host(`<custom_domain_based_on_traefik_configuration>`)
kind: Rule
services:
- name: argocd-server
port: 443
middlewares:
- name: <middleware_if_any>
tls:
secretName: <cert_manager_kuberters_secret_name>
Now, we can do a kubectl apply -f ingress.yaml which will allow us to access the argocd dashboard at the specified custom domain.
NOTE
In order for this to work, make sure that the cert-manager certificate, which is a kubernetes certificate, is available or copied to the newly created namespace
Crossplane
For this demonstration, I will be using Azure but we can also use crossplane to create resources in AWS and GCP. For more information refer the documentation: Getting started with Crossplane.
In order to install Crossplane, We will be utilizing the helm chart that crossplane offers, there are also several alternatives available to install crossplane mentioned in the documentation
1
2
3
4
5
6
# add crossplane helm repo
helm repo add \
crossplane-stable https://charts.crossplane.io/stable
# udpate install helm repos
helm repo update
verify the installation
1
kubectl get pods -n crossplane-system
this should output something like
1
2
3
NAME READY STATUS RESTARTS AGE
crossplane-5755f8bcb6-246nn 1/1 Running 0 173m
crossplane-rbac-manager-b98b48664-wrp2g 1/1 Running 0 173m
with this installation, we have basically created new kubernets api endpoints, which can be verified using
1
kubectl api-resources | grep crossplane
which should output something like below,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
compositeresourcedefinitions xrd,xrds apiextensions.crossplane.io/v1 false CompositeResourceDefinition
compositionrevisions comprev apiextensions.crossplane.io/v1 false CompositionRevision
compositions comp apiextensions.crossplane.io/v1 false Composition
environmentconfigs envcfg apiextensions.crossplane.io/v1beta1 false EnvironmentConfig
usages apiextensions.crossplane.io/v1beta1 false Usage
configurationrevisions pkg.crossplane.io/v1 false ConfigurationRevision
configurations pkg.crossplane.io/v1 false Configuration
controllerconfigs pkg.crossplane.io/v1alpha1 false ControllerConfig
deploymentruntimeconfigs pkg.crossplane.io/v1beta1 false DeploymentRuntimeConfig
functionrevisions pkg.crossplane.io/v1 false FunctionRevision
functions pkg.crossplane.io/v1 false Function
imageconfigs pkg.crossplane.io/v1beta1 false ImageConfig
locks pkg.crossplane.io/v1beta1 false Lock
providerrevisions pkg.crossplane.io/v1 false ProviderRevision
providers pkg.crossplane.io/v1 false Provider
storeconfigs
Now, that we have verified the installation, we need to install CRDs required for access cloud providers api. These providers will help crossplane with creating resources using the kube api’s. For Azure, my provider.yaml file looks something like this
1
2
3
4
5
6
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-family-azure
spec:
package: xpkg.upbound.io/upbound/provider-family-azure:v1
With this manifest, I am basically installing all the CRDs required for crossplane to communicate with azure. But there are other smaller and more focused provider that will work for any specific type of azure resources. For example, if we just want to create some virtual networks, we can use provider-azure-networks, which can be installed using the below configuration
1
2
3
4
5
6
7
8
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-azure-network
spec:
package: xpkg.crossplane.io/crossplane-contrib/provider-azure-network:v1.11.2
EOF
Moving on to the authentication piece, in order for crossplane to create resources, it should be able to authenticate with the environment, for this we also have couple of options like federated credentials or secrets. for simiplicity we are going to use a client secret.
1
2
3
4
5
az ad sp create-for-rbac \
--sdk-auth \
--role Owner \
--display-name <name_of_app_regirstration> \
--scopes /subscriptions/<subscription_id> > azure.json
For more information of differnet authentication methods, visit the crossplane documentation Authentications for Crossplane
The above CLI command will create a service principal with owner permissions on the specified scope and will also generate client_id, tenant_id and client_secret and store them locally in a json file. Now, this file is highly sensitive, make sure to keep it safe. Once we have this file, we are going to convert this to a kubernetes secret, which we will later use in all our crossplane configurations to create managed resources.
1
2
3
4
kubectl create secret \
generic azure-secret \
-n crossplane-system \
--from-file=creds=./azure.json
Now that we have create this secrets, we can reference this in a providerConfig.yaml file, which is basically points crossplane to which secrets to use for differnet resources. My providerConfig, looks something like this,
1
2
3
4
5
6
7
8
9
10
11
apiVersion: azure.upbound.io/v1beta1
metadata:
name: <metadata_name>
kind: ProviderConfig
spec:
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: <secret_name>
key: <secret_key>
Since we are trying a gitops approach, we also have to have an argocd application that is monitoring this repo for any changes and to apply them in our environment.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: <application_name>
namespace: <argocd_namespace>
spec:
project: default
source:
repoURL: <gitlab/github_url>
path: <folder_path>
targetRevision: <branch>
destination:
server: <kubernetes_server_url> ## for default, leave it as is.
namespace: <crossplane_namespace>
syncPolicy:
automated:
prune: true
selfHeal: true
and apply this file.
Now that we have all pieces on the management side of things to create a managed resources using crossplane, I am going to create a resource group, using the follwoing yaml. Now push this file to gitlab repo
1
2
3
4
5
6
7
8
9
10
kind: ResourceGroup
metadata:
name: <resource_group_name>
spec:
forProvider:
location: <location>
tags:
<tag_key>: <tag_value>
providerConfigRef:
name: <providerConfig_ref>
When we push this file to the git repo, the argo cd application will look at the changes made to this repo and will try to apply those changes. Remeber, argocd needs to have a reference, so, first we have to apply the resource group yaml file manually before leveraging argocd.
Reference
For more information on crossplane intergration with different providers, visit crossplane