Starting GitOps on the Homelab

I managed to get a barebones GitOps setup on my homelab.

I’m currently using this stack:

• Forgejo to manage git repositories
• Woodpecker to build container images
• ArgoCD to monitor changes in my helm charts and update the kubernetes cluster

The stack

Forgejo

Getting git repositories on forgejo was surprisingly easy. There’s a lot of options to configure but at the moment it’s a simple system with a small PVC storage. Once I have a s3 storage setup I’ll use that to store the container images. I have set my k3s cluster too small to be used as storage for large files, and it seems like the wrong approach anyway. For now I’m still using my Docker VM registry I installed a couple of weeks ago.

Woodpecker

That’s the part that got me the most troubles. Interestingly, not so much because of woodpecker itself, but because of the CA I used to handle all my name.local hosts, and the fact that I rely on /etc/hosts to access these.

The /etc/hosts issue

The hostnames I use work fine on my machine, but my cluster doesn’t have knowledge of it, obviously. I found out that I can create overrides for the coredns to map cluster hosts to the same names I used on my machine. That solved the core problem of getting forgejo and woodpecker to talk to each other.

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  forgejo.override: |
    rewrite name forgejo.local traefik.kube-system.svc.cluster.local
  woodpecker.override: |
    rewrite name woodpecker.local traefik.kube-system.svc.cluster.local

The CA issue

I’m using mkcert to create my own TLS certificates. I couldn’t manage to get woodpecker to trust it, so I had to use a few config tricks:

• WOODPECKER_FORGEJO_SKIP_VERIFY on the woodpecker config
• GITEA__webhook__SKIP_TLS_VERIFY on the forgejo config
• configuring the clone step in workflows to skip tls as well:

  - name: clone
    image: woodpeckerci/plugin-git
    settings:
      skip_verify: true

Not ideal, but until I step up to use a real domain name and external CA that will do. That will come soon anyway.

ArgoCD

Quick to install and get working. The longest step was to learn to convert my pageturner app into a Helm chart and plug that into ArgoCD, but I’ve already made a few releases and it looks like it’s syncing up fine :) I can see how this will be a great way to manage the whole homelab cluster state.

Next steps

I’m happy I got the whole flow working, but I’m unsure about using Woodpecker. I would like to try out the forgejo runner and the argo workflow tool. It’s a matter of wanting things to either be more integrated on the repo side or on the argo side.

But before that there’s a few core things I want to do:

• Convert all existing apps on the cluster to Helm chart and use ArgoCD to manage them.
• Install a S3 node into my proxmox system.
• Learn how to backup / migrate the cluster storage.
• Use real domains, both for private and externally visible parts of the system.
• Add a second hardware node to proxmox.

Once that’s done the system should feel more solid and ready to take larger workloads.