Skip to main content

Multi-Environment Namespaces

Namespace-based multi-tenancy strategy deployed in Phase 56. A single k3s cluster hosts dev, staging, and prod environments via Kubernetes namespaces, with Kustomize overlays and an ArgoCD ApplicationSet generating all environments automatically.


Design​

ConcernDecision
Isolation unitKubernetes Namespace
Naming convention{team}-{env} (e.g. insurance-dev, collab-staging)
Source of truthminicloud-gitops/environments/overlays/{env}/{team}/
Deployment engineArgoCD ApplicationSet β€” matrix generator
Resource governanceNamespace-scoped ResourceQuota + LimitRange

Teams currently: insurance, collab. Environments: dev, staging, prod.


Namespace Layout​

environments/
└── overlays/
β”œβ”€β”€ dev/
β”‚ β”œβ”€β”€ insurance/
β”‚ β”‚ β”œβ”€β”€ kustomization.yaml
β”‚ β”‚ β”œβ”€β”€ 00-namespace.yaml
β”‚ β”‚ β”œβ”€β”€ 01-resourcequota.yaml
β”‚ β”‚ └── 02-limitrange.yaml
β”‚ └── collab/
β”‚ └── … (same structure)
β”œβ”€β”€ staging/
β”‚ β”œβ”€β”€ insurance/
β”‚ └── collab/
└── prod/
β”œβ”€β”€ insurance/ # namespace only β€” no quota
└── collab/

Resource Quotas​

EnvironmentCPU RequestMemory RequestStoragePodsPVCs
dev500m1 Gi20 Gi105
staging12 Gi50 Gi2010
prodβ€” (no quota)β€”β€”β€”β€”

prod has no ResourceQuota β€” resource usage is enforced via Grafana monitoring and alerts, not hard limits.

LimitRange Defaults​

EnvironmentDefault CPUDefault MemMax CPUMax Mem
dev200m / req 100m256Mi / req 128Mi12Gi
staging500m / req 200m512Mi / req 256Mi24Gi
prod500m / req 200m512Mi / req 256Mi48Gi

LimitRange defaults ensure every container in the namespace gets CPU/memory limits even when not set explicitly β€” required by the Gatekeeper require-resource-limits policy.


ArgoCD ApplicationSet​

apps/environments.yaml uses a matrix generator (3 envs Γ— 2 teams) to create 6 ArgoCD Applications automatically:

generators:
- matrix:
generators:
- list:
elements:
- env: dev
- env: staging
- env: prod
- list:
elements:
- app: insurance
- app: collab
template:
metadata:
name: "{{ .app }}-{{ .env }}"
spec:
source:
path: "environments/overlays/{{ .env }}/{{ .app }}"
destination:
namespace: "{{ .app }}-{{ .env }}"
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true

The 6 resulting apps: insurance-dev, insurance-staging, insurance-prod, collab-dev, collab-staging, collab-prod. All currently Synced + Healthy.


Kustomize Overlay Example (insurance-dev)​

# environments/overlays/dev/insurance/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: insurance-dev
resources:
- 00-namespace.yaml
- 01-resourcequota.yaml
- 02-limitrange.yaml
commonLabels:
env: dev
team: insurance

commonLabels propagates env and team labels to every resource in the overlay.


Cloudflare Tunnel Routes​

15 new routes registered at Phase 56 for app services in each env. Pattern:

SubdomainMaps to (originServerName)
dev.erp.devandre.sbsdev.erp.10.0.0.200.nip.io
staging.erp.devandre.sbsstaging.erp.10.0.0.200.nip.io
erp.devandre.sbs (prod)erp.10.0.0.200.nip.io

Apps covered: erp, intranet, wiki, chat, mail.

Tunnel config (~/.cloudflared/config.yml on controller):

- hostname: dev.erp.devandre.sbs
service: https://10.0.0.200
originRequest:
originServerName: dev.erp.10.0.0.200.nip.io
noTLSVerify: true

Adding a New Team​

  1. Create environments/overlays/dev/{team}/, staging/{team}/, prod/{team}/ with the 4 files from an existing team as template.
  2. Add - app: {team} to the ApplicationSet list generator in apps/environments.yaml.
  3. Commit and push β†’ ArgoCD creates the 3 new namespaces automatically.
  4. Register Cloudflare Tunnel DNS CNAMEs for any new public subdomains.

Verification​

# All 6 apps synced
kubectl --context minicloud get applications -n argocd \
| grep -E 'insurance|collab'

# Namespaces with labels
kubectl --context minicloud get namespaces \
-l team=insurance

# ResourceQuota in insurance-dev
kubectl --context minicloud get resourcequota -n insurance-dev

Expected: 6 namespaces, all Synced+Healthy, quotas showing 0 / <limit> for each resource.