Testing Challenges

Testing a challenge before submission ensures it works correctly and provides a good learning experience. This page covers testing strategies from initial deployment to edge cases.

Recommended workflow

# 1. Lint: validate challenge.yaml structure (no cluster needed)
kubeasy dev lint

# 2. Test: deploy and run validations in one step
kubeasy dev test --clean

# 3. Iterate with watch mode
kubeasy dev apply --watch    # terminal 1: auto-redeploys on file changes
kubeasy dev validate --watch # terminal 2: re-runs validations every 5s

# 4. Check pod state and logs while debugging
kubeasy dev status
kubeasy dev logs --follow

# 5. Clean up
kubeasy dev clean

Step 1: Lint first

Before deploying, validate your challenge.yaml structure:

kubeasy dev lint

This checks required fields, valid enum values, objective key uniqueness, sequential order values, and the existence of the manifests/ directory — all without a cluster. Fix all errors before proceeding.

Step 2: Verify the broken state

Deploy the challenge and confirm the problem is reproducible:

kubeasy dev apply --clean

Then inspect the cluster:

kubectl get pods
kubectl describe pod <pod-name>
kubectl logs <pod-name>
kubectl get events --sort-by='.lastTimestamp'

Ask yourself: if I were a user encountering this for the first time, would I see clear evidence of the problem? The broken state should be observable using standard kubectl commands.

Run validations against the broken state — they should all fail:

kubeasy dev validate

If any validation passes before the user applies a fix, review your objective logic.

Step 3: Test Kyverno policies

Apply the bypass that your policies should block, and confirm it's rejected:

# If you protect an image:
kubectl set image deployment/my-app app=nginx:latest
# Expected: admission webhook denied the request

Check policy status:

kubectl get clusterpolicies
kubectl describe clusterpolicy <policy-name>

Policies use kind: Policy (namespace-scoped) with metadata.namespace set to the challenge slug. This ensures they only enforce within that specific namespace and don't affect the rest of the cluster.

Step 4: Apply the fix and verify validations pass

Manually apply a valid solution, then run validations:

kubeasy dev validate

All objectives should pass. If some fail unexpectedly:

1. Read the failure message — it tells you exactly what was checked and why it failed
2. Inspect the resource manually with kubectl
3. Check that your label selectors actually match the resource

kubectl get pods --show-labels
kubectl get pods -l app=my-app

Step 5: Test alternative solutions

One of the most important tests: confirm that multiple valid approaches all pass your objectives.

If your challenge teaches fixing a memory limit, both 128Mi and 256Mi should work:

# Test solution 1
kubectl patch deployment my-app --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "128Mi"}]'
kubeasy dev validate

# Reset and test solution 2
kubeasy dev apply --clean
kubectl patch deployment my-app --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/resources/limits/memory", "value": "256Mi"}]'
kubeasy dev validate

If your validations only pass for one specific value, revisit your objective design. See Validation Overview — Philosophy.

Debugging validation failures

Condition checks

kubectl get pod <pod-name> -o jsonpath='{.status.conditions}' | jq .

Status checks

# Check a specific status field
kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].restartCount}'
kubectl get deployment <name> -o jsonpath='{.status.readyReplicas}'

Log checks

kubectl logs <pod-name>
kubectl logs <pod-name> --previous  # for terminated containers

Verify the exact strings from expectedStrings appear in the output.

Event checks

kubectl get events --sort-by='.lastTimestamp'
kubectl get events --field-selector reason=OOMKilled

After applying the fix, forbidden reasons should stop appearing within the sinceSeconds window.

Connectivity checks

# Test internal connectivity
kubectl run -it --rm debug --image=curlimages/curl --restart=Never -- \
  curl -v http://service-name:port/path

Timing considerations

Some objectives are time-sensitive:

• event and log validations use sinceSeconds (default: 300). After applying a fix, forbidden events from before the fix may still appear within the window. This is expected — wait for the window to expire, or test with a freshly-deployed challenge (--clean).
• triggered validations rely on waitAfterSeconds. Ensure this is long enough for the cluster to respond.
• condition validations check current state — no timing issues.

Testing on a fresh cluster

Always test on a fresh namespace before submitting:

kubeasy dev test --clean

The --clean flag deletes the namespace before redeploying, simulating a fresh start for the user.

Testing checklist

• kubeasy dev lint passes with no errors
• Broken state is reproducible (kubectl get pods shows the problem)
• All validations fail before any fix is applied
• Kyverno policies block the intended bypasses
• All validations pass after applying the fix
• Multiple valid solutions are accepted
• Validation titles don't reveal the solution
• Estimated time is accurate (test with someone unfamiliar with the problem)
• Challenge works from a clean state (--clean)

Clean up

kubeasy dev clean

Next steps

• Contributing Guidelines — PR requirements and the review process
• CLI Reference — Dev Mode — full reference for all kubeasy dev commands