In a Kubernetes multi-cluster deployment environment, scripts were intended to target a local cluster but defaulted to a global cluster context due to an oversight in a shared helper function. This caused deployment failures as resources were applied to the wrong cluster.
Issue
Deployment scripts failed with namespace not found errors despite the namespace existing on the target local cluster. The scripts were incorrectly applying resources to the global cluster context instead of the configured local cluster, disrupting the entire deployment pipeline.
Culprit
The issue stemmed from a helper function (get_kubectl()) in shared deployment scripts. This function was responsible for constructing kubectl commands but did not include the cluster context flag, even when the environment variable CLUSTER_CONTEXT was set. As a result, all kubectl commands defaulted to the global cluster context, bypassing the intended local cluster configuration.
The function originally returned only the base kubectl command without context awareness:
- No handling for the
CLUSTER_CONTEXTenvironment variable. - No fallback mechanism for environments where
kubectlis not directly available.
Solution
Updated Helper Function
The get_kubectl() function was modified to:
- Include context awareness: Automatically append the
--contextflag whenCLUSTER_CONTEXTis set. - Support fallback mechanisms: Handle environments where
kubectlis not in the PATH (e.g., usingsudo k3s kubectl). - Preserve backward compatibility: Maintain the original behavior when no context is specified.
Key Changes
- Conditional context flag: The function now checks for the presence of
CLUSTER_CONTEXTand appends--context ${CLUSTER_CONTEXT}to the command if set. - Environment compatibility: Ensures the function works in both standard and non-standard
kubectlenvironments.
Verification
To ensure the fix was effective, the following steps were taken:
- Verified that all
kubectlcommands in scripts now include the context flag. - Tested deployments by explicitly setting the global context to a different cluster to confirm the correct cluster was targeted.
- Conducted a search for all
kubectlusages in deployment scripts to ensure consistency.
This change resolved the issue by ensuring kubectl commands respected the configured cluster context, preventing misdeployments to unintended clusters.