Terraform remote state backend fails with Azure Key Vault and CMEK setup

Issue

The deployment of a rule-compliant Terraform remote state backend with Azure Storage and customer-managed encryption keys (CMEK) failed due to the following issues:

1. Existing Key Vault resources could not be deleted due to compliance retention requirements.
2. Diagnostic settings were misconfigured at the incorrect scope (storage account vs. blob service level).
3. The service principal lacked the SetRotationPolicy permission, preventing automatic key rotation.
4. Existing encryption keys could not be recreated, violating data continuity requirements.

Culprit

Existing Resources Conflict

Azure compliance rules prevent deletion of Key Vaults and encryption keys containing audit trails. The initial Terraform configuration attempted to recreate these resources instead of importing existing ones.

Diagnostic Settings Misconfiguration

Diagnostic settings were incorrectly applied at the storage account level for logs, while Azure requires:

• Metrics configured at the storage account level.
• Logs configured at the blob service level.

Insufficient Key Vault Permissions

The service principal had GetRotationPolicy permissions but was missing the critical SetRotationPolicy permission, which is required for automatic key rotation.

Resource Import Requirements

To maintain compliance and continuity, existing encryption keys and Key Vaults needed to be imported into Terraform state rather than recreated. This ensures preservation of audit trails, encryption history, and compliance adherence.

Solution

Resource Import

Existing Key Vault and encryption key resources were imported into Terraform state to maintain compliance and continuity.

Diagnostic Settings Correction

Diagnostic settings were split into two distinct configurations:

• Storage Account Metrics: Applied at the storage account level to capture transaction metrics.
• Blob Service Logs: Applied at the blob service level to log storage read operations.

Permission Elevation

The Key Vault access policy was updated to include the SetRotationPolicy permission, enabling automatic key rotation.

Banking-Grade Security Implementation

The following security measures were implemented:

1. Customer-Managed Encryption: Storage account configured to use a customer-managed key from Azure Key Vault.
2. Automatic Key Rotation: Encryption keys configured with a rotation policy ensuring 90-day expiration and automatic renewal.
3. State Locking: Blob versioning enabled with a 30-day retention policy for container deletions to ensure data integrity.