Prepare a CockroachDB Cloud BYOC Deployment in Azure

This page describes how to prepare a cloud service account to host a of CockroachDB Cloud Advanced in Microsoft Azure.

Prerequisites

Review the . Make sure you understand and acknowledge the responsibilities you hold for management of your cloud infrastucture and the necessary permissions you must grant to Cockroach Labs.
if you do not already have one.
More than one BYOC cluster can run in a single Azure subscription.
The BYOC deployment option is not available by default and must be requested. Reach out to your account team to express interest in BYOC.
Once your cloud account is prepared for a CockroachDB BYOC deployment, cluster configuration and management is identical to a CockroachDB Cloud Advanced cluster. Review the documentation to plan your cluster sizing and resource allocation.
BYOC on Azure supports a restricted set of VM series. Confirm the supported VM series with your Cockroach Labs account team before sizing a deployment.
Review cloud service regions supported by .
The Azure subscription must not contain resources that can affect the control plane’s ability to operate on CockroachDB clusters.
(Optional) Create an to use the with your CockroachDB Cloud organization.

Step 1. Create a new Azure subscription

Provision a new Azure subscription dedicated to CockroachDB infrastructure. The account configuration for BYOC requires you to grant Cockroach Labs permissions to access and modify resources in this subscription, so this step is necessary to isolate these permissions from non-Cockroach Cloud resources. This subscription can be reused for multiple CockroachDB clusters.

Once this Azure subscription has been created and configured to host CockroachDB Cloud clusters, do not make additional modifications to the account. Changes to the cloud account can cause unexpected problems with cluster operations.

Step 2. Set up the admin App Registration

When BYOC is enabled for your account, Cockroach Labs dynamically provisions a multi-tenant admin App Registration associated with your CockroachDB Cloud organization and provides you with a URL to grant tenant-wide admin consent to the application. Granting admin consent creates an admin Service Principal in your tenant, which is used by Cockroach Labs support to act on the Kubernetes cluster, running automation that initializes support infrastructure. Visit this URL with a user account that is authorized to consent on behalf of your organization. Once the Cockroach Labs App Registration has been granted admin consent in the tenant, grant the following set of roles to the admin Service Principal:

Role Based Access Control Administrator
Azure Kubernetes Service Cluster User Role
Azure Kubernetes Service Contributor Role
Azure Kubernetes Service RBAC Cluster Admin
Managed Identity Contributor
Network Contributor
Storage Account Contributor
Storage Blob Data Contributor
Virtual Machine Contributor
A custom role, Resource Group Manager, with the following permissions:
- Microsoft.Resources/subscriptions/resourceGroups/read
- Microsoft.Resources/subscriptions/resourceGroups/write
- Microsoft.Resources/subscriptions/resourceGroups/delete
- Microsoft.Resources/subscriptions/resourceGroups/moveResources/action
- Microsoft.Resources/subscriptions/resourceGroups/validateMoveResources/action
- Microsoft.Resources/subscriptions/resourcegroups/deployments/read
- Microsoft.Resources/subscriptions/resourcegroups/deployments/write
- Microsoft.Resources/subscriptions/resourcegroups/resources/read
- Microsoft.Resources/subscriptions/resourcegroups/deployments/operations/read
- Microsoft.Resources/subscriptions/resourcegroups/deployments/operationstatuses/read

The custom Resource Group Manager role is required to create and manage resource groups in the subscription. This role is used instead of requesting the more broad Contributor role.

Step 3. Set up the reader App Registration

In addition to the admin application, Cockroach Labs provisions the CockroachDB Cloud BYOC Reader App Registration. This App Registration is used by Cockroach Labs support for read access to Kubernetes infrastructure. This reader application also requires admin consent to deploy the reader Service Principal:

Open the following URL in your browser:

https://login.microsoftonline.com/adminconsent?client_id=7f6538cb-f687-4411-9bbe-2f96bfbce028

If you have multiple tenants, replace customer-tenant-id in the following URL with the tenant containing your newly-created Azure subscription:

https://login.microsoftonline.com/{customer-tenant-id}/adminconsent?client_id=7f6538cb-f687-4411-9bbe-2f96bfbce028

Review the requested permissions and click Accept.
Once the CockroachDB Cloud BYOC Reader App Registration has been granted admin consent in the tenant, grant the following set of roles to the reader Service Principal:
- Reader
- Azure Kubernetes Service Cluster User
- Azure Kubernetes Service RBAC Reader

Step 4. Grant permissions to Entra groups with Azure Lighthouse

Use Azure Lighthouse to enable cross-tenant management that establishes the support infrastructure that allows Cockroach Labs to assist in the event of a support escalation. Permissions extended at the time of escalation are granted least-privilege access and full visibility, allowing you to review and remove access at any time from the Azure portal.

The permission policy described in this step is the minimum required for Cockroach Labs to provision, operate, patch, back up, and heal your cluster over its full lifecycle. This is not a temporary permission set used only at time of creation.Do not remove, narrow, or add conditions to these permissions after setup. Many of these are exercised only during a specific lifecycle event, such as replacing a failed node, applying a patch, rotating an encryption key, or running a managed backup. Removing a permission might not cause an immediate error. A later operation that requires it, such as replacing a failed node or running a backup, can fail. Reducing the permission set voids the availability commitment for the cluster.Guardrails that restrict the account to particular regions or services, without touching the permissions the deployment requires, are compatible. Azure policies that restrict this permission set are not supported.

This Azure Lighthouse deployment grants permissions to Cockroach Labs’s managed tenant, which has a tenant ID of a4611215-941c-4f86-b53b-348514e57b45, by assigning the following roles to the reader and admin Entra groups within the tenant:

Reader Entra group:
- Reader
- Azure Kubernetes Service Cluster User Role
Admin Entra group:
- Azure Kubernetes Service Contributor Role
- Azure Kubernetes Service Cluster Admin
- Managed Identity Contributor
- Network Contributor
- Storage Account Contributor
- Virtual Machine Contributor

Follow these steps to enable secure, scoped access for Cockroach Labs to your subscription using Azure Lighthouse:

Save the following ARM template to a file named byoc-lighthouse.json:

{
"$schema": "https://schema.management.azure.com/schemas/2019-08-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
  "mspOfferName": {
  "type": "string",
  "metadata": {
    "description": "Specify a unique name for your offer"
  },
  "defaultValue": "CockroachDB Cloud BYOC"
  },
  "mspOfferDescription": {
  "type": "string",
  "metadata": {
    "description": "Name of the Managed Service Provider offering"
  },
  "defaultValue": "Template for secure access to customer clusters in CockroachDB Cloud BYOC"
  }
},
"variables": {
  "mspRegistrationName": "[guid(parameters('mspOfferName'))]",
  "mspAssignmentName": "[guid(parameters('mspOfferName'))]",
  "managedByTenantId": "a4611215-941c-4f86-b53b-348514e57b45",
  "authorizations": [
  {
    "principalId": "c4139366-960c-431d-afad-29c65fd68087",
    "roleDefinitionId": "acdd72a7-3385-48ef-bd42-f606fba81ae7",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Reader Entra Group"
  },
  {
    "principalId": "c4139366-960c-431d-afad-29c65fd68087",
    "roleDefinitionId": "4abbcc35-e782-43d8-92c5-2d3f1bd2253f",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Reader Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "ed7f3fbd-7b88-4dd4-9017-9adb7ce333f8",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "0ab0b1a8-8aac-4efd-b8c2-3ee1fb270be8",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "e40ec5ca-96e0-45a2-b4ff-59039f2c2b59",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "4d97b98b-1d4f-4787-a291-c67834d212e7",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "17d1049b-9a84-46fb-8f53-869881c3d3ab",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  },
  {
    "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
    "roleDefinitionId": "9980e02c-c2be-4d73-94e8-173b1dc7cf3c",
    "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
  }
  ]
},
"resources": [
  {
  "type": "Microsoft.ManagedServices/registrationDefinitions",
  "apiVersion": "2022-10-01",
  "name": "[variables('mspRegistrationName')]",
  "properties": {
    "registrationDefinitionName": "[parameters('mspOfferName')]",
    "description": "[parameters('mspOfferDescription')]",
    "managedByTenantId": "[variables('managedByTenantId')]",
    "authorizations": "[variables('authorizations')]"
  }
  },
  {
  "type": "Microsoft.ManagedServices/registrationAssignments",
  "apiVersion": "2022-10-01",
  "name": "[variables('mspAssignmentName')]",
  "dependsOn": [
    "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
  ],
  "properties": {
    "registrationDefinitionId": "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
  }
  }
],
"outputs": {
  "mspOfferName": {
  "type": "string",
  "value": "[concat('Managed by', ' ', parameters('mspOfferName'))]"
  },
  "authorizations": {
  "type": "array",
  "value": "[variables('authorizations')]"
  }
}
}

Deploy the template at the subscription scope using Azure CLI, Azure PowerShell, or Azure Portal. The following example command uses the Azure CLI:

az deployment sub create \
  --name cockroach-byoc-lighthouse \
  --location {region} \
  --template-file byoc-lighthouse.json

Step 5. Register resource providers

Microsoft.ContainerService
Microsoft.ManagedIdentity
Microsoft.Network
Microsoft.Quota
Microsoft.Storage

Step 6. Create the CockroachDB Cloud cluster

In BYOC deployments, CockroachDB clusters can be deployed in the CockroachDB Cloud Console or with the .

Create a cluster with the CockroachDB Cloud Console

Follow these steps to create a CockroachDB cluster in the CockroachDB Cloud console:

Open the CockroachDB Cloud and select the organization that has been enabled for BYOC.
Click Create cluster.
Under Select a plan, click Advanced.
Under Cloud & Regions, click Bring Your Own Cloud and select Azure.
Under Cloud account, click Select your cloud account > Add new cloud account. Enter the tenant ID and subscription ID associated with your Azure subscription.
Follow the rest of the Create Cluster steps to configure your cluster’s regions, capacity, and features as desired. Read the documentation for more details.

Create a cluster with the CockroachDB Cloud API

Send a POST request to the the /v1/clusters endpoint to . The following example request creates a 3-node Advanced cluster in the centralus region, specifying the subscription-id and customer-tenant-id associated with your Azure subscription:

curl --request POST \
  --url https://cockroachlabs.cloud/api/v1/clusters \
  --header "Authorization: Bearer {secret_key}" \
  --json '{
    "name":"byoc-azure-cluster-1",
    "provider": "AZURE",
    "plan": "ADVANCED",
    "spec": {
      "customer_cloud_account": {
        "azure": {
          "subscription_id": "{subscription-id}",
          "tenant_id": "{customer-tenant-id}"
        }
      },
      "dedicated": {
        "hardware": {
          "machine_spec": {
            "num_virtual_cpus": 4
          },
          "storage_gib": 16
        },
        "region_nodes": {
          "centralus": 3
        }
      }
    }
  }'

Create additional BYOC clusters

An Azure subscription prepared for BYOC can support multiple CockroachDB Cloud clusters. To create an additional BYOC cluster in the same account, follow the Create the CockroachDB Cloud cluster steps again.

Delete a BYOC cluster

To delete a BYOC cluster, delete the cluster from the or with the . Deleting the cluster is permanent and cannot be undone. When you delete a BYOC cluster, Cockroach Labs removes the managed resources that it provisioned in your Azure subscription for the cluster. The admin and reader Service Principals created when you granted admin consent during setup and the Azure Lighthouse registration you deployed are not removed automatically and remain under your control. You can remove them once deletion is complete.

Do not remove the Service Principals, the Azure Lighthouse registration, or their role assignments before deletion has finished. Cockroach Labs needs them to remove the managed resources from your Azure subscription. Removing access early can leave resources behind that you will need to clean up manually, and that will continue to incur charges from your cloud provider.

Verify that deletion is complete

After the cluster reports as deleted in the CockroachDB Cloud Console, verify that no managed resources remain in your Azure subscription. Review the resource groups in the subscription for resources associated with the cluster. If resources remain, contact your Cockroach Labs account team before removing them manually.

Reuse the Azure subscription

Before you reuse the Azure subscription for a subsequent BYOC deployment, verify that deletion of the previous cluster is complete and contact your Cockroach Labs account team to confirm that the subscription is ready for reuse.

Get Started

Basic Deployments

Standard Deployments

Advanced Deployments

BYOC Deployments

Troubleshooting

Prepare a CockroachDB Cloud BYOC Deployment in Azure

Prerequisites

Step 1. Create a new Azure subscription

Step 2. Set up the admin App Registration

Step 3. Set up the reader App Registration

Step 4. Grant permissions to Entra groups with Azure Lighthouse

Step 5. Register resource providers

Step 6. Create the CockroachDB Cloud cluster

Create a cluster with the CockroachDB Cloud Console

Create a cluster with the CockroachDB Cloud API

Create additional BYOC clusters

Delete a BYOC cluster

Verify that deletion is complete

Reuse the Azure subscription

Next steps

​Prerequisites

​Step 1. Create a new Azure subscription

​Step 2. Set up the admin App Registration

​Step 3. Set up the reader App Registration

​Step 4. Grant permissions to Entra groups with Azure Lighthouse

​Step 5. Register resource providers

​Step 6. Create the CockroachDB Cloud cluster

​Create a cluster with the CockroachDB Cloud Console

​Create a cluster with the CockroachDB Cloud API

​Create additional BYOC clusters

​Delete a BYOC cluster

​Verify that deletion is complete

​Reuse the Azure subscription

​Next steps

Prerequisites

Step 1. Create a new Azure subscription

Step 2. Set up the admin App Registration

Step 3. Set up the reader App Registration

Step 4. Grant permissions to Entra groups with Azure Lighthouse

Step 5. Register resource providers

Step 6. Create the CockroachDB Cloud cluster

Create a cluster with the CockroachDB Cloud Console

Create a cluster with the CockroachDB Cloud API

Create additional BYOC clusters

Delete a BYOC cluster

Verify that deletion is complete

Reuse the Azure subscription

Next steps