Configure data flow endpoints for Microsoft Fabric Real-Time Intelligence

SUBSCRIPTION_ID=<subscription-id>
RESOURCE_GROUP=<resource-group-name>
AIO_INSTANCE_NAME=<instance-name>
CLUSTER_NAME=<cluster-name>
LOCATION=<region>

$SUBSCRIPTION_ID = "<subscription-id>"
$RESOURCE_GROUP = "<resource-group-name>"
$AIO_INSTANCE_NAME = "<instance-name>"
$CLUSTER_NAME = "<cluster-name>"
$LOCATION = "<region>"

This method uses a managed identity to authenticate with the eventstream. Use either system-assigned managed identity or user-assigned managed identity when you configure the data flow endpoint.

1. Go to the connection details in the Fabric portal under the Sources section of your eventstream.

2. In the Details pane for the custom endpoint, select the Kafka protocol.

3. Select the Entra ID Authentication section to view the connection details.

4. Copy the details for the Bootstrap server and Topic name values. You use these values to configure the data flow endpoint.

   

   Settings	Description
   Bootstrap server	The bootstrap server address is used for the host name property in the data flow endpoint.
   Topic name	The event hub name is used as the Kafka topic and is in the format es_aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb.

1. Go to the connection details in the Fabric portal under the Sources section of your eventstream.

2. In the Details pane for the custom endpoint, select the Kafka protocol.

3. Select the SAS Key Authentication section to view the connection details.

4. Copy the details for the Bootstrap server, Topic name, and Connection string-primary key values. You use these values to configure the data flow endpoint.

   

   Settings	Description
   Bootstrap server	The bootstrap server address is used for the host name property in the data flow endpoint.
   Topic name	The event hub name is used as the Kafka topic and is in the format es_aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb.
   Connection string-primary key	The connection string with the primary key.

1. In the Azure IoT Operations experience portal, select the Data flow endpoints tab.

2. Under Create new data flow endpoint, select Microsoft Fabric Real-Time Intelligence > New.

3. Enter the following settings for the endpoint.

   

   Setting	Description
   Name	The name of the data flow endpoint.
   Host	The host name of the eventstream custom endpoint in the format <bootstrap-server>.servicebus.windows.net:9093. Use the bootstrap server address noted previously.
   Authentication method	The method used for authentication. Choose System-assigned managed identity, User-assigned managed identity, or SASL.

   Use the authentication method that matches how you want Azure IoT Operations to connect to the custom endpoint of the Fabric eventstream:

   • System-assigned managed identity: Use the Azure IoT Operations Azure Arc extension identity to authenticate with the eventstream. Before you create the endpoint, add the extension identity to the Fabric workspace with permissions of Contributor or higher. To learn more, see System-assigned managed identity.
   • User-assigned managed identity: Use a user-assigned managed identity configured for Azure IoT Operations cloud connections. Before you create the endpoint, add the user-assigned managed identity to the Fabric workspace with permissions of Contributor or higher. To learn more, see User-assigned managed identity.
   • SASL: Use the custom endpoint connection string of the Fabric eventstream. This method requires Simple Authentication and Security Layer (SASL) settings and a synced Kubernetes secret with the username and password values. To learn more, see SASL.

4. Select Apply to provision the endpoint.

Create or replace

Use the following az iot ops dataflow endpoint create fabric-realtime command to create or replace a Real-Time Intelligence data flow endpoint:

az iot ops dataflow endpoint create fabric-realtime --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --host "$BOOTSTRAP_SERVER_ADDRESS"

Use the following example command to create or replace a Real-Time Intelligence data flow endpoint named fabric-realtime-endpoint:

az iot ops dataflow endpoint create fabric-realtime --resource-group myResourceGroup --instance myAioInstance --name fabric-realtime-endpoint --host "fabricrealtime.servicebus.windows.net:9093"

Create or change

Use the following az iot ops dataflow endpoint apply command to create or change a Real-Time Intelligence data flow endpoint:

az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json

The --config-file parameter is the path and file name of a JSON configuration file that contains the resource properties.

In this example, assume that a configuration file named fabric-realtime-endpoint.json with the following content is stored in the user's home directory:

{
  "endpointType": "Kafka",
  "kafkaSettings": {
    "host": "<BootstrapServerAddress>",
    "authentication": {
      "method": "Sasl",
      "saslSettings": {
        "saslType": "Plain",
        "secretRef": "<SecretName>"
      }
    },
    "tls": {
      "mode": "Enabled"
    }
  }
}

Use the following example command to create a new Real-Time Intelligence data flow endpoint named fabric-realtime-endpoint:

az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-realtime-endpoint --config-file ~/fabric-realtime-endpoint.json

Create a .bicep file with the following content:

param aioInstanceName string = '<AIO_INSTANCE_NAME>'
param customLocationName string = '<CUSTOM_LOCATION_NAME>'
param endpointName string = '<ENDPOINT_NAME>'
param hostName string = '<BOOTSTRAP_SERVER_ADDRESS>'
param secretName string = '<SECRET_NAME>'

resource customLocation 'Microsoft.ExtendedLocation/customLocations@2021-08-31-preview' existing = {
  name: customLocationName
}

resource aioInstance 'Microsoft.IoTOperations/instances@2026-07-01' existing = {
  name: aioInstanceName
}

resource fabricRealtimeEndpoint 'Microsoft.IoTOperations/instances/dataflowEndpoints@2026-07-01' = {
  parent: aioInstance
  name: endpointName
  extendedLocation: {
    name: customLocation.id
    type: 'CustomLocation'
  }
  properties: {
    endpointType: 'Kafka'
    kafkaSettings: {
      host: hostName
      authentication: {
        method: 'Sasl'
        saslSettings: {
          saslType: 'Plain' // Or ScramSha256, ScramSha512
          secretRef: secretName
        }
      }
      tls: {
        mode: 'Enabled'
      }
    }
  }
}

Deploy the file via the Azure CLI:

az deployment group create --resource-group $RESOURCE_GROUP --template-file main.bicep

kubectl create secret generic $SECRET_NAME -n azure-iot-operations \
  --from-literal=username='$ConnectionString' \
  --from-literal=password="$CONNECTION_STRING_PRIMARY_KEY"

Create a Kubernetes manifest .yaml file with the following content:

apiVersion: connectivity.iotoperations.azure.com/v1
kind: DataflowEndpoint
metadata:
  name: <ENDPOINT_NAME>
  namespace: azure-iot-operations
spec:
  endpointType: Kafka
  kafkaSettings:
    host: <BOOTSTRAP_SERVER_ADDRESS>
    authentication:
      method: Sasl
      saslSettings:
        saslType: Plain # Or ScramSha256, ScramSha512
        secretRef: <SECRET_NAME>
    tls:
      mode: Enabled

Apply the manifest file to the Kubernetes cluster:

kubectl apply -f main.yaml

On the operations experience tab for the data flow endpoint settings page, select the Basic tab, and then select Authentication method > System-assigned managed identity.

Create or replace

Use the az iot ops dataflow endpoint create fabric-realtime command with the --auth-type parameter set to SystemAssignedManagedIdentity for system-assigned managed identity authentication. In most cases, you don't need to specify --audience. The default audience matches the endpoint host in the form https://<NAMESPACE>.servicebus.windows.net.

az iot ops dataflow endpoint create fabric-realtime --auth-type SystemAssignedManagedIdentity --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --host "$BOOTSTRAP_SERVER_ADDRESS"

Create or change

Use the az iot ops dataflow endpoint apply command with the --config-file parameter.

In this example, assume a configuration file with the following content:

{
    "endpointType": "Kafka",
    "kafkaSettings": {
        "host": "fabricrealtime.servicebus.windows.net:9093",
        "authentication": {
            "method": "SystemAssignedManagedIdentity",
            "systemAssignedManagedIdentitySettings": {}
        },
        "tls": {
            "mode": "Enabled"
        }
    }
}

kafkaSettings: {
  authentication: {
    method: 'SystemAssignedManagedIdentity'
    systemAssignedManagedIdentitySettings: {}
  }
}

kafkaSettings:
  authentication:
    method: SystemAssignedManagedIdentity
    systemAssignedManagedIdentitySettings:
      {}

On the operations experience tab for the data flow endpoint settings page, select the Basic tab, and then select Authentication method > User-assigned managed identity.

Create or replace

Use the az iot ops dataflow endpoint create command with the --auth-type parameter set to UserAssignedManagedIdentity for user-assigned managed identity authentication.

az iot ops dataflow endpoint create fabric-realtime --auth-type UserAssignedManagedIdentity --client-id $CLIENT_ID --tenant-id $TENANT_ID --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --host "$BOOTSTRAP_SERVER_ADDRESS"

Create or change

Use the az iot ops dataflow endpoint apply command with the --config-file parameter.

In this example, assume that a configuration file has the following content:

{
    "endpointType": "Kafka",
    "kafkaSettings": {
        "host": "<BootstrapServerAddress>",
        "authentication": {
          "method": "UserAssignedManagedIdentity",
          "userAssignedManagedIdentitySettings": {
            "clientId": "<ID>",
            "tenantId": "<ID>"
          }
        },
        "tls": {
            "mode": "Enabled"
        }
    }
}

The scope property is optional. If needed, set it to the audience for the Kafka broker, such as https://<BootstrapServerAddress>.

kafkaSettings: {
  authentication: {
    method: 'UserAssignedManagedIdentity'
    userAssignedManagedIdentitySettings: {
      clientId: '<CLIENT_ID>'
      tenantId: '<TENANT_ID>'
      // Optional
      // scope: 'https://<SCOPE_URL>'
    }
  }
  ...
}

kafkaSettings:
  authentication:
    method: UserAssignedManagedIdentity
    userAssignedManagedIdentitySettings:
      clientId: <CLIENT_ID>
      tenantId: <TENANT_ID>
      # Optional
      # scope: https://<SCOPE_URL>

1. On the operations experience tab for the data flow endpoint settings page, select the Basic tab, and then select Authentication method > SASL.

2. Enter the following settings for the endpoint:

   Setting	Description
   SASL type	The type of SASL authentication to use. For Fabric custom endpoints, select Plain.
   Synced secret name	Enter a name for the synced secret. A Kubernetes secret with this name is created on the cluster.
   Username reference of token secret	The reference to the username or token secret used for SASL authentication.
   Password reference of token secret	The reference to the password or token secret used for SASL authentication.

3. Select Add reference to create a new Key Vault reference or choose an existing Key Vault reference for the username and password references:

   • Username reference of token secret: A Key Vault secret whose value is the literal string $ConnectionString (including the leading $).

     

   • Password reference of token secret: The secret value must be the connection string with the primary key from the custom endpoint of the eventstream.

     

Create or replace

Use the az iot ops dataflow endpoint create command with the --auth-type parameter set to Sasl for SASL authentication.

az iot ops dataflow endpoint create fabric-realtime --auth-type Sasl --sasl-type $SASL_TYPE --secret-name $SECRET_NAME --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --host "$BOOTSTRAP_SERVER_ADDRESS"

Create or change

Use the az iot ops dataflow endpoint apply command with the --config-file parameter.

In this example, assume that a configuration file has the following content:

{
    "endpointType": "Kafka",
    "kafkaSettings": {
        "host": "<BootstrapServerAddress>",
        "authentication": {
          "method": "Sasl",
          "saslSettings": {
            "saslType": "<SaslType>",
            "secretRef": "<SecretName>"
          }
        },
        "tls": {
            "mode": "Enabled"
        }
    }
}

kafkaSettings: {
  authentication: {
    method: 'Sasl'
    saslSettings: {
      saslType: 'Plain' // Or ScramSha256, ScramSha512
      secretRef: '<SECRET_NAME>'
    }
  }
}

kubectl create secret generic $SECRET_NAME -n azure-iot-operations \
  --from-literal=username='$ConnectionString' \
  --from-literal=password="$CONNECTION_STRING_PRIMARY_KEY"

kafkaSettings:
  authentication:
    method: Sasl
    saslSettings:
      saslType: Plain # Or ScramSha256, ScramSha512
      secretRef: <SECRET_NAME>