Skip to content

Using OAuth from your Kafka Clients

This guide provides configuration details and code examples for implementing OAuth authentication in Kafka client applications.

End-to-End Implementation Examples

For a complete, production-ready implementation of a Kafka client using OAuth, see the following resources:

  • Golang Example with CI/CD: The kaas-pilot-client repository on innersource-nn Github provides a full end-to-end implementation using Go and Confluent's confluent-kafka-go v2 (built on librdkafka C/C++ Apache Kafka client library), including CI/CD deployment to Ymir.

Code Examples

Confluent provides code examples and client implementations for multiple programming languages:

  • Golang: confluent-kafka-go/examples - Official Confluent examples
  • Other Languages: Confluent Cloud documentation provides examples for Java, Python, C#, Node.js, and more

For more details on OAuth implementations across different languages, refer to: - Use OAuth/OIDC to Authenticate to Confluent Cloud - OAuth Client Configuration Overview

OAuth Configuration for Kafka Clients

The following sections outline the configuration parameters needed for your Kafka client to use OAuth. Specific parameter names may vary per library and programming language, but are generally easy to identify.

Note: Providing an identity pool ID to your Kafka client is not required, due to the auto pool mapping feature. Confluent Cloud automatically matches your token to identity pools whose filter matches it.

Fixed Values for All Clients

The following values target Novo Nordisk's Azure Entra OAuth 2.0 token endpoint and the required kaas-oauth resource API. These values must be set in all Kafka clients to use OAuth:

1
2
sasl.oauthbearer.token.endpoint.url=https://login.microsoftonline.com/fdfed7bd-9f6a-44a1-b694-6e39c468c150/oauth2/v2.0/token
sasl.oauthbearer.scope=api://af713fe2-aab8-4437-8db5-47df799d134b/.default

What these values do: - token.endpoint.url: Points to Azure Entra ID's OAuth 2.0 token endpoint for Novo Nordisk - scope: Specifies the kaas-oauth resource API access scope needed for Kafka authentication

Fixed OIDC Parameters

These parameters should be the same in all officially supported OIDC-capable client libraries:

1
2
security.protocol=SASL_SSL
sasl.mechanism=OAUTHBEARER

What these values do: - security.protocol: Enables SSL/TLS encryption with SASL authentication - sasl.mechanism: Specifies the OAuth Bearer token authentication mechanism

Librdkafka, supported in Go, Python, C#, and other language wrappers, generally supports fully automated OIDC including token refresh using the OIDC method. This is the recommended approach:

1
sasl.oauthbearer.method=OIDC

Benefits: - Automatic token refresh without manual intervention - Simplified credential management - Consistent behavior across different client libraries

Variable Values - Kafka Connection Details

These values depend on the specific Confluent Cloud cluster you want to connect to. You can find these values under Cluster Settings in the Confluent Cloud UI:

1
2
3
bootstrap.servers=<bootstrap URL for the cluster you want to connect to>
sasl.oauth.logical.cluster=<cluster ID you are deploying to>
group.id=sa-<your identity-pool-name>

Configuration details: - bootstrap.servers: The bootstrap server address for your Confluent Cloud cluster (e.g., pkc-zd997.eu-central-1.aws.confluent.cloud:9092) - sasl.oauth.logical.cluster: The logical cluster ID for your Confluent Cloud cluster - group.id: Required for consumers only. Use the naming convention sa-<your-identity-pool-name> to stay within allowed consumer group prefixes

Client ID and Secret

Provision these credentials under your Azure Entra ID app registration in the Azure Portal. Handle sensitive values with necessary security precautions:

1
2
sasl.oauth.client.id=<your-client-id>
sasl.oauth.client.secret=<your-client-secret>

Security considerations: - Never commit secrets to version control - Use secrets management tools (e.g., Azure Key Vault, Kubernetes Secrets) - Rotate credentials regularly - Consider federated credentials as an alternative (see below)

Client ID and secret can be replaced with a secretless federated credential for completely federated authentication with no static access keys. This is the recommended secure approach.

How Federated Credentials Work

A federated credential establishes a trust relationship in Azure Entra ID between your app registration and an identity provider in your deployment environment (e.g., Kubernetes, GitHub Actions, EKS, AKS).

Token flow: 1. Runtime generates OIDC JWT: Your deployment environment (AKS/EKS/GitHub Actions) issues an OIDC JWT 2. Azure SDK exchanges for Entra token: The Azure SDK validates the JWT against the configured issuer and subject, then exchanges it for an access token 3. Kafka uses the token: The Kafka client uses the access token via SASL/OAUTHBEARER

No client secrets needed — the OIDC JWT itself is the proof of identity.

Configuration Steps

  1. Create a federated credential in your Azure Entra ID app registration (instead of a client secret)

    • Configure the external identity provider (e.g., AKS Workload Identity, EKS OIDC, GitHub Actions)
    • Set the subject claim that matches your workload

  2. Set up Azure SDK credential chain:

    • Default approach: Use WorkloadIdentityCredential, which reads the JWT from the AZURE_FEDERATED_TOKEN_FILE environment variable
    • Alternative: Use ClientAssertionCredential with a callback that returns the JWT (e.g., from an environment variable or in-memory cache)

  3. Implement OAuthBearer token refresh in your Kafka client:

    • Create a callback function that obtains an Azure Entra access token (using the Azure SDK with federated credentials)
    • Call SetOAuthBearerToken(...) to provide the token to the Kafka client
    • Handle token refresh as tokens expire

Supported OIDC Providers

Federated credentials work with any OIDC provider that has: - A stable issuer (iss claim) - A publicly available JWKS (JSON Web Key Set) - A predictable subject (sub claim)

Examples include: - AKS Workload Identity - EKS OIDC Provider - GitHub Actions - Other external OIDC issuers

Benefits of Federated Credentials

✅ No static secrets: Eliminates the risk of leaked credentials
✅ Automatic rotation: Tokens are short-lived and automatically refreshed
✅ Audit trail: All token requests are logged in Azure Entra ID
✅ Environment-specific: Different credentials per environment without management burden
✅ Compliance: Meets modern security and compliance requirements

Resources