| Title | Kafka as a Service Solution Design - Connectors |
|---|---|
| Version | 1.0 |
| Approval Pull Request (PR) | PR229067 |
Solution Design - Connectors¶
Purpose & Scope¶
This Solution Design covers Connectors in Confluent Cloud - pre-built, managed Apache Kafka connectors, that make it easier to connect to popular data sources and sinks (targets). The connectors are built by Confluent or their partners, and receive full support. A list of connectors can be seen here: Connect to External Systems in Confluent Cloud | Confluent Documentation
While the platform team cannot assess and guarantee all connector behaviour, accesses are setup in a standardized way, that isolates the connectors between teams/namespaces. In a later version it is the target to get the Data Product Pipeline Template to deploy in traceable way with pipelines, See the Solution Designs section for an overview of the entire KaaS Solution.
Abbreviations and definitions¶
See Abbreviations and definitions
Solution Design Requirements¶
| ID | Name |
|---|---|
| 1 | Connector Access |
| 2 | Connector Deployment |
Connector Access¶
All Confluent Cloud connectors require credentials to allow the connector to operate and access Kafka. In the initial enablement of Connectors, user permissions are also added to existing roles, so that user can create Connectors (which are later permission mapped by a Platform Administrator)
- The detailed access for a connector service account and users are captured in Access Management
- The service accounts are configured in and deployed by the Teams Provisioning Pipeline
Connector Deployment¶
Usage of connectors will be defined and deployed via the UI initially, as the terraform provider used in pipelines does not adequately support security management. A developer role will be able to setup and configure Connectors in Dev/Tst/QA and an Operator in PRD.
Later, Usage of connectors will be defined and deployed via the Data Product Pipeline Template in a similar way as existing topics, schemas and other KaaS Assets. This ensures a standardized flow, with peer-reviews, approval gates etc. that are traceable in Azure DevOps GIT- and Pipeline history.
Traceability between Requirements¶
Traceability between Requirements (URS) and (SDR)¶
| ID | Work Item Type | URS (Status Dynamically Updated) | GIT SDR |
|---|---|---|---|
| 3144028 | URS | #3144028 | Solution Design Requirement: Connector Access Solution Design Requirement: Connector Deployment |
Change Log¶
| Version no. | CHG no. | Approved Pull Request | Date | Change Description |
|---|---|---|---|---|
| 1.0 | CHG0184608 | PR229067 | Date of acknowledgement | New document |