Skip to content

Main Configuration Item List (CIL)

Purpose & Scope

Many KaaS products will have a CIL. This page is created to create an overview of the CILs, which are GIT-versioned and stored centrally or with their respective detailed documentation and source code, and a description of how they are managed.

Product CILs

CIL Product name CIL location Products covered Technology
Confluent Cloud (Platform Pipeline) confluent-infra CIL Access Management
Confluent Cloud		 Terraform
Team provisioning Pipeline access-mgmt CIL Access Management
Team Provisioning Pipeline		 Terraform
Data Product Pipeline Template N/A* Data Product Pipeline Template Terraform
MAP KaaS Pilot Application KaaS MAP Pilot CIL Pilot Application MAP
KaaS Pilot Application KaaS Pilot App - Tags Pilot Application Terraform
KaaS Audit Log Forwarder Kaas-log-forwarder CIL Logging MAP
Azure Pipeline Agent N/A** Azure Pipeline Agent AWS, Docker
Confluent Cloud (Network Connectivity) network-connectivity CIL Confluent Cloud
(network connectivity)		 AWS
Stream Governance Pipeline stream-gov CIL Stream Governance Pipeline Terraform

*configuration and code is still version-controlled with GIT and policies. kaas-pipeline-templates - Repos

**configuration and code is still version-controlled with GIT and policies. This includes the ECR-repositories in AWS, where the image is stored. Since we expect to patch the image frequently, and the component does not influence Confluent Cloud at runtime, semantic versioning is not used.

All changes to CIL will reference a CHG ID, or a RITM ID (for -config changes to Teams Pipeline, e.g. addition of Service Account) in serviceNow

CILs are managed in different ways depending on product; - Pipelines (Platform, Teams Provisioning and Stream governance) - Each environment will contain a commit hash, referencing all configuration (CIs) for areas: Organization (used by all environments) & the following environments: DEV, TST, QA, PROD, as well as the latest Pipeline-run ID (deployed configuration) and a human-entered CHG/RITM reference. The pipeline execution automatically generates the references and stores in the "platform-cil". - KaaS Audit Log forwarder/KaaS Mgmgt Application and KaaS MAP Pilot Application will have CILs stored as markdown files, that are manually updated and approved with the source code. - KaaS Pilot Application will have a version tag snapshot created, when the pipeline deploys the configuration. - Network connectivity is part of configuration of AWS TGW and whenever GITO updates this configuration, CIL requires the manual update to capture the current code Tag version.

Refactoring - Historic CILs

As part of CHG0136055 many of the KaaS Azure DevOps resources were refactored (restructured and split into other resources, but with unchanged functionality).

To remove the risk of using, editing or executing resources that were used in the deprecated, original ADO implementation, all old GIT repos have been disabled, and Pipelines and associated permissions and accesses have been deleted. GIT repos can be re-enabled by a ADO Project administrator. - Platform Pipeline GIT repo - Disabled - Teams Pipeline GIT repo - Disabled - Stream Governance Pipeline GIT repo - Disabled - Application Pipeline Template GIT repo - Disabled (new name = Data Product Pipeline template) - Azure Pipeline Agent GIT repo - Disabled

This has been assessed as acceptable, since the core platform infrastructure and configuration have not been changed (only users, service accounts and data products have been added on top), and the Refactor Change has re-established a new baseline.

There are no changes to functionality or to processes for product changes - Functional Code reviews and approvals follow the approach that was established initially. However, by splitting the core implementation logic from the input configuration, an 'ADO Operator' (Core Platform team member) can now peer review and approve configuration changes and deployment to the Team Provisioning Pipeline to optimize operational flows (e.g. adding new Service Accounts).

Roles & CIL Updates

Product Owner will approve this overview of CILs as per KaaS Artefacts, Reviews and Approvals, via Pull Request.

Lead architect to approve Product CILs in the following way; - Approve code and configuration changes, and deployment gates between environments on platform-pipeline which will automatically update CIL code tags - Approve deployment gates between environments on KaaS Pilot Application (using Data Product Pipeline template), which will automatically create a CIL code tag - Code Review and approve PR on MAP KaaS Pilot application CIL between QA & PRD deployment approvals - Code Review and approve PR on MAP application 'Kaas-Audit-log-forwarder' forwarding Logs CIL between QA and PRD deployment approvals - Code Review and approve PR on network connectivity CIL before deploying into Production - Approve code and configuration changes, and deployment gates between environments on Stream governance pipeline which will automatically update CIL code tags - Approve core functional logic changes on access-mgmt-modules (Team Provisioning Pipeline) which will automatically update CIL code tags when the pipeline is executed.

Platform Operator to approve Product config CILs in the following way; - Approve configuration input changes to access-management-config (Team Provisioning Pipeline) - Approve deployment gates between environments on access-management-prd (Team Provisioning Pipeline)

Scope of Product CILs with Terraform content

  • Functional implementation logic is stored in -module GIT Repos
  • Input configuration - e.g. environment names, cluster names and sizes - are stored in -config GIT Repos
    • All Terraform variable definitions are stored in /terraform/ folders
    • The variables and context are written in a human-readable format
  • All code and configuration is under GIT version control, and GIT versions (commit hashes) and Pipeline Run ID are stored in the 'platform-cil' every time the pipeline is run (also capturing CHG/RITM reference and a timestamp)
  • Terraform 'Plan step' allows users to have an overview of the impacted resources by running the pipeline, e.g. what is created, updated and deleted.
  • Access Management is covered in Platform-pipeline and Team Provisioning Pipeline. Platform Pipeline sets up the identity provider on Confluent Cloud (Azure AD), and the Team Provisioning Pipeline sets up service accounts and associates them with specified roles and permissions.
  • Code is subject to code review.
  • Code review is part of Pull request approval

Therefore the scope of CIs is limited to code tag versions only.

Scope of Product CILs with MAP content

CIs are build and code tag versions but do also contain configuration for MAP applications.