| Title | Kafka as a Service - Test Traceability |
|---|---|
| Version | 22.0 |
| Approval Pull Request (PR) | PR294922 |
Test Traceability¶
Purpose & Scope¶
This document shows the traceability from User Requirements to Test cases.
Traceability from Requirements to Test Cases¶
This document shows the traceability from User Requirements to Test cases. This will be used as part of test planning. Before finalising testing this page must be approved via a PR. The page will be subject to change control and updates must be captured in change log and via new PR.
Abbreviations and definitions¶
See Abbreviations and definitions
Traceability between Requirements¶
Traceability between Requirements (URS) and Test Cases¶
| Product | URS | Unit Test Case(s) | IV Test Case(s) | p-IV Test Case(s) | OV/IT Test Case(s) | Test Plan(s) |
|---|---|---|---|---|---|---|
| Access Management | #1318367 | N/A | N/A | #1775328 | Verified in RBAC requirement #1703076 #1704733 #1704734 #1704735 #1704736 #1704737 |
#1699873 |
| Access Management | #1337204 | N/A | N/A | N/A - prerequisite Identity Provider configuration is deployed by platform-pipeline #1610649 | Verified in Pilot App #1547354 #1786565 #1786567 #1786585 |
#1701743 |
| Access Management | #1374619 | N/A | N/A | #1775401 | #1775392 | #1699873 |
| Access Management | #1374669 | N/A | N/A | N/A - Roles are deployed by Team Provisioning Pipeline #1701705 #1773872 |
#1703076 #1704733 #1704734 #1704735 #1704736 #1704737 #2005135 #2423088 #2423078 #2788354 #2788355 | #1703044 #1997592 #2284163 #2775445 |
| Access Management | #1421694 | N/A | N/A | #1775450 | N/A - Standard NN Process not verified | #1699873 |
| Access Management & Confluent Cloud | #1374692 | N/A | N/A | N/A - Deployed by platform Pipeline #1610649 #1719526 |
Verified in RBAC requirement #1703076 #1704733 #1704734 #1704735 #1704736 #1704737 #2423088 #2423078 |
N/A #2284163 #2775445 |
| Confluent Cloud | #1422279 | N/A | #1610649 #1651011 | #1719526 #1719536 #2228384 | N/A #2534820 | #1827361 #2213009 #2541155 #2619274 |
| Confluent Cloud | #1422561 | N/A | N/A | #1827759 | #1701554 | #1827361 #1704657 |
| Confluent Cloud | #1425495 | N/A | N/A - Standard functionality | N/A - Standard Functionality | #1701540 #1704705 | #1704657 |
| Azure Pipeline Agent | #1533706 | N/A | N/A | #1701581 #1773775 #2376316 | N/A - Preqreuisite for and verified by Teams-Provisioning Pipeline & Application Template/KaaS Pilot App successfully executing #2376830 #2376842 #2376847 | #1701569 #2364200 #2517519 #2527458 #2594883 #2792531 |
| Team Provisioning Pipeline | #1445237 | N/A | #1701705 | #1773872 #2228424 | Verified in RBAC requirement #1703076 #1704733 #1704734 #1704735 #1704736 #1704737 #2423071 #2423090 #2423089 #2423078 #2423088 #2535212 |
#1701597 #2213009 #2284163 #2541155 #2619274 #2775445 #2792531 |
| Data Product Pipeline Template (Application Pipeline Template) | #1933853 #1422289 | N/A | N/A | #1775789 #2125326 #2125397 #2565556 | Verified in KaaS Pilot App (using template) #1767831 #2122483 #2541165 #2600223 | #1701723 #2077515 #2213009 #2517519 #2541155 #2594883 #2598959 #2792531 |
| Pilot Application | #1422189 | #1547347 | #1699759 | #1767831 | #1547354 #1786565 #1786567 #1786585 | #1701743 |
| Pilot Application | #1422220 | N/A | #1701752 | #1767811 #2125397 | Verified in KaaS MAP Pilot App Integration Test (IT) #1547354 #1786565 #1786567 #1786585 #2122483 #2541165 | #1701743 #2077515 #2517519 #2541155 #2594883 #2725962 |
| Logging | #1417078 | #1547347 | N/A | #1701187 | #1845705 | #1701171 #2076884 #2593913 |
| Logging | #1520146 | Verified in System Audit Log #1547347 | N/A | Verified in System Audit Log #1701187 | #1704648 | #1703080 |
| Stream Governance Pipeline | #1932759 | N/A | N/A | #2047240 #2228478 | #2130646 #2535214 | #2131253 #2213009 #2541155 #2619274 |
| Connectors | #3144028 | N/A | N/A | #3069897 | #3154277 #3123224 | #3137723 |
| Flink | #3185126 | N/A | N/A | #3249247 | #3249630 #3249929 | #2618623 |
| Flink | #3185126 | N/A | N/A | N/A | #3255713 #3255715 | #3255717 |
| Client Quota | #3351539 | N/A | N/A | N/A | #3341008 #3355965 | #3334158 |
| OAuth -Access Management & Confluent Cloud | #1337204 | N/A | N/A | N/A | #3382773 #3385460 #3392690 | #3303697 |
| Public Clusters(Confluent Cloud) | #1422279 | N/A | N/A | N/A | #3382773 | #3445108 |
| Public Clusters(Access Management) | #1374692 #1374669 | N/A | N/A | N/A | #3453746 #3450370 | #3445108 |
| Public Clusters(Data Product Pipeline) | #1933853 | N/A | N/A | N/A | #3453743 | #3445108 |
| Access Management & Confluent Cloud & Pilot Application | #1374669 #1422279 | N/A | N/A | N/A | #3585867 #3585887 #3585855 | #3580830 |
| Access Management & Confluent Cloud & Pilot Application | #3185126 | N/A | N/A | N/A | #3620629 #3620632 #3621756 | #3591462 |
Traceability between Test cases and Test Runs¶
Traceability between Test cases and related Test Run IDs are captured in the Reporting test case runs that tagged with all the above Test plans.
Traceability for Unscripted Testing¶
| Change ID | Short Description | Product | User Story | Documentation Update |
|---|---|---|---|---|
| CHG0198880 | Compliance remediation for S3 buckets | Access Management Azure Pipeline Agent |
https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3041029 | Technical documents(Design Specification , Operation Guide , Solution Design) have been created which explains everything about KMS keys like how they are created.https://dev.azure.com/novonordiskit/KaaS%20(Kafka%20as%20a%20Service)/_git/aws-infra-generic?path=%2Fdocs |
| CHG0200435 | Updating of Terraform and Terraform providers for all products | All Products(Minor Impact) | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3226610 | NO Impact |
| CHG0182060 | Update KaaS Wiki to support Developer Portal | No Impact | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2938692 | No Impact |
| CHG0183356 | Update all users group mapping for kaas | Access Management | User Story 2652594 Add [novonordiskit]\Novo Nordisk Users as \Readers to KaaS Workspace | No Impact |
| CHG0178587 | Data product pipeline template - update | Team Provisioning Pipeline & Data Product Pipeline Template | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2879063 | Operation Guide updated Pull request 211342: CHG0178587 prerequisite: Terraform state storage centralization - Repos |
| CHG0211982 | Implement using larger instances on ECS for pipeline-agents and download terraform binaries from S3 for access-mgmt- | No Impact | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3356531 | No Impact |
| CHG0195612 | Flink Enablement in Playground | Access Management & Confluent Cloud & Logging | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3140935 | User Guide updated for Playground Environment PR#239856 |
| CHG0207186 | Retention of Production Pipeline Runs (Automation) | No Impact | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3271892 | PR#252340 - for creation of Retention Policy Wiki [Repo- Wiki] |
| CHG0206434 | Critical vulnerability detections: Continuous scanning and alerting for ECR image repository | No Impact | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2648666 | No Impact |
| CHG0179254 | Access-mgmt pipeline - Granular access support | Team Provisioning Pipeline | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2814974 | Operation Guide updated |
| CHG0202176 | Bugfix for Change CHG0200193, broken pipelines of Karen Team | Access Management Confluent Cloud Stream Governance Pipeline Azure Pipeline Agent |
N/A | Configuration and Operation Guides updated |
| CHG0188386 | Implementation of ADO Automation (Terraform) - KaaS Workspace | No Impact | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2846820 | No Impact |
| CHG0186725 | Policy Tests: Checkov/Prisma on existing IaaC | Azure Pipeline Agent | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2746688 | No Impact |
| CHG0201356 | Deprecate non-sensitive AWS secrets, and use already configured SSM parameters instead, enable auto rotation on KMS keys. | Data Product Pipeline Template | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/2613539 | PR#240818 - Wiki update to Platform Handbook/Platform Build /Configuration Guide/AWS [Repo- Wiki] |
| CHG0205531 | Prevent privilege escalation with Permissions Boundary on iam:CreateRole | Access Management Azure Pipeline Agent |
https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3185118 | - Configuration Guide updated - PR#244757 |
| CHG0200193 | Deprecate the use of IAM User API keys in Platform Pipelines | Access Management Confluent Cloud Stream Governance Pipeline Azure Pipeline Agent |
https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3045975 | Operation Guides and Configuration Guides updated - PR#239075 |
| CHG0216487 | Implement Checkov Terraform Scanning Across Pipelines | Confluent Cloud & Azure Pipeline Agent | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3353983 | Operation Guides updated |
| CHG0214068 | Speed up access-mgmt pipeline execution time by implementing warm pool agent pool in ECS and download terraform binaries from S3 | Access Management & Data Product Pipeline Template & Azure Pipeline Agent | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3356531 | PR#259730 : Platform Handbook , User Guides and Solution design (Azure Pipeline Agent) has been updated |
| CHG0211204 | Configure TLS for Kafka Clusters to support enabling SCI on KaaS platform | Confluent Cloud | https://dev.azure.com/novonordiskit/FIT-Integrations/_workitems/edit/3373764 | User Guide/Configuration Guide and Solution Design update - PR#256516 |
| CHG0221016 | Enhancement for "kaas-audit-log-forwarder" Mule Application | No Impact | User Story 3492310 Enhance kaas-audit-log-forwarder Mule App to push Confluent Catalog Data to Splunk | CIL and documents impacted at repo level- PR#269341 |
| CHG0224561 | Update a configuration parameter in access management | Access Management | User Story 3524263 Developermanage role correction | Access Management Solution Design updated - PR#273345 |
| CHG0217598 | Speed up data product pipelines execution time by implementing warm pool agent pool in ECS and download terraform binaries from S3 | Data Product Pipeline Template, Azure Pipeline Agent | User Story 3424015 Implement warm pool feature with robust scaling for self hosted agents in ECS | Wiki updates to Operation Guide , Service Account file and Solution Design of Azure Pipeline Agent-PR#265358 |
| CHG0221216 | AI Code Review and Hotfix for CHG0217598 and CHG0214068 | No Impact | User Story 3418267 Implement AI Code Review in ADO | AI PR review template and documentation added - PR#268214 |
| CHG0224575 | Code variable change on all repos | Access Management , Confluent Cloud , Stream Governance Pipeline , Azure Pipeline Agent | User Story 3442794 Replace name of critical tenant variable to ensure understanding | Configuration Guide , Operation Guide , User Guide updated - PR265450 |
| CHG0223635 | Implementation for Strictly Confidential Information on KaaS Platform | Access Management, Confluent Cloud | Feature 3281026 Enable handling SCI (Strictly Confidential Information) | This Change was created retrospectively for recording SCI. The documents/deliverables updated were TM, ITRA, Onboarding agreement, O&M, Test Plan. the information in detail is specified in the mentioned Change. |
| CHG0226489 | Setup dns forwarding for private networks on KaaS | Confluent Cloud | User Story 3496532 Setup DNS forwarders for Confluent Environments | No Impact |
Change Log¶
| Version no. | CHG no. | Approved Pull Request | Date | Change Description |
|---|---|---|---|---|
| 1.0 | CHG0100614 | PR114366 | See PR | New document |
| 2.0 | CHG0127934 | PR130403 | See PR | Updated the Trace matrix to include KaaS_viewer testing references |
| 3.0 | CHG0131184 | PR131425 | See PR | Trace matrix is updated with Test plan 2076884 reference |
| 4.0 | CHG0122187 | PR137599 | See PR | Testing evidences of Stream governance pipeline and Application pipeline template updates are captured |
| 5.0 | CHG0136055 | PR147693 | See PR | Testing evidences of ADO Resource refactoring updates are captured |
| 6.0 | CHG0126415 | PR159304 | See PR | Testing evidences of Azure Pipeline Agent refactoring updates are capture |
| 7.0 | CHG0139543 | PR166843 | See PR | Testing evidences of AD Group mapping change is updated |
| 8.0 | CHG0156215 | PR172442 | See PR | Testing evidences of Service connection to AWS change is updated |
| 9.0 | CHG0156154 & CHG0157722 | PR173517 | See PR | Testing evidences of Terraform & Confluent provider update and Pipeline Agent: ECR Image Policy is added |
| 10.0 | CHG0159095, CHG0158641 & CHG0160499 | PR179684 | See PR | Testing evidences of KaaS Workspace ADO project release, IAM Service followup verification and Audit log forwarder update is captured |
| 11.0 | CHG0159095 | PR180665 | See PR | appropriate Test plan and test case details are update for KaaS workspace verification |
| 12.0 | CHG0163128 | PR182565 | See PR | Test Plan details of TF Plan update change is updated |
| 13.0 | CHG0170084 | PR194250 | See PR | Test Plan details of KaaS pilot application migration change is updated |
| 14.0 | CHG0173645 & CHG0172529 | PR200215 | See PR | Test Plan details of Playground evn and playground user access verification and Pipeline Agent Customize image update change are captured |
| 15.0 | CHG0184608 | PR233231 | See PR | Test Plan details of Connectors Enablement in Confluent Cloud UI is updated |
| 16.0 | CHG0184608 | PR234715 | See PR | Old Test Plan Superseded due to Change in Release Section. Details of New Test Plan added |
| 17.0 | CHG0201019 | PR241625 | See PR | Test Plan details of Flink feature enablement in Confluent Cloud UI is added |
| 18.0 | CHG0201019 | PR242344 | See PR | OV Test Cases were Superseded. Added Details of New Test Plan and OV Test Cases |
| 19.0 | CHG0207193 & CHG0209137 & CHG0206213 | PR267644 | See PR | Test Plan and Test Cases details for Client Quota, OAuth and Public Clusters has been added A separate section created for Traceability for Changes with Unscripted Testing |
| 20.0 | CHG0221016 & CHG0224561 & CHG0217598 & CHG0221216 & CHG0219076 | PR275448 | See PR | Details updated for the mentioned changes in Unscripted Testing section |
| 21.0 | CHG0224575 & CHG0223635 & CHG0230525 | PR283675 | See PR | Details updated for the mentioned changes in both Scripted and Unscripted Testing section |
| 22.0 | CHG0226489 & CHG0231927 | PR294922 | See PR | Details added for the Flink Enablement and setting up dns forwarding for private networks on KaaS |