Add flow_timeout_in_minutes to VNETs. #661

SteffenBoThomsen · 2024-11-05T09:18:12Z

Problem

When either using a Firewall in a brown field deployment or deploying a firewall through SDAF in a green field deployment. It's important to set the VNET flow timeout in accordance with the Firewall idle timeout.

As the idle timeout can be increased only by request to Microsoft support, we can't infer the actual value to set and should therefore delegate to the users.

Furthermore, when having a firewall not controlled by SDAF, it should be possible to disable route propagation on route tables manually.

Solution

Add the option to specify *_flow_timeout_in_minutes for management and workload zone VNETs, defaults to null.
Add the option to specify network_enable_route_propagation for subnet route tables, defaults to true.

## Summary of Changes ### Authentication and Identity Management - **Web Application Authentication Configuration**: Repeated updates to refine and simplify the authentication configuration and identity management scripts. ### Repository and Package Management - **SLES Repositories**: Added repositories for SLES 15.3, 15.4, and 15.5. - **WAAgent Updates**: Updated WAAgent package, configuration checks, and systemd service reloads across multiple roles. ### Deployment Configuration - **NAT Gateway**: Added support for provisioning a NAT gateway, including configuration variables in Terraform files. - **AutoUpdate Configuration**: Updated AutoUpdate.Enabled settings and added Extensions.WaitForCloudInit across various roles. - **Oracle Simplification**: Simplified Oracle-related configurations, including ASM backup process and Data Guard tasks. - **SAP Deployment Playbooks**: Various updates to SAP deployment playbooks, including fixing conditions, resource flag settings, and systemd service paths. ### Infrastructure and Pipeline Enhancements - **Control Plane Pipeline**: Multiple fixes and improvements to error handling, logging, environment variables, and Azure AD authentication. - **Terraform and Ansible Versions**: Updated versions in deployment scripts to 1.7.5. - **Dotnet SDK**: Bumped dotnet SDK installation to the latest version. ### Miscellaneous - **Error Handling and Logging**: Improved error handling and logging across various deployment scripts and playbooks. - **Validation Fixes**: Fixed validation conditions for disk space, OS version checks, and cluster initialization commands. --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]>

* Fix typo in terraform-units/modules/sap_landscape/providers.tf * Remove duplicate of resource azurerm_network_security_rule/nsr_controlplane_storage In terraform-units/modules/sap_landscape/nsg.tf * Remove fourth argument from nat_gateway_name definition In terraform-units/modules/sap_landscape/variables_local.tf * Remove duplicate for database_kdump_disks In terraform-units/modules/sap_system/anydb_node/outputs.tf * Remove all duplicates from terraform-units/modules/sap_system/app_tier/vm-scs.tf * Remove duplicates in terraform-units/modules/sap_system/output_files/sap-parameters.tmpl --------- Co-authored-by: Csaba Daradics <[email protected]>

* script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook

…ng location constraints

…subscription and identity lists

…mat for subscription and identity lists" This reverts commit e86dff1.

…sequently running the ps1 script, where we already have the trust setup for the SSH key. Not doing it this way, leads to either needing to manually create an SSH session inside pwsh with POSH-SSH to ensure the known_hosts entry is updated or having to update the quality check script upstream, to allow the -Force flag for the SSH session. (Azure#603)

…zure#604) * Add fast_stop=no to pacemaker fileystem resources Pacemaker isn't respecting the stop timeout on filesystem resources due to the default setting fast_stop=yes. Without setting fencing will occur because if SAP (A)SCS / ERS isn't stopped in time processes will be terminated which are restarted by sapstartsrv and node will be fenced because fileystem can't be unmounted. https://www.suse.com/support/kb/doc/?id=000020860 https://access.redhat.com/solutions/4801371 * Distribute systemd services between SCS / ERS nodes and stop services Both (A)SCS and ERS systemd services should be present on SCS and ERS nodes otherwise pacemaker only handles SCS on the SCS node and ERS on the ERS node with the systemd integration. * Add resource clear for move contrainsts on (A)SCS resource group * Bugfix folders on local disks to be managed after mounting local disk * sid_private_key isn't required * Add sdu_secret prefix/suffix to manage custom Key Vault secret naming

* update: added fixed encountered during RHEL94 testing * chore: Update Red Hat and SLES package versions for Red Hat 9.4 * update: add network rules to deployer diagnostic storage account * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Add ${SID}_virtual_machines.json to git if it exists * chore: Update sap_system module to use database_server_vm_resource_ids for database server VMs * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to use comma-separated database server VM resource IDs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap_system module to include application server VM resource IDs * chore: Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor use_spn variable to default to false for all SAP deployment modules * Refactor systemd process limit configuration for pacemaker cluster * Refactor systemd process limit configuration for pacemaker cluster

…r role (Azure#614)

* Add AVG support for Scale out scenarios (Azure#577) * Add data and log volumes * Refactor AVG logic * Fix proximity_placement_group_id calculation in avg.tf * Refactor for_each condition in avg.tf * Refactor for_each condition in avg.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor zone calculation logic in variables_local.tf * Refactor proximity_placement_group_id calculation in avg.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in vm-app.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in infrastructure.tf * Refactor package update condition in 1.4.3-update-packages-RedHat.yaml --------- Co-authored-by: Kimmo Forss <[email protected]> * Update subnet_cidr_storage in sap-parameters.tmpl * Update hosts jinja for client subnet * Update SAP-specific configuration playbook for HANA database scale-out scenario * Version update * Simplify Web App Identity management * Update Azure package versions in SDAFWebApp.csproj * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Add SLES 15.3, 15.4, and 15.5 repositories * Update Web Application authentication configuration script and simplify Web App Identity management * Refactor Web App Identity management and update authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script and simplify Web App Identity management * Commented out SSH trust relationship checks in 1.17.2-provision.yml * Revert "Commented out SSH trust relationship checks in 1.17.2-provision.yml" This reverts commit 09cd30d. * ACSS updates * Oracle simplification * Add AutoUpdate.Enabled configuration in 1.1-swap role and enable package cache update in 1.4-packages role * Update deployment type configuration in OS and SAP specific playbooks * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update WAAgent package and restart service in 1.1-swap role * Updated key_vault_sap_landscape.tf * Revert "Updated key_vault_sap_landscape.tf" * Update WAAgent package and restart service in 1.1-swap role * Add SAP CAL Integration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Revert "Add SAP CAL Integration" This reverts commit adae666. * Update WAAgent package and restart service in 1.4-packages role * Update waagent configuration check in 1.4-packages role * Update waagent configuration check and systemd service reload in 1.4-packages role * Update AutoUpdate.Enabled configuration and add Extensions.WaitForCloudInit configuration in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update database_high_availability condition in playbook_04_00_01_db_ha.yaml * Add the ability to block app registration * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update web_instance_number and add web_sid variable in sap_system/transform.tf * Fix validation error message for web dispatcher sid in variables_global.tf * Remove chkconfig package from os-packages.yaml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update OS version check for RHEL 8.2 and SLES 15 in 5.6.1-set_runtime_facts.yml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update Oracle ASM backup process and fix file permissions * Fix file path in 1.4.0-packages-RedHat-prep.yaml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update file path and preserve file permissions in 1.4.0-packages-RedHat-prep.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Update wait time for StartService in 5.6 SCS/ERS Validation * Update Terraform version to 1.8.0 in deployment scripts and tfvar_variables.tf files * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Update virtual machine extension reference in vm.tf * Update virtual machine extension version to 1.0 in vm.tf * Fix missing else statement in deploy control plane pipeline * Update network interface and virtual machine counts in vm-observer.tf * Update database high availability configuration * Update use_spn property to false in LandscapeModel and SystemModel * Update Terraform and Ansible versions to 1.7.5 in deployment scripts and variables * Update Display value in SystemDetails.json * Fix validation condition in variables_global.tf * Add ORACLE Post Processing: Reboot after Enabling HugePages task * Fix typo in Oracle Data Guard - Observer: Change UID for Oracle user task * install passlib * Add patch_mode support * Update deployment playbook to set single_server fact based on host count * Update patch_mode configuration in Terraform files * Update file permissions in SAP deployment playbook * Update deployment playbooks to set single_server fact consistently * Fix waagent configuration in swap role * Fix indentation in swap role tasks/main.yaml * Fix cluster group move command in 5.6 SCS/ERS Validation playbook * Fix condition in 1.17-generic-pacemaker playbook to exclude node_tier 'hana' * Fix commented out corosync configuration in 1.17-generic-pacemaker playbook * Create the SID subfolder * Update verbosity level in 5.6.7-config-systemd-sap-start.yml * Add passlib * Simplify Python logic * Update app_bom_id variable in 5.3-app-install/tasks/main.yaml * Update passlib installation in Ansible playbooks * Update reboot timeout and post-reboot delay in 5.6.4.2-sap-resources-Suse.yml * Update swap role and package tasks * Fix condition in 1.17-generic-pacemaker playbook to exclude node_tier 'hana' * Fix failed_when condition in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * enable corosync and pacemaker on Suse * change from command to shell * Update verbosity level for debug message in 5.6.4.0-cluster-Suse.yml * Refactor command to shell in 5.6-scsers-pacemaker tasks * Refactor command to shell in 5.6-scsers-pacemaker tasks * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 5.6-scsers-pacemaker tasks * Refactor path in ora-dg-observer-setup.yaml to include sap_sid variable * Refactor cluster initialization commands in 5.6-scsers-pacemaker tasks and add SAP component installation check * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook and 5.6-scsers-pacemaker tasks * add missing quotes * Fix disk space validation in playbook_00_validate_parameters.yaml * Refactor SAP resource flag setting in Ansible playbooks * Refactor SAP component installation check in 5.6-scsers-pacemaker tasks * Refactor SAP resources installed message in 5.6-scsers-pacemaker tasks * Refactor SCS/ERS validation tasks in 5.6-scsers-pacemaker playbook * Refactor SAP resource flag setting in Ansible playbooks * Refactor ORACLE: Find MOPatch tasks in 4.1.0-ora-install playbook * support for pools with auto qos * support for pools with auto qos * support for pools with auto qos * provide a way to override the oracle user * Update Web Application Configuration documentation * Fix default value for SAP_installed in 5.6-scsers-pacemaker tasks * Fix default value for SAP_installed in 5.6-scsers-pacemaker tasks * Fix shell command in 5.6-scsers-pacemaker pre_checks.yml * Passwordless Web App * Passwordless * Update variable group creation in New-SDAFDevopsProject.ps1 script * Fix client_id reference in app_service.tf * Update packages * Update Web Application Configuration to use resource group scope for role assignments * Update Web Application Configuration documentation * Fix target_nodes value in 2.6.1-anf-mounts.yaml * Web App updates * Update enable_db_lb_deployment logic in variables_local.tf * Bump up the dotnet version * Remove PAT * Remove PAT * Fix TF_VAR_agent_pat assignment in deploy control plane pipeline * Fix PAT assignment in deploy control plane pipeline * Update TF_VAR_agent_pool assignment in deploy control plane pipeline * Add MSI registration * Fix typo * Update versionLabel to v3.11.0.2 in New-SDAFDevopsProject.ps1 * Fix typo in New-SDAFDevopsProject.ps1 + add PAT back for Control Plane * Update ANF mount paths in 2.6.1-anf-mounts.yaml * Fix PostBuildCleanup task in deploy control plane pipeline * Update PostBuildCleanup task to version 4 in deploy control plane pipeline * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update DEPLOYER folder and file validations in deploy control plane pipeline * Update deploy control plane pipeline with environment and location information * Update deploy control plane pipeline with Deployer TFvars variable * Update deploy control plane pipeline with Library TFvars variable * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update installer.sh to display parameter file and current directory * Update deploy control plane pipeline with Library and Deployer TFvars variables * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update PostBuildCleanup task to version 3 in deploy control plane pipeline * Update dotnet-sdk installation in configure_deployer.sh.tmpl * Update deploy control plane pipeline with TF_VAR_agent_pat variable * Update deploy control plane pipeline with Azure CLI version display * Update deploy control plane pipeline with Workload TFvars variable * Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * Update deploy/ansible/roles-db/4.1.3-ora-dg/tasks/ora-dg-setup-secondary.yaml to remove the 'recurse' option in the ansible.builtin.file task * Update deploy/ansible/roles-db/4.1.3-ora-dg/tasks/ora-dg-setup-secondary.yaml to fix failed_when condition in rman restore tasks * chore: Update app_service.tf to add WHICH_ENV variable * Update app_service.tf to allow specific app registrations * chore: Update NuGet.Packaging dependency to version 6.9.1 * chore: Update app_service.tf to remove unused app setting and add WHICH_ENV variable * chore: Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * chore: Update AFS Mount task to exclude 'app' node tier * chore: Update hosts.j2 template to exclude virtual hosts for non-high availability scenarios * chore: Update New-SDAFDevopsProject.ps1 to improve App Registration creation process * Change the ID to add * chore: Update New-SDAFDevopsProject.ps1 to improve App Registration creation process * Add SAP-CAL Integration * Linting * chore: Update deploy control plane pipeline with necessary environment variables * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use System.AccessToken for AZURE_DEVOPS_EXT_PAT * chore: Update deploy control plane pipeline to remove unused agent pool check * chore: Remove unused agent pool check in deploy control plane pipeline * chore: Update deploy control plane pipeline to use $(PAT) for AZURE_DEVOPS_EXT_PAT * changes to ERS group * chore: Update deploy control plane pipeline to improve error handling and logging * chore: Update deploy control plane pipeline to enable Azure AD authentication * chore: Update deploy control plane pipeline to extract deployer_random_id from environment file * chore: Improve error handling and logging in deploy control plane pipeline * chore: Update deploy control plane pipeline to extract deployer_random_id from environment file * chore: Update deploy control plane pipeline to create variable group variables for key vault, terraform remote storage subscription, and deployer random ID seed * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to use $(PAT) instead of $(System.AccessToken) for AZURE_DEVOPS_EXT_PAT * chore: Update deploy control plane pipeline to improve error handling and logging * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update bootstrap flag to false in sap_library module * chore: Update storage account network rules for tfstate and sapbits * chore: Update dotnet-sdk installation to version 8.0 * chore: Update dotnet-sdk installation to latest version * chore: Update HttpClient usage in RestHelper.cs and Azure SDK versions in SDAFWebApp.csproj * chore: Update random_id_b64 format in output.tf files * chore: Update RestHelper.cs to accept a type parameter in the constructor * chore: Ignore changes to app_settings in azurerm_windows_web_app resource * chore: Update random_id_b64 format in output.tf files * chore: Update RestHelper.cs to use HttpClient instead of HttpClientGH * chore: Add Build Service user to Build Administrators group * Add the ability to authenticate using PAT * chore: Update RestHelper.cs to use HttpClient instead of HttpClientGH * Update on devops login * chore: Update New-SDAFDevopsProject.ps1 to use tsv output for project creation * chore: Refactor RestHelper.cs to use HttpClient and support PAT authentication * Change module name * update: SAP ASCS/SCS/ERS start resources configuration for SUSE - ENSA1 and ENSA2 when using simple mount. This commit updates the configuration of SAP ASCS/SCS/ERS start resources for SUSE - ENSA1 and ENSA2. * chore: Update SAP Directories creation in ansible playbook This commit updates the ansible playbook to create SAP Directories. It modifies the tasks to create the directories "/usr/sap/trans" and "/sapmnt/{{ sap_sid | upper }}". These changes improve the handling of SAP Transport Filesystems in the deployment process. * feat: Add additional destination port ranges for NSG rules This commit updates the NSG rules in the `sap_landscape` module to include additional destination port ranges. The destination port ranges for the `nsr_controlplane_app`, `nsr_controlplane_web`, `nsr_controlplane_storage`, `nsr_controlplane_db`, and `nsr_controlplane_admin` rules have been expanded to include ports 2049 and 111. * Update error message * Update SAP ASCS/SCS/ERS start resources configuration for SUSE - ENSA1 and ENSA2 when using simple mount. * Add the MSI to the project * Added debug statement to playbook_sapcal_integration.yaml * Added debug statement to playbook_sapcal_integration.yaml * Revert "Added debug statement to playbook_sapcal_integration.yaml" This reverts commit 839170e. * Revert "Added debug statement to playbook_sapcal_integration.yaml" This reverts commit 5170d0b. * Skip all BOM related tasks if enable_sap_cal is true * Updated the variable name for consistency * Ensured tasks run with appropriate privileges * Store SAP-CAL API response/file in the repository * Lint code and set default values * Use a secure tempfile --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: Devansh Jain <[email protected]>

* Add the ability to split out the privatelink resources * feat: Add privatelinkdnsmanagement provider configuration * refactor: Update storage_accounts.tf to use var.dns_settings.dns_zone_names.table_dns_zone_name * refactor: Update DNS zone names in dns.tf and storage_accounts.tf * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names.table_dns_zone_name * refactor: Update DNS zone names in infrastructure.tf, key_vault.tf, and keyvault_endpoint.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in keyvault_endpoint.tf and storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update count condition in dns.tf to use local.use_local_privatelink_dns instead of negation of it * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf and sap_deployer/tfvar_variables.tf to use var.dns_settings.dns_zone_names * Add the ability to split out DNS records for privatelink resources * refactor: Update DNS zone names to use var.dns_settings.dns_zone_names * refactor: Add privatelink DNS resource group and subscription properties to LandscapeModel * refactor: Update DNS zone names in infrastructure.tf, key_vault.tf, and keyvault_endpoint.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in LandscapeDetails.json, storage_accounts.tf, infrastructure.tf, and transform.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in transform.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * Add register_virtual_network_to_dns attribute * Add the ability to control the patch mode * add vm_agent_platform_updates_enabled * refactor: Remove patch_mode from vm-scs.tf * refactor: Remove patch_mode from vm-anchor.tf * Add auto update of the extensions * refactor: Tweak the Windows patch mode * Windows update settings * Debug show SystemD version * refactor: Update SystemD version debug message in 1.17 Generic Pacemaker role * refactor: Update VM patch information in SystemModel and LandscapeModel * refactor: Update Process limit configuration in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for pacemaker version in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for systemd version in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for systemd version in 1.17 Generic Pacemaker role * Remove the white space * fix: Associate the iSCSI subnet with the route table * refactor: Add python3-pip package for different node tiers in HA setup * refactor: remove the lower pipe from distro name * refactor: Split out OracleLinux tasks * refactor: Update iSCSI subnet association with route table * chore: Update NuGet.Packaging dependency to version 6.11.0 * TEswt if we can handle no read access scenarios to key vault * revert casing * refactor: Split out OracleLinux tasks * chore: Add condition to include custom repositories in 1.3 Repository tasks * refactor: Update 1.3 Repository tasks to include custom repositories for SUSE and RedHat * refactor: Remove unnecessary OracleLinux tasks and custom repositories * refactor: Update VM deployment configuration * Remove the token check * refactor: Add TF_VAR_agent_pat to control plane deployment pipeline * refactor: Fix private DNS zone ID in keyvault_endpoint.tf * Web App and version updates * Restore patch_mode * Web App updates * chore: Add System.Data.SqlClient package reference * refactor: Update 1.3 Repository tasks to include custom repositories for SUSE and RedHat * refactor: Update tfvar_variables.tf with new variables for tfstate storage account and deployer's tfstate file * Remove some of the python packages * Remove unnecessary python packages * refactor: Remove trailing spaces in LandscapeDetails.json and SystemDetails.json * refactor: Remove trailing spaces in LandscapeDetails.json and SystemDetails.json * Fix reboot on RHEL * refactor: Fix typo in DBLoad task names * refactor: Update cluster resource monitor intervals to 20 seconds * LINT fixes --------- Co-authored-by: Kimmo Forss <[email protected]>

* Update fence_kdump_nodes configuration in /etc/kdump.conf * Update DNS check in OS configuration playbook * Update database cluster IP address variable * Update disk configuration variables * Fix domain\service account names in mssql-alwayson-prerequisites.yaml * Update domain account names in mssql-alwayson-config.yaml * Update SQL Server service account format * Update domain service account format in SQL login * Add mount for local kdump file path * Update SQL Server service account names*** * Update supported_tiers to node_tier in main.yaml * Fix formatting in mssql-alwayson-prerequisites.yaml * Update SQL service account names in mssql-alwayson-prerequisites.yaml * Fix proximity placement group logic in vm-scs.tf * Update kdump mount path, update kdump path and restart kdump service * Refactor proximity_placement_group_id calculation in vm-app.tf * Fix SQL Server Always On configuration * Update SQL Server SPNs and service account * Update SQL Server AlwaysOn configuration * Add error handling for deployment account access * Update AMS subnet configuration * Update default_action in key_vault.tf * Update ams.tf to use arm_id for subnet name and network name * Refactor Agent_IP variable to conditionally include it in storage and key vault firewalls * Refactor admin password authentication in vm-deployer.tf * Add Agent IP configuration option * Add USE_MSI check before ARM_CLIENT_ID check * Refactor password handling in sap_deployer module * Update login process in deployment pipeline * Add random password generation for deployer and update dns_label variable description * Update Azure Pipeline script to handle MSI usage*** * Remove unnecessary echo statement in login section * Update transform.tf to include additional conditions for app_use_avset * Fix zone ignore_changes in app_tier VMs * Add azure-devops extension installation if not already installed * Add files.pythonhosted.org to list of URLs * Update tf_version to 1.7.4 in SDAF-General variable group * Add log file functionality to Test-SDAFReadiness.ps1 script * Refactor sbd_device variable assignment in 1.17.1.2-sbd.yaml * Refactor sbd_device variable assignment * Fix variable naming inconsistency in ANF HANA data and log volumes * Update Agent_IP handling in sap_landscape module * Add conditional logic for Agent IP in module.tf and add add_Agent_IP variable in tfvar_variables.tf * Fix virtual host assignment in hosts file task * Variable name adjustments * Remove create_vaults_and_storage_dns_a_records variable * Add systemd reload and stop SBD service in cluster setup * Refactor 2.4 Hosts: Remove unnecessary condition and update debug message * Update crm resource command to use status instead of show * Update proximity_placement_group_id in vm-app.tf * Update proximity_placement_group_id in vm-app.tf * Add proximity placement group ID to virtual machine scale set resource * AvSet logic * Refactor STONITH Azure fence agent creation in 1.17 Generic Pacemaker role * Add file share and key vault creation * Remove unnecessary blank lines in Test-SDAFReadiness.ps1 script * Refactor cluster configuration and stop SBD service * convert to list * Refactor virtual host retrieval in 2.4 Hosts playbook * Add a script for just checking URLs * Refactor virtual host retrieval logic in main.yaml * - Add cluster restart and wait tasks - Update virtual host fact in hosts file * refactor the hosts file * This commit refactors the logic for retrieving the virtual host in the 2.4-hosts-file task. * Refactor 2.4 Hosts: Set virtual_host fact from the fetched PAS server list * Refactor 2.4 Hosts file tasks to use pas_server_temp for virtual host * Fix pas_virtual_hostname assignment in 2.4-hosts-file * Fix virtual host duplication issue in 2.4 Hosts file * Remove the wait for the first run * Update Stonith SBD configuration in cluster * When Terraform plan failed, stop the script from executing Terraform apply (#560) * Update return_value with new return code to stop when plan failed When the Terraform Plan has errors, the script will continue to run. The return_value was not being updated with the return code of the plan, so the check is not working. * Sanitaze shebang for bash scripts * Update installer.sh * Update installer.sh * Remove unnecessary variable assignment in Test-SDAFURLs.ps1 script * Fix syntax error in installer.sh * Update github-actions-ansible-lint.yml update to use setup-python@v5 * Fix firewalld module fqcn in pre_checks.yml * Add optional extended log collection, fix kdump_enabled undefined variable error (#562) * Add optional, extended log collection functionality * Use default filter with kdump_enabled variable Prevent 'kdump_enabled' is undefined errors --------- Co-authored-by: Csaba Daradics <[email protected]> * Fix conditional check for sapinst_instdir_exists * Add additional destination port ranges to NSG rule * Oracle-non-asm (#566) * lsnrctl status update * oracle-asm dataguard setup * asm file permission updates * asm variable update * asm file conditional change * oracle-asm file creation updates * oracle-asm listener handling * asm listener handling * asm lsnrctl reload test * asm listener adjustments * oracle asm initSID.ora file updates * asm * asm testing * rman command update * update the replace function * oracle-asm * fail message update * asm * remove fail flag * asm restore update * oracle-asm db updates * oracleasm changes * oracle-asm update1 * oracle-asm2 * oracle-asm restore adjustments * syntax error whilst creating asm files * asm retrofit with spfile and restore scripts * asm update * asm flashback folder creation * asm * enable fsfo on secondary * asm * asm clusterware config * asm oracle cluster ware restart updates * asm updates for registering stdby in srvctl * asm * asm * ASM changes * Oracle DG automated trigger testing * listener change * dbload changes * ASM DG changes * asm * asm * asm * asm * oracle grid sbp location update * grid file permissions * gsbp file permissions * grid sbp change * gsbp patching * ' updated * oracle * sidadm creation on secondary db node * sidadm for oracle secondary * user change * sidadm for oracle * asm dg * oracle asm dg * oracle asm opatch * updated SBPFUSER variable for grid patching * added oracle-asm node * updated the post-install file * grid sbp patch * grid sbp * grid sbp copy * grid * grid update * tnsnames update for sap app * sbp grid error handling * spell corrrection * error handling * grid patching * sbp 2308 fixes * grid * grid sbp test * grid sbp new version testing * grid * grid * ok * ok * sbp * grid sbp * GRID SBP * gsbp oradism paermissions * SBp * grid sbp * debug * debug * grid pre-install patch * debug * debug1 * debug * old mopatch and opatch to test * old sbp * opatch 11 * sbp updates * sbp * grid sbp folder creation * change the order of SBP GRID first then RDBMS * permission update * oracle grid * grid patching 2311 testing * comment out GRID Patching * sga and pga adjustments * oracle changes * repo update added 8.9 * added packages for OEL8.9 * repo * spfile changes * oracle pga caluculated based on the oracle documentation. * lsnrctl on the secondary is ot running yet remove the lsnrctl stop. * lsnrctl for asm updated * syntax correction * syntax correction for paramter * oracle asm lsnrctl parameter update * syntax update * updated the register parameter * re-arranged the order for evaluation of node_tier * updated the syntax for node_tier * rearranged the order of the conditions * non-asm ha setup * creating sidadm user on secondary * rman restore for non-asm updates * initsid.ora parameter updates for oracle HA * updating the spfile for ora non-asm secondary * update the flag file * rman duplicate change * update the spfile * spfile update for oracle * oracle dataguard replication * oracle non-asm dg changes * reduced the temp disk space to 50 from 100 on the deployer * reduced the tmp disk space setting * updated the /mnt value check to bypass the free space check * updated code to handle oracle bug whilst creating the redologs on secondary * oracle non-asm dg setup update * updated the local_listener value in primary to re-register as secondary after failback * converted caps to small for local_listener values * added rebstart of secondary to activate HA service for SAP * adding oraflash filesystem creation * oracle non-asm changes * updates the oracle sga and pga caluculation * oracle listener on primary update * spfile scope update * oracle local_listener changes * commenting the local_listener as it is now implemented after dbload * oracle listner updates * debug the file update * debug listener change * file update for local_listener value * commeting out the debug lines * changing the sequeunce of reboot * added reboot block to allow database to set correct huge pages * error handling for lsnrctl restart on primary * correcting the typos * Remove whitespace * Linting * More linting * Fixing typo * Support zonal shared disks (#567) * Make SCS Shared disk Premium_ZRS * Add support for Premium_ZRS zonal disks * Add Cluster disk info to the Web App * Don't specify a zone for Premium_ZRS disks * Ensure that the --username parameter is correct * Fix error message in variables_global.tf * Remove unused variables in variables_global.tf * Refactor error message in variables_global.tf * Added database_cluster_disk_type * Update scs_cluster_disk_lun and database_cluster_disk_lun properties in SystemModel.cs * Remove trailing spaces in parameter names in SystemDetails.json * Fix ARM_CLIENT_ID variable interpolation in configure_deployer.sh.tmpl * Update proximity_placement_group_id in vm-app.tf * Fix zone assignment for scs_cluster_disk_type in vm-scs.tf * Update disk zone configuration * Update proximity_placement_group_id in vm-app.tf * Update proximity placement group ID in app tier VM configuration * Merge branch 'experimental' --------- Co-authored-by: Kimmo Forss <[email protected]> * Hotfix/scaleout anf multi-node standby (optional ) (#568) * adding variable for deploying HANA scale out - ANF without a standby node. * Add disclaimer * Bring in Scale out improvements (#569) * Make SCS Shared disk Premium_ZRS * Add support for Premium_ZRS zonal disks * Add Cluster disk info to the Web App * Don't specify a zone for Premium_ZRS disks * Ensure that the --username parameter is correct * Fix error message in variables_global.tf * Remove unused variables in variables_global.tf * Refactor error message in variables_global.tf * Added database_cluster_disk_type * Update scs_cluster_disk_lun and database_cluster_disk_lun properties in SystemModel.cs * Remove trailing spaces in parameter names in SystemDetails.json * Fix ARM_CLIENT_ID variable interpolation in configure_deployer.sh.tmpl * Update proximity_placement_group_id in vm-app.tf * Fix zone assignment for scs_cluster_disk_type in vm-scs.tf * Update disk zone configuration * Update proximity_placement_group_id in vm-app.tf * Update proximity placement group ID in app tier VM configuration * Merge branch 'experimental' * Add storage subnet for SAP VNET * Add storage_subnet_id and storage_nsg_id outputs to sap_landscape module * Add SAP storage subnet NSG and associate it with the storage subnet. Add SSH network security rule for connectivity to SAP application subnet from Control Plane. * Update NSG rule protocol to allow all traffic * Refactor proximityPlacementGroup assignment in avg.tf * Update NSG protocol to allow all traffic * Refactor subnet_storage configuration in transform.tf * Add storage subnet to Web App * Get the latest * Add scaleout variables * Update proximityPlacementGroup in avg.tf * Add storage subnet variables for scale-out configuration * Update storage subnet condition for ANF support * Refactor storage subnet count logic * Fix conditional expression in subnets.tf * Refactor network_interface_ids in vm-hdb.tf * Custom Mount fix * Add "Custom" sizing to UX * Create the root folder if it does not exist * Add ANF & Storage subnet prefixes to sap-parameters.yaml * Update storage subnet netmask variable name * Fix storage subnet ID in variables_local.tf * Fix ANF_subnet_prefix value in outputs.tf * Refactor sap-parameters.tmpl file * Update subnet prefixes for storage and ANF * Update principal_id in azurerm_role_assignment * Fix src path in custom mount task * Update IP address retrieval in Ansible playbooks * Update variable name in main.yaml * Update debug message to use correct variable name * Update debug message to print ipadd variable * Update networking tasks and hosts file template * Add client subnet * Add subnet_prefix_client to sap-parameters_yml resource * Update IMDS URL in networking tasks * Refactor host entries in hosts.j2 template * Fix formatting and typo in sap-parameters.tmpl * Refactor host entries generation in hosts.j2 template * Add routes and restart VM for HANA scaleout * Add app subnet * Add use_msi_for_clusters variable to sap-parameters.tmpl * Fix conditional value assignment in outputs.tf * Update network configuration details in main.yaml * Update sap-parameters.tmpl file * Update enable_storage_subnet condition in variables_local.tf * Debug * Web App updates * Refactor virtual host name assignment in hosts.j2 template * Add description property * Add the download ability * Add the MSI to the extension object * Fix virtual host names duplication issue * Fix virtual host name iteration in hosts.j2 template * Add support for secondary IP addresses in azure_interfaces.j2 template * Add dependencies for storage network interface creation * Refactor networking configuration and route creation. * Add DB subnet to sap-parameters.yaml * Update network interface conditions in main.yaml * Update VM-Images.json with new SKUs * Update internal network configuration in HDB installation playbook * Update database host roles in main.yaml * Update internal network configuration in HDB install playbook * Fix missing quotation mark in hdblcm command * Update internal network configuration in HDB install playbook * Refactor HDB installation command in main.yaml * Add additional destination port ranges to NSG rule * Task naming and Linting * Add database_HANA_no_standby_role variable --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]> * Refactor networking tasks for HANA scaleout * Update network interface conditions in main.yaml * Add ability to deploy the Monitoring extension (#570) * Add the Monitoring Extension to the Application Tier * Replace Windows virtual machine with Linux virtual machine in monitoring extension * Add monitoring extension to database tier * Update destination address prefixes in NSG rules * Fix destination_address_prefixes in nsg.tf * Update monitoring extension count based on database OS type * Add Monitoring extension for iSCSI servers * Add deploy_monitoring_extension parameter * Add storage subnet data source * Add Download link in Landscape/Edit view and update default value for deploy_monitoring_extension * Update monitoring extension names for utility VMs * Refactor monitoring extension deployment conditions in vm.tf --------- Co-authored-by: Kimmo Forss <[email protected]> * Update subnet prefixes to use CIDR notation * Control Private Endpoint DNS registration (#571) * Add the ability to control if Private Endpoints are registered with DNS * Merge branch 'experimental' * Add register_endpoints_with_dns option to common_infrastructure and hdb_node modules * Add register_endpoints_with_dns property to LandscapeModel and SystemModel --------- Co-authored-by: Kimmo Forss <[email protected]> * Update SAP HANA network details extraction and display * Update SAP OS configuration playbook and hosts file configuration tasks * Update count condition for azurerm_private_dns_zone * Fix subnet_client_cidr calculation in SAP OS configuration playbook * Update DB virtual hostname resolution in 2.4 Hosts file * Fix calculation of db_so_virtualhost_ip in 2.4-hosts-file * Install defender extension (#572) * Add the Defender Agent * Update monitoring_defender_app_lnx and monitoring_defender_app_win resource names * Update source_address_prefixes in NSG rules * Update Azure Monitor Linux Agent name * Fix Azure Monitor agent name in VM extensions * Update NSG rules to use the first address space in vnet_sap * Refactor NSG rules to use address_space directly * Fix subnet_client_cidr calculation in SAP OS configuration playbook * Add conditional block for non-Windows systems in SAP OS configuration playbook * Remove unnecessary condition for non-Windows systems in SAP OS configuration playbook * Update DB virtual hostname resolution in 2.4 Hosts file --------- Co-authored-by: Kimmo Forss <[email protected]> * add calculation of virtual host for scaleout * Fix IP address resolution in main.yaml * Update github-actions-ansible-lint.yml * Change to use import_role instead of include_role (#574) * Add tags to tasks * Refactor OS configuration playbook roles to use import_role instead of include_role * Refactor import_role to include_role in OS configuration playbook * Fix import_role in OS configuration playbook * Update swap role to include reboot and wait for connection tasks * Update timeout for wait_for_connection task * add tags and change to use import_role * Add deploy_defender_extension variable and update resource configurations * Add update_only flag to package update task * Add platform condition for oracle-asm tier * Update SAP system configuration * Update auto_upgrade_minor_version to true * Add tags to Ansible tasks in playbook_01_os_base_config.yaml and playbook_03_bom_processing.yaml * Add condition to skip reboot on Oracle Linux 8 * Add task to clear host errors and improve system reachability in 1.1-swap role * Refactor Oracle post-processing tasks*** * Update swap role tasks to include reboot and wait for connection * Add tags * Add post-reboot delay in swap role * Fix failed_when condition in swap role * Add become: true to reboot task * Add become flag for reboot task * Add tasks to clear host errors and wait for system to become reachable * Add kmod-oracleasm package to os-packages.yaml * Add reboot task and set failed_when to false in oracle-postprocessing.yaml * Add Microsoft AD collection to ansible-galaxy installations * Fix VM Agent Status check in 2.10.1 sap-notes * Remove oracleasm-support and kmod-oracleasm packages from oraclelinux8.9 --------- Co-authored-by: Kimmo Forss <[email protected]> * Add No log for access token and SAP Media Share as part of Install experience (#575) * No Log for access token * Changes for SAP Media File share as Install experience * Update enable_db_lb_deployment condition in variables_local.tf * Remove redundant code for monitoring defender extensions * Refactor monitoring defender extensions for app and scs tiers * Refactor enable_db_lb_deployment logic in variables_local.tf * Update enable_db_lb_deployment logic in variables_local.tf * Remove redundant code for iscsi monitoring defender * Update enable_db_lb_deployment logic in variables_local.tf * Update failed_when condition in oracle-postprocessing.yaml * Refactor enable_db_lb_deployment logic in variables_local.tf * Add AVG support for Scale out scenarios (#577) * Add data and log volumes * Refactor AVG logic * Fix proximity_placement_group_id calculation in avg.tf * Refactor for_each condition in avg.tf * Refactor for_each condition in avg.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor zone calculation logic in variables_local.tf * Refactor proximity_placement_group_id calculation in avg.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in vm-app.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in infrastructure.tf * Refactor package update condition in 1.4.3-update-packages-RedHat.yaml --------- Co-authored-by: Kimmo Forss <[email protected]> * Update subnet_cidr_storage in sap-parameters.tmpl * Update hosts jinja for client subnet * Update SAP-specific configuration playbook for HANA database scale-out scenario * Version update * Simplify Web App Identity management * Update Azure package versions in SDAFWebApp.csproj * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Add SLES 15.3, 15.4, and 15.5 repositories * Update Web Application authentication configuration script and simplify Web App Identity management * Refactor Web App Identity management and update authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script and simplify Web App Identity management * Commented out SSH trust relationship checks in 1.17.2-provision.yml * Revert "Commented out SSH trust relationship checks in 1.17.2-provision.yml" This reverts commit 09cd30de6003a891b5c8c31b4c96b495b676aa9b. * ACSS updates * Oracle simplification * Add AutoUpdate.Enabled configuration in 1.1-swap role and enable package cache update in 1.4-packages role * Update deployment type configuration in OS and SAP specific playbooks * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update WAAgent package and restart service in 1.1-swap role * Update WAAgent package and restart service in 1.1-swap role * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update WAAgent package and restart service in 1.4-packages role * Update waagent configuration check in 1.4-packages role * Update waagent configuration check and systemd service reload in 1.4-packages role * Update AutoUpdate.Enabled configuration and add Extensions.WaitForCloudInit configuration in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update database_high_availability condition in playbook_04_00_01_db_ha.yaml * Add the ability to block app registration * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update web_instance_number and add web_sid variable in sap_system/transform.tf * Fix validation error message for web dispatcher sid in variables_global.tf * Remove chkconfig package from os-packages.yaml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update OS version check for RHEL 8.2 and SLES 15 in 5.6.1-set_runtime_facts.yml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update Oracle ASM backup process and fix file permissions * Fix file path in 1.4.0-packages-RedHat-prep.yaml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update file path and preserve file permissions in 1.4.0-packages-RedHat-prep.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Update wait time for StartService in 5.6 SCS/ERS Validation * Update Terraform version to 1.8.0 in deployment scripts and tfvar_variables.tf files * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Update virtual machine extension reference in vm.tf * Update virtual machine extension version to 1.0 in vm.tf * Add Observer VM for HANA * Local Feature/scaleout hsr to upstream scale-out HSR branch (#581) * overhaul of pacemaker approach for Scale out Shared nothing HSR cluster. * add code to 2.6 AFS mount to support /hana/shared for scale out HSR code * port changes to simple mount AFS task * Update 2.6.0-afs-mounts.yaml * Update 2.6.0-afs-mounts.yaml * Update 2.6.0-afs-mounts.yaml * anf mount bugfix * cleanup of stale code - anf /hana/shared mount * Update playbook_04_00_00_db_install.yaml * Update main.yaml * Update main.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.2-provision.yml * Update 1.18.2-provision.yml * Update 1.18.2-provision.yml * experimental attempt to split pacemaker from hana setup * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * rollback * Update playbook_04_00_00_db_install.yaml * Update 1.18.2.0-cluster-Suse.yml * Update 1.18.2.0-cluster-Suse.yml * Update main.yml * Update main.yml * Update 1.18.2-provision.yml * pacemaker 1.18 overhaul * Update 1.18.2.0-cluster-RedHat.yml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update 1.18.2-provision.yml * Update 1.18.2.0-cluster-Suse.yml * re-enable SSH key based authentication * Update main.yml * fixes * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * troubleshooting skipping of observer_db node * Update playbook_04_00_00_db_install.yaml * update to cluster config * Update playbook_04_00_00_db_install.yaml * task naming simplification for accurate debugging * Update 1.18.0-set_runtime_facts.yml * Update 1.18.1-pre_checks.yml * Update 1.18.2-provision.yml * Update 1.18.2.0-cluster-Suse.yml * corosync template error * Update corosync.conf.j2 * variables port * Update main.yml * Update 1.18.2.0-cluster-Suse.yml * switch to using primary and secondary instance names * Update 1.18.2.0-cluster-RedHat.yml * add code for cluster configuration for SUSE post pacemaker configuration * Update 5.5.4.2-cluster-ScaleOut-Suse.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * add 20-saphana.j2 file * Update 20-saphana.j2 * Update 20-saphana.j2 * Update 20-saphana.j2 * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * fix for RHEL * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update main.yaml * Update 1.18.2.0-cluster-Suse.yml * Update 1.18.2.0-cluster-Suse.yml * update to 20-saphana sudoers file * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * rename scale out task file * bug fix * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * Update 5.5.4.0-clusterPrep-ScaleOut-Suse.yml * minor fix * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * Update 5.5.4.0-clusterPrep-ScaleOut-Suse.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * fix to SLES clustering code and installing SAPHANASr-Scaleout package for majority maker * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * minor fix for SAPHANAsr-multitarget subtask * fix for majority maker node not installing hook plugin * Added code for RHEL pacemaker configuration * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * MM-01 plugin issue bugfix * SAP HANA hook fix 2 * remove SCS references from 1.18 task as its specific to HANA scale out pacemaker creation * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.2-sbd.yaml * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update main.yml * minor fix for RHEL pacemaker * Update 1.18.3-post_provision_report.yml * add pacemaker code for RHEL * minor fix to hana basepaths * Update main.yaml * slight update to hana paths * Update main.yaml * syntax fix * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update os-packages.yaml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * changes to packages for scale out * development test push * Update ansible-input-api.yaml * Update ansible-input-api.yaml * Update ansible-input-api.yaml * Update ansible-input-api.yaml * package fix for RHEL * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * change to how sap starts and stops * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * remove HTTPS prot flag in saphostagent * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update main.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * pacemaker fix * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update os-packages.yaml * Update os-packages.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * rhel pcs node attribute fix * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * replace availability variable replace db_high_availability with database_high_availability * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * post provision fix for scale out * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update ansible-input-api.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * scale out cluster config overhaul * refactor, move scale out code seperate * scale out specific provision tasks * Update main.yml * Update main.yml * Update playbook_04_00_01_db_ha.yaml * Update 1.18.0-set_runtime_facts.yml * Update main.yml * Update main.yml * Update 5.5.2-pre_checks.yml * fix error on import vs include in 5.5 sub tasks * add majority maker constraints * post-provision code, minor code fixes for HANA ha config * Update 1.18.3-post_provision_report.yml * Update main.yaml * Update main.yaml * Update 1.18.3-post_provision_report.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * error code fix * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update main.yml * Update 5.5.3-SAPHanaSR.yml * changes to HANA Replication python hook configi * Update 20-saphana-rhel.j2 * Scale out code overhaul !!! * Update main.yaml * Update main.yaml * Update 5.8.4-provision-ScaleOut.yml * Update 5.8.4-provision-ScaleOut.yml * chore: Refactor deployment scripts and Terraform modules * Refactor deployment scripts and Terraform modules * chore: Mount SAP TransFilesystems and handle remounting when not using external NFS * Fix zonal code for observer * Don't add the client network if not used * Add become: true for reboot * Don't fail the reboot * Fix the merge conflicts * Rename variables * chore: Remove commented out code for database_high_availability in ANF Mount task * chore: Update variable names and fix merge conflicts * Use UID and not the actual account * Remove SAS tokes from storage account * chore: Refactor ANF Mount task to improve readability and remove commented out code * Split the task * Use AzureAD provider for storage * Move network rules outside the storage account * Support SAS less deployments * SAS less deployer * Refactor backend.tf files to use Azure AD authentication for storage * chore: Enable Azure AD authentication for Terraform remote state imports * chore: Refactor ANF Mount task to improve readability and remove commented out code * Refactor ANF Mount task to improve readability and remove unnecessary code * Fix shared volume * Fix the outputs * Change the if statement * Fix Mount output * Fixes for secondary * Add hana_shared_basepath * feat: Add hana_usrsap_basepath variable to ansible-input-api.yaml * fixed typo * Refactor ANF Mount task to for SHA scaleout * Change the until condition * Make the HANA packages * remove the package * Fix path * remove duplicate * chore: Update count condition in dns.tf for local private DNS usage * chore: Update and remove duplicate resource definitions for kdump disks output in anydb_node, kudmp disks and extensions in app_tier module * Add tags for deployer * refactor: Update tags assignment in sap_deployer module * Use the built in token to authenticate to ADO * Updates to scale out HSR code (#598) * Fix the deployment using SPN * Ensure that custom_logical_volumes can be striped + have sensible stripesize default if a lvm consists of more than one pv. (#587) Use already established pattern from framework specific LVMs to define stripesize on custom logical volumes. * Perform az login before the az access token in ams provider creation steps * Revert "Perform az login before the az access token in ams provider creation" * Update AMS provider creation tasks in ansible playbook * Update deploy control plane pipeline to use service principal for authentication * chore: Temporarily set identity type to "SystemAssigned, UserAssigned" in app_service.tf * test for new /etc/hosts configuration for HSR scale out * Update hosts.j2 * Update hosts.j2 * Update hosts.j2 * Various Ansible fixes (#591) * Add Red Hat 8.10 repo to 1.3-repository vars * Create entries for Red Hat 8.10 in 1.4-packages vars * Add 'pam' to OS packages list for DB2 with state 'latest' Ensures that x86_64 package is updated, avoiding conflict with libpam.so.0 install, which requires i686 version of pam * Add "state: 'latest'" to loops * Correct cluster version check in 1.17-generic-pacemaker * Correct cluster version check in 5.6-scsers-pacemaker * Correct cluster version check in 5.5-hanadb-pacemaker * Create entries for Red Hat 8.10 in 1.17-generic-pacemaker --------- Co-authored-by: Csaba Daradics <[email protected]> * add code for scaleout - hook * fix to HSR code branch * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update 1.4.3-update-packages-RedHat.yaml * Update 1.4.3-update-packages-RedHat.yaml * chore: Adjust Azure fence agent packages and remove unused Azure Python packages from list when deploying on SLES 15 SP5 * chore: Override changed status for saptune_check and active_saptune_solution tasks in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Update New-SDAFDevopsWorkloadZone.ps1 script to fix variable group creation issue * Update main.yaml * Update playbook_04_00_01_db_ha.yaml * Update main.yaml * Update main.yaml * chore: Refactor saptune_solution_to_apply logic in 2.10.3.yaml * Set HDB schema name for ABAP and JAVA systems (#593) * Set HDB Schema Name task * fix command error and remove ignore_errors * Fix parsing error * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * chore: Comment out unnecessary role assignments in New-SDAFDevopsProject.ps1 script * Update main.yaml * exclude sapmnt from observer * bug fix, Scale out tasks not running on Majority maker node in playbook 4_01 * Update playbook_04_00_01_db_ha.yaml * replace pause with wait-for * replace pause with wait_for * enable AUTOMATED_REGISTER to true * Release testing (#597) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Rename quality check to quality assurance (#600) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * chore: Remove cacheable flag from 3.3 BoM Processing task * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * chore: move SAP on Azure quality checks after post configuration * chore: Update quality check paths to quality_assurance * chore: Update quality assurance file paths * chore: Refactor YAML files to improve code organization and readability * chore: Add cacheable flag to 3.3 BoM Processing task --------- Co-authored-by: dkSteBTh <[email protected]> * Release v3.11.0.3 (#599) * Bring in the manual updates * Fixing Merge conflicts * Fix Linting * Pacemaker changes, saptune and NAT Gateway (#583) ## Summary of Changes ### Authentication and Identity Management - **Web Application Authentication Configuration**: Repeated updates to refine and simplify the authentication configuration and identity management scripts. ### Repository and Package Management - **SLES Repositories**: Added repositories for SLES 15.3, 15.4, and 15.5. - **WAAgent Updates**: Updated WAAgent package, configuration checks, and systemd service reloads across multiple roles. ### Deployment Configuration - **NAT Gateway**: Added support for provisioning a NAT gateway, including configuration variables in Terraform files. - **AutoUpdate Configuration**: Updated AutoUpdate.Enabled settings and added Extensions.WaitForCloudInit across various roles. - **Oracle Simplification**: Simplified Oracle-related configurations, including ASM backup process and Data Guard tasks. - **SAP Deployment Playbooks**: Various updates to SAP deployment playbooks, including fixing conditions, resource flag settings, and systemd service paths. ### Infrastructure and Pipeline Enhancements - **Control Plane Pipeline**: Multiple fixes and improvements to error handling, logging, environment variables, and Azure AD authentication. - **Terraform and Ansible Versions**: Updated versions in deployment scripts to 1.7.5. - **Dotnet SDK**: Bumped dotnet SDK installation to the latest version. ### Miscellaneous - **Error Handling and Logging**: Improved error handling and logging across various deployment scripts and playbooks. - **Validation Fixes**: Fixed validation conditions for disk space, OS version checks, and cluster initialization commands. --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]> * Fix regex necessary to comment lines in /usr/sap/sapservices (#584) Co-authored-by: Csaba Daradics <[email protected]> * remove duplicate resource * Fix path * chore: Fix the count for the table resource * Misc fixes * Fix systemd service reload in 1.4 Packages role * Various Terraform code fixes (#586) * Fix typo in terraform-units/modules/sap_landscape/providers.tf * Remove duplicate of resource azurerm_network_security_rule/nsr_controlplane_storage In terraform-units/modules/sap_landscape/nsg.tf * Remove fourth argument from nat_gateway_name definition In terraform-units/modules/sap_landscape/variables_local.tf * Remove duplicate for database_kdump_disks In terraform-units/modules/sap_system/anydb_node/outputs.tf * Remove all duplicates from terraform-units/modules/sap_system/app_tier/vm-scs.tf * Remove duplicates in terraform-units/modules/sap_system/output_files/sap-parameters.tmpl --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Update count condition in dns.tf for local private DNS usage * chore: Update NAT Gateway public IP name format * chore: Update NAT Gateway public IP lifecycle configuration * chore: Update NAT Gateway provider to azureng * chore: Fix typo in azureng provider configuration alias in sap_landscape module * chore: Update NAT Gateway provider to azurerm.main * Update 01-deploy-control-plane.yaml * chore: Update app_service_plan name format in sap_deployer module * Update ARM_CLIENT_SECRET assignment in deploy control plane pipeline * Add the compliance extensions also to the deployers * Ensure that custom_logical_volumes can be striped + have sensible stripesize default if a lvm consists of more than one pv. (#587) Use already established pattern from framework specific LVMs to define stripesize on custom logical volumes. * Update AMS provider creation tasks in ansible playbook * Update deploy control plane pipeline to use service principal for authentication * chore: Temporarily set identity type to "SystemAssigned, UserAssigned" in app_service.tf * Various Ansible fixes (#591) * Add Red Hat 8.10 repo to 1.3-repository vars * Create entries for Red Hat 8.10 in 1.4-packages vars * Add 'pam' to OS packages list for DB2 with state 'latest' Ensures that x86_64 package is updated, avoiding conflict with libpam.so.0 install, which requires i686 version of pam * Add "state: 'latest'" to loops * Correct cluster version check in 1.17-generic-pacemaker * Correct cluster version check in 5.6-scsers-pacemaker * Correct cluster version check in 5.5-hanadb-pacemaker * Create entries for Red Hat 8.10 in 1.17-generic-pacemaker --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Adjust Azure fence agent packages and remove unused Azure Python packages from list when deploying on SLES 15 SP5 * chore: Override changed status for saptune_check and active_saptune_solution tasks in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Update New-SDAFDevopsWorkloadZone.ps1 script to fix variable group creation issue * chore: Refactor saptune_solution_to_apply logic in 2.10.3.yaml * Set HDB schema name for ABAP and JAVA systems (#593) * Set HDB Schema Name task * fix command error and remove ignore_errors * Fix parsing error * chore: Comment out unnecessary role assignments in New-SDAFDevopsProject.ps1 script * Release testing (#597) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Rename quality check to quality assurance (#600) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * chore: Remove cacheable flag from 3.3 BoM Processing task * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * chore: move SAP on Azure quality checks after post configuration * chore: Update quality check paths to quality_assurance * chore: Update quality assurance file paths * chore: Refactor YAML files to improve code organization and readability * chore: Add cacheable flag to 3.3 BoM Processing task --------- Co-authored-by: dkSteBTh <[email protected]> --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: daradicscsaba <[email protected]> Co-authored-by: Csaba Daradics <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: Nadeen Noaman <[email protected]> Co-authored-by: Steffen Bo Thomsen <[email protected]> Co-authored-by: Jesper Severinsen <[email protected]> * Bump Azure.Identity from 1.11.3 to 1.11.4 in /Webapp/SDAF (#594) Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.11.3 to 1.11.4. - [Release notes](https://github.com/Azure/azure-sdk-for-net/releases) - [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Identity_1.11.3...Azure.Identity_1.11.4) --- updated-dependencies: - dependency-name: Azure.Identity dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Update os-packages.yaml for redhat8.6 * enable EPEL repositories for RedHat, enable unrar package to allow for newer database schemas * Update repos.yaml * restrict unrar to SCS tier only * Update bom_processor.yaml * Update os-packages.yaml * test "unar" no directory option to force files unpack to CD_EXPORT in flatten structure * experimental change to check for nested directories when running unar for Redhat on schema files * Update bom_processor.yaml * port RAR exe fix from windows to Linux 3.3 bom for testing * Update process_exe_archives.yaml * Update process_exe_archives.yaml * made a minor typo in path due to habitual CMD.exe user * Update process_exe_archives.yaml * Update process_exe_archives.yaml * Update process_exe_archives.yaml * Update process_exe_archives.yaml * chore: Create directories for SAP deployment automation * chore: Update authentication prompt for App Registration configuration * Update configuration_menu.sh * chore: Update PostBuildCleanup task to version 4 in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 for all stages in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 in deploy pipelines * chore: Update clusterPrep-RedHat.yml to avoid resource discovery during location constraints * Do not fail on saptune solution verify (#602) Set failed_when to false, so that saptune does not fail on N/A parameters. * chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists * Revert "chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists" This reverts commit e86dff14a149d8c866b2ce5b4570f2212959c062. * chore: Update accelerated networking configuration in Terraform modules, as enable_accelerated_networking is deprecated; new parameter is accelerated_networking_enabled * Ensure we are in the right context when getting access tokens and subsequently running the ps1 script, where we already have the trust setup for the SSH key. Not doing it this way, leads to either needing to manually create an SSH session inside pwsh with POSH-SSH to ensure the known_hosts entry is updated or having to update the quality check script upstream, to allow the -Force flag for the SSH session. (#603) * Fix for catching AHCO_INA_SERVICE delivery Unit import failure (#605) Co-authored-by: jasksingh <[email protected]> * Web App Component updates * Several (bug)fixes for RHEL deployments and deployments in general (#604) * Add fast_stop=no to pacemaker fileystem resources Pacemaker isn't respecting the stop timeout on filesystem resources due to the default setting fast_stop=yes. Without setting fencing will occur because if SAP (A)SCS / ERS isn't stopped in time processes will be terminated which are restarted by sapstartsrv and node will be fenced because fileystem can't be unmounted. https://www.suse.com/support/kb/doc/?id=000020860 https://access.redhat.com/solutions/4801371 * Distribute systemd services between SCS / ERS nodes and stop services Both (A)SCS and ERS systemd services should be present on SCS and ERS nodes otherwise pacemaker only handles SCS on the SCS node and ERS on the ERS node with the systemd integration. * Add resource clear for move contrainsts on (A)SCS resource group * Bugfix folders on local disks to be managed after mounting local disk * sid_private_key isn't required * Add sdu_secret prefix/suffix to manage custom Key Vault secret naming * Update Web App to support NAT Gateway * Hotfix release after testing with new RHEL image (#611) * update: added fixed encountered during RHEL94 testing * chore: Update Red Hat and SLES package versions for Red Hat 9.4 * update: add network rules to deployer diagnostic storage account * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Add ${SID}_virtual_machines.json to git if it exists * chore: Update sap_system module to use database_server_vm_resource_ids for database server VMs * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to use comma-separated database server VM resource IDs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap_system module to include application server VM resource IDs * chore: Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor use_spn variable to default to false for all SAP deployment modules * Refactor systemd process limit configuration for pacemaker cluster * Refactor systemd process limit configuration for pacemaker cluster * Update os-packages.yaml (#613) * chore: Refactor Azure Fencing Agent creation in 1.17 Generic Pacemaker role (#614) * Add SAP-CAL Integration for non-HA Installation (#608) * Add AVG support for Scale out scenarios (#577) * Add data and log volumes * Refactor AVG logic * Fix proximity_placement_group_id calculation in avg.tf * Refactor for_each condition in avg.tf * Refactor for_each condition in avg.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor zone calculation logic in variables_local.tf * Refactor proximity_placement_group_id calculation in avg.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in vm-app.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in infrastructure.tf * Refactor package update condition in 1.4.3-update-packages-RedHat.yaml --------- Co-authored-by: Kimmo Forss <[email protected]> * Update subnet_cidr_storage in sap-parameters.tmpl * Update hosts jinja for client subnet * Update SAP-specific configuration playbook for HANA database scale-out scenario * Version update * Simplify Web App Identity management * Update Azure package versions in SDAFWebApp.csproj * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Add SLES 15.3, 15.4, and 15.5 repositories * Update Web Application authentication configuration script and simplify Web App Identity management * Refactor Web App Identity management and update authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script and simplify Web App Identity management * Commented out SSH trust relationship checks in 1.17.2-provision.yml * Revert "Commented out SSH trust relationship checks in 1.17.2-provision.yml" This reverts commit 09cd30de6003a891b5c8c31b4c96b495b676aa9b. * ACSS updates * Oracle simplification * Add AutoUpdate.Enabled configuration in 1.1-swap role and enable package cache update in 1.4-packages role * Update deployment type configuration in OS and SAP specific playbooks * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update WAAgent package and restart service in 1.1-swap role * Updated key_vault_sap_landscape.tf * Revert "Updated key_vault_sap_landscape.tf" * Update WAAgent package and restart service in 1.1-swap role * Add SAP CAL Integration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Revert "Add SAP CAL Integration" This reverts commit adae6662ba478d9f1d4d0de7f5c175e4f5da739b. * Update WAAgent package and restart service in 1.4-packages role * Update waagent configuration check in 1.4-packages role * Update waagent configuration check and systemd service reload in 1.4-packages role * Update AutoUpdate.Enabled configuration and add Extensions.WaitForCloudInit configuration in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update database_high_availability condition in playbook_04_00_01_db_ha.yaml * Add the ability to block app registration * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update web_instance_number and add web_sid variable in sap_system/transform.tf * Fix validation error message for web dispatcher sid in variables_global.tf * Remove chkconfig package from os-packages.yaml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update OS version check for RHEL 8.2 and SLES 15 in 5.6.1-set_runtime_facts.yml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update Oracle ASM backup process and fix file permissions * Fix file path in 1.4.0-packages-RedHat-prep.yaml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update file path and preserve file permissions in 1.4.0-packages-RedHat-prep.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Update wait time for StartService in 5.6 SCS/ERS Validation * Update Terraform version to 1.8.0 in deployment scripts and tfvar_variables.tf files * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Update virtual machine extension reference in vm.tf * Update virtual machine extension version to 1.0 in vm.tf * Fix missing else statement in deploy control plane pipeline * Update network interface and virtual machine counts in vm-observer.tf * Update database high availability configuration * Update use_spn property to false in LandscapeModel and SystemModel * Update Terraform and Ansible versions to 1.7.5 in deployment scripts and variables * Update Display value in SystemDetails.json * Fix validation condition in variables_global.tf * Add ORACLE Post Processing: Reboot after Enabling HugePages task * Fix typo in Oracle Data Guard - Observer: Change UID for Oracle user task * install passlib * Add patch_mode support * Update deployment playbook to set single_server fact based on host count * Update patch_mode configuration in Terraform files * Update file permissions in SAP deployment playbook * Update deployment playbooks to set single_server fact consistently * Fix waagent configuration in swap role * Fix indentation in swap role tasks/main.yaml * Fix cluster group move command in 5.6 SCS/ERS Validation playbook * Fix condition in 1.17-generic-pacemaker playbook to exclude node_tier 'hana' * Fix commented out corosync configuration in 1.17-generic-pacemaker playbook * Create the SID subfolder * Update verbosity level in 5.6.7-config-systemd-sap-start.yml * Add passlib * Simplify Python logic * Update app_bom_id variable in 5.3-app-install/tasks/main.yaml * Update passlib installation in Ansible playbooks * Update reboot timeout and post-reboot delay in 5.6.4.2-sap-resources-Suse.yml * Update swap role and package tasks * Fix condition in 1.17-generic-pacemaker playbook to exc…

* Refactor hana_shared_afs_path output to conditionally include values based on var.database.scale_out * Add the ability to control access key based access to storage accounts * Web App SAS updates * Refactor CustomValidators.cs to handle null values in IsValid method * check string length * Refactor CustomValidators.cs to handle null values in IsValid method * Refactor playbook_04_00_00_db_install.yaml to handle null values in secondary_instance_name * Refactor Azure DevOps project configuration * Refactor site.js to handle additional storage and network options * Refactor storage_accounts.tf to conditionally count private endpoint connections * Refactor sap_namegenerator/output.tf to handle null values in ANYDB_COMPUTERNAME, ANYDB_SECONDARY_DNSNAME, and ANYDB_VMNAME * Cluster validation logic * Refactor 5.6 SCS/ERS validation playbook to improve efficiency and readability * Refactor 5.6 SCS/ERS validation playbook to set sapcontrol path and improve efficiency * remove the unar package * Refactor 5.6 SCS/ERS validation playbook to set sapcontrol path and improve efficiency * Refactor repository vars to remove epel repo for all Red Hat versions Install unar package on RHEL for extracting exe archives * Refactor 1.1-swap playbook to use dnf module for updating WALinuxAgent package * allow_world_readable_tmpfiles for find * Refactor become_user in DBLoad, PAS Install, and APP Install playbooks to use "root" instead of "{{ sid_to_be_deployed.sid | lower }}adm" * add Environment variable to allow for silent installations * Add the ability to set the shared access key access for the sap system storage accounts * Add 'resource-agents-cloud' package for db2 * Refactor OS image configuration in sap_system/transform.tf * Refactor HANA_2_00_install.rsp template to fix use_master_password formatting * Refactor HANA_2_00_install.rsp template to fix password_copy formatting * Refactor cluster stabilization check for RHEL 8.x and 9.x * Refactor HANA_2_00_install.rsp template to omit password_copy when hana_use_master_password is 'n' * Refactor HANA_2_00_install.rsp template to remove password_copy and fix formatting * Refactor cluster stabilization check for RHEL 8.x and 9.x * Refactor HANA_2_00_install.rsp template to fix password_copy formatting * Refactor DB2 cluster tasks for RHEL 8.x and 9.x * Refactor cluster stabilization check for RHEL 8.x and 9.x * Refactor cluster stabilization check for RHEL 8.x and 9.x * Refactor cluster stabilization check for RHEL 8.x and 9.x * Refactor allow_world_readable_tmpfiles formatting in HANA installation role * Update azuread provider version to >=3.0 in sap_deployer, sap_library, sap_landscape, sap_library, and sap_system providers.tf files * Fix the spacing in the ansible role for the SAP 5.2 PAS install * Refactor spacing in HANA installation role and cluster stabilization check for RHEL 8.x and 9.x * Refactor password_copy logic in HANA installation role * Refactor password_copy logic in HANA installation role * Refactor password_copy logic in HANA installation role * Refactor password_copy logic in HANA installation role * Refactor password handling in HANA installation role * Refactor password handling in HANA installation role * Refactor password handling and hdbuserstore path in HANA installation role * Refactor password handling and hdbuserstore path in HANA installation role * Refactor password handling in HANA installation role * Refactor ownership in PAS installation role * Refactor PAS installation role to remove unnecessary debug message * Refactor become_user in PAS installation role to use lowercase SID adm user * Refactor become_user in PAS installation role to use root user * Refactor HANA installation role to include required packages * Refactor tfvar_variables.tf to include shared access key enabled variables * Web App updates for shared access keys * Add tier info for packages task * Enable world-readable temporary files in ansible.cfg * add the missing quotes * Refactor SAP OS configuration playbook to include directory permissions task * Refactor SAP OS configuration playbook to include directory permissions task * Add 'resource-agents-sap-hana' to os-packages.yaml * Refactor SAP OS configuration playbook to include directory permissions task * Refactor playbook to use 'import_tasks' instead of 'include_tasks' for clustering based on OS in VM * chore: update ansible.cfg log path to /var/tmp/ansible.log * Refactor clustering tasks in HANADB Pacemaker playbook * Refactor clustering tasks in HANADB Pacemaker playbook * Refactor clustering tasks in HANADB Pacemaker playbook to use 'import_tasks' instead of 'include_tasks' for clustering based on OS in VM * Refactor clustering tasks in HANADB Pacemaker playbook to use 'import_tasks' instead of 'include_tasks' for clustering based on OS in VM * Refactor clustering tasks in HANADB Pacemaker playbook to use 'import_tasks' instead of 'include_tasks' for starting HANA Database * Refactor clustering tasks in HANADB Pacemaker playbook to use 'import_tasks' instead of 'include_tasks' for starting HANA Database * Remove the include tasks * Linting --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]>

…field deployment where an existing firewall is used. Defaults to false

SteffenBoThomsen changed the title ~~(WiP) Add flow_timeout_in_minutes to VNETs.~~ Add flow_timeout_in_minutes to VNETs. Nov 11, 2024

SteffenBoThomsen marked this pull request as ready for review November 11, 2024 08:28

SteffenBoThomsen requested a review from a team as a code owner November 11, 2024 08:28

SteffenBoThomsen requested review from KimForss and removed request for a team November 11, 2024 08:28

SteffenBoThomsen changed the base branch from experimental to development November 11, 2024 10:18

SteffenBoThomsen changed the base branch from development to experimental November 11, 2024 10:19

Kimmo Forss and others added 23 commits November 11, 2024 11:20

Bring in the manual updates

0ebfddc

remove duplicate resource

c26e3c2

chore: Update NAT Gateway provider to azureng

ab73849

chore: Update NAT Gateway provider to azurerm.main

e118420

chore: Update clusterPrep-RedHat.yml to avoid resource discovery duri…

2805e06

…ng location constraints

chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for …

afa1eb1

…subscription and identity lists

Revert "chore: Update New-SDAFDevopsProject.ps1 to use tsv output for…

1274761

…mat for subscription and identity lists" This reverts commit e86dff1.

chore: Refactor Azure Fencing Agent creation in 1.17 Generic Pacemake…

72dee87

…r role (Azure#614)

chore: Update use_local_privatelink_dns condition in sap_library module

ad5495e

chore: Update private DNS configuration in sap_library module

1454488

chore: Update Test-SDAFReadiness.ps1 script

00d852b

Add flow_timeout_in_minutes to management VNET.

29a7299

Add flow_timeout_in_minutes to sap VNET.

9e5589b

SteffenBoThomsen added 2 commits November 11, 2024 11:24

update validation condition check

429dae8

bgp_enable_route_propagation as an option, useful when doing a brown …

7c1ef5d

…field deployment where an existing firewall is used. Defaults to false

SteffenBoThomsen force-pushed the vnet-flow-timeout branch from ea636ba to 7c1ef5d Compare November 11, 2024 10:26

SteffenBoThomsen changed the base branch from experimental to development November 11, 2024 10:27

SteffenBoThomsen marked this pull request as draft November 11, 2024 10:28

SteffenBoThomsen closed this Nov 11, 2024

SteffenBoThomsen deleted the vnet-flow-timeout branch November 11, 2024 10:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flow_timeout_in_minutes to VNETs. #661

Add flow_timeout_in_minutes to VNETs. #661

SteffenBoThomsen commented Nov 5, 2024 •

edited

Loading

Add flow_timeout_in_minutes to VNETs. #661

Add flow_timeout_in_minutes to VNETs. #661

Conversation

SteffenBoThomsen commented Nov 5, 2024 • edited Loading

Problem

Solution

SteffenBoThomsen commented Nov 5, 2024 •

edited

Loading