Add support for HANA Scaleout with HSR #637

…into experimental

… key vault firewalls

…able description

…into experimental

…variable in tfvar_variables.tf

…into experimental

…sage

… role

…into experimental

- Update virtual host fact in hosts file

…e 2.4-hosts-file task.

… list

…apply (#560) * Update return_value with new return code to stop when plan failed When the Terraform Plan has errors, the script will continue to run. The return_value was not being updated with the return code of the plan, so the check is not working. * Sanitaze shebang for bash scripts * Update installer.sh * Update installer.sh

update to use setup-python@v5

…iable error (#562) * Add optional, extended log collection functionality * Use default filter with kdump_enabled variable Prevent 'kdump_enabled' is undefined errors --------- Co-authored-by: Csaba Daradics <[email protected]>

…ntal

* lsnrctl status update * oracle-asm dataguard setup * asm file permission updates * asm variable update * asm file conditional change * oracle-asm file creation updates * oracle-asm listener handling * asm listener handling * asm lsnrctl reload test * asm listener adjustments * oracle asm initSID.ora file updates * asm * asm testing * rman command update * update the replace function * oracle-asm * fail message update * asm * remove fail flag * asm restore update * oracle-asm db updates * oracleasm changes * oracle-asm update1 * oracle-asm2 * oracle-asm restore adjustments * syntax error whilst creating asm files * asm retrofit with spfile and restore scripts * asm update * asm flashback folder creation * asm * enable fsfo on secondary * asm * asm clusterware config * asm oracle cluster ware restart updates * asm updates for registering stdby in srvctl * asm * asm * ASM changes * Oracle DG automated trigger testing * listener change * dbload changes * ASM DG changes * asm * asm * asm * asm * oracle grid sbp location update * grid file permissions * gsbp file permissions * grid sbp change * gsbp patching * ' updated * oracle * sidadm creation on secondary db node * sidadm for oracle secondary * user change * sidadm for oracle * asm dg * oracle asm dg * oracle asm opatch * updated SBPFUSER variable for grid patching * added oracle-asm node * updated the post-install file * grid sbp patch * grid sbp * grid sbp copy * grid * grid update * tnsnames update for sap app * sbp grid error handling * spell corrrection * error handling * grid patching * sbp 2308 fixes * grid * grid sbp test * grid sbp new version testing * grid * grid * ok * ok * sbp * grid sbp * GRID SBP * gsbp oradism paermissions * SBp * grid sbp * debug * debug * grid pre-install patch * debug * debug1 * debug * old mopatch and opatch to test * old sbp * opatch 11 * sbp updates * sbp * grid sbp folder creation * change the order of SBP GRID first then RDBMS * permission update * oracle grid * grid patching 2311 testing * comment out GRID Patching * sga and pga adjustments * oracle changes * repo update added 8.9 * added packages for OEL8.9 * repo * spfile changes * oracle pga caluculated based on the oracle documentation. * lsnrctl on the secondary is ot running yet remove the lsnrctl stop. * lsnrctl for asm updated * syntax correction * syntax correction for paramter * oracle asm lsnrctl parameter update * syntax update * updated the register parameter * re-arranged the order for evaluation of node_tier * updated the syntax for node_tier * rearranged the order of the conditions * non-asm ha setup * creating sidadm user on secondary * rman restore for non-asm updates * initsid.ora parameter updates for oracle HA * updating the spfile for ora non-asm secondary * update the flag file * rman duplicate change * update the spfile * spfile update for oracle * oracle dataguard replication * oracle non-asm dg changes * reduced the temp disk space to 50 from 100 on the deployer * reduced the tmp disk space setting * updated the /mnt value check to bypass the free space check * updated code to handle oracle bug whilst creating the redologs on secondary * oracle non-asm dg setup update * updated the local_listener value in primary to re-register as secondary after failback * converted caps to small for local_listener values * added rebstart of secondary to activate HA service for SAP * adding oraflash filesystem creation * oracle non-asm changes * updates the oracle sga and pga caluculation * oracle listener on primary update * spfile scope update * oracle local_listener changes * commenting the local_listener as it is now implemented after dbload * oracle listner updates * debug the file update * debug listener change * file update for local_listener value * commeting out the debug lines * changing the sequeunce of reboot * added reboot block to allow database to set correct huge pages * error handling for lsnrctl restart on primary * correcting the typos

* Make SCS Shared disk Premium_ZRS * Add support for Premium_ZRS zonal disks * Add Cluster disk info to the Web App * Don't specify a zone for Premium_ZRS disks * Ensure that the --username parameter is correct * Fix error message in variables_global.tf * Remove unused variables in variables_global.tf * Refactor error message in variables_global.tf * Added database_cluster_disk_type * Update scs_cluster_disk_lun and database_cluster_disk_lun properties in SystemModel.cs * Remove trailing spaces in parameter names in SystemDetails.json * Fix ARM_CLIENT_ID variable interpolation in configure_deployer.sh.tmpl * Update proximity_placement_group_id in vm-app.tf * Fix zone assignment for scs_cluster_disk_type in vm-scs.tf * Update disk zone configuration * Update proximity_placement_group_id in vm-app.tf * Update proximity placement group ID in app tier VM configuration * Merge branch 'experimental' --------- Co-authored-by: Kimmo Forss <[email protected]>

* adding variable for deploying HANA scale out - ANF without a standby node. * Add disclaimer

* Make SCS Shared disk Premium_ZRS * Add support for Premium_ZRS zonal disks * Add Cluster disk info to the Web App * Don't specify a zone for Premium_ZRS disks * Ensure that the --username parameter is correct * Fix error message in variables_global.tf * Remove unused variables in variables_global.tf * Refactor error message in variables_global.tf * Added database_cluster_disk_type * Update scs_cluster_disk_lun and database_cluster_disk_lun properties in SystemModel.cs * Remove trailing spaces in parameter names in SystemDetails.json * Fix ARM_CLIENT_ID variable interpolation in configure_deployer.sh.tmpl * Update proximity_placement_group_id in vm-app.tf * Fix zone assignment for scs_cluster_disk_type in vm-scs.tf * Update disk zone configuration * Update proximity_placement_group_id in vm-app.tf * Update proximity placement group ID in app tier VM configuration * Merge branch 'experimental' * Add storage subnet for SAP VNET * Add storage_subnet_id and storage_nsg_id outputs to sap_landscape module * Add SAP storage subnet NSG and associate it with the storage subnet. Add SSH network security rule for connectivity to SAP application subnet from Control Plane. * Update NSG rule protocol to allow all traffic * Refactor proximityPlacementGroup assignment in avg.tf * Update NSG protocol to allow all traffic * Refactor subnet_storage configuration in transform.tf * Add storage subnet to Web App * Get the latest * Add scaleout variables * Update proximityPlacementGroup in avg.tf * Add storage subnet variables for scale-out configuration * Update storage subnet condition for ANF support * Refactor storage subnet count logic * Fix conditional expression in subnets.tf * Refactor network_interface_ids in vm-hdb.tf * Custom Mount fix * Add "Custom" sizing to UX * Create the root folder if it does not exist * Add ANF & Storage subnet prefixes to sap-parameters.yaml * Update storage subnet netmask variable name * Fix storage subnet ID in variables_local.tf * Fix ANF_subnet_prefix value in outputs.tf * Refactor sap-parameters.tmpl file * Update subnet prefixes for storage and ANF * Update principal_id in azurerm_role_assignment * Fix src path in custom mount task * Update IP address retrieval in Ansible playbooks * Update variable name in main.yaml * Update debug message to use correct variable name * Update debug message to print ipadd variable * Update networking tasks and hosts file template * Add client subnet * Add subnet_prefix_client to sap-parameters_yml resource * Update IMDS URL in networking tasks * Refactor host entries in hosts.j2 template * Fix formatting and typo in sap-parameters.tmpl * Refactor host entries generation in hosts.j2 template * Add routes and restart VM for HANA scaleout * Add app subnet * Add use_msi_for_clusters variable to sap-parameters.tmpl * Fix conditional value assignment in outputs.tf * Update network configuration details in main.yaml * Update sap-parameters.tmpl file * Update enable_storage_subnet condition in variables_local.tf * Debug * Web App updates * Refactor virtual host name assignment in hosts.j2 template * Add description property * Add the download ability * Add the MSI to the extension object * Fix virtual host names duplication issue * Fix virtual host name iteration in hosts.j2 template * Add support for secondary IP addresses in azure_interfaces.j2 template * Add dependencies for storage network interface creation * Refactor networking configuration and route creation. * Add DB subnet to sap-parameters.yaml * Update network interface conditions in main.yaml * Update VM-Images.json with new SKUs * Update internal network configuration in HDB installation playbook * Update database host roles in main.yaml * Update internal network configuration in HDB install playbook * Fix missing quotation mark in hdblcm command * Update internal network configuration in HDB install playbook * Refactor HDB installation command in main.yaml * Add additional destination port ranges to NSG rule * Task naming and Linting * Add database_HANA_no_standby_role variable --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]>

* Add the Monitoring Extension to the Application Tier * Replace Windows virtual machine with Linux virtual machine in monitoring extension * Add monitoring extension to database tier * Update destination address prefixes in NSG rules * Fix destination_address_prefixes in nsg.tf * Update monitoring extension count based on database OS type * Add Monitoring extension for iSCSI servers * Add deploy_monitoring_extension parameter * Add storage subnet data source * Add Download link in Landscape/Edit view and update default value for deploy_monitoring_extension * Update monitoring extension names for utility VMs * Refactor monitoring extension deployment conditions in vm.tf --------- Co-authored-by: Kimmo Forss <[email protected]>

* Add the ability to control if Private Endpoints are registered with DNS * Merge branch 'experimental' * Add register_endpoints_with_dns option to common_infrastructure and hdb_node modules * Add register_endpoints_with_dns property to LandscapeModel and SystemModel --------- Co-authored-by: Kimmo Forss <[email protected]>

* Add the Defender Agent * Update monitoring_defender_app_lnx and monitoring_defender_app_win resource names * Update source_address_prefixes in NSG rules * Update Azure Monitor Linux Agent name * Fix Azure Monitor agent name in VM extensions * Update NSG rules to use the first address space in vnet_sap * Refactor NSG rules to use address_space directly * Fix subnet_client_cidr calculation in SAP OS configuration playbook * Add conditional block for non-Windows systems in SAP OS configuration playbook * Remove unnecessary condition for non-Windows systems in SAP OS configuration playbook * Update DB virtual hostname resolution in 2.4 Hosts file --------- Co-authored-by: Kimmo Forss <[email protected]>

* Add tags to tasks * Refactor OS configuration playbook roles to use import_role instead of include_role * Refactor import_role to include_role in OS configuration playbook * Fix import_role in OS configuration playbook * Update swap role to include reboot and wait for connection tasks * Update timeout for wait_for_connection task * add tags and change to use import_role * Add deploy_defender_extension variable and update resource configurations * Add update_only flag to package update task * Add platform condition for oracle-asm tier * Update SAP system configuration * Update auto_upgrade_minor_version to true * Add tags to Ansible tasks in playbook_01_os_base_config.yaml and playbook_03_bom_processing.yaml * Add condition to skip reboot on Oracle Linux 8 * Add task to clear host errors and improve system reachability in 1.1-swap role * Refactor Oracle post-processing tasks*** * Update swap role tasks to include reboot and wait for connection * Add tags * Add post-reboot delay in swap role * Fix failed_when condition in swap role * Add become: true to reboot task * Add become flag for reboot task * Add tasks to clear host errors and wait for system to become reachable * Add kmod-oracleasm package to os-packages.yaml * Add reboot task and set failed_when to false in oracle-postprocessing.yaml * Add Microsoft AD collection to ansible-galaxy installations * Fix VM Agent Status check in 2.10.1 sap-notes * Remove oracleasm-support and kmod-oracleasm packages from oraclelinux8.9 --------- Co-authored-by: Kimmo Forss <[email protected]>

…perience (#575) * No Log for access token * Changes for SAP Media File share as Install experience

* Add data and log volumes * Refactor AVG logic * Fix proximity_placement_group_id calculation in avg.tf * Refactor for_each condition in avg.tf * Refactor for_each condition in avg.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor zone calculation logic in variables_local.tf * Refactor proximity_placement_group_id calculation in avg.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in vm-app.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in infrastructure.tf * Refactor package update condition in 1.4.3-update-packages-RedHat.yaml --------- Co-authored-by: Kimmo Forss <[email protected]>

…into experimental

…t scenario

…fy Web App Identity management

…uration script

…fy Web App Identity management

…on.yml" This reverts commit 09cd30d.

…age cache update in 1.4-packages role

…nsions.WaitForCloudInit configuration

…packages role

…udInit configuration in 1.1-swap role

…swap role

…a.yaml

…nsform.tf

…al.tf

…_facts.yml

…t-prep.yaml

…at-prep.yaml

…ora-dg/tasks/main.yaml

…iables.tf files

* overhaul of pacemaker approach for Scale out Shared nothing HSR cluster. * add code to 2.6 AFS mount to support /hana/shared for scale out HSR code * port changes to simple mount AFS task * Update 2.6.0-afs-mounts.yaml * Update 2.6.0-afs-mounts.yaml * Update 2.6.0-afs-mounts.yaml * anf mount bugfix * cleanup of stale code - anf /hana/shared mount * Update playbook_04_00_00_db_install.yaml * Update main.yaml * Update main.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.0-set_runtime_facts.yml * Update 1.18.2-provision.yml * Update 1.18.2-provision.yml * Update 1.18.2-provision.yml * experimental attempt to split pacemaker from hana setup * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * rollback * Update playbook_04_00_00_db_install.yaml * Update 1.18.2.0-cluster-Suse.yml * Update 1.18.2.0-cluster-Suse.yml * Update main.yml * Update main.yml * Update 1.18.2-provision.yml * pacemaker 1.18 overhaul * Update 1.18.2.0-cluster-RedHat.yml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * Update 1.18.2-provision.yml * Update 1.18.2.0-cluster-Suse.yml * re-enable SSH key based authentication * Update main.yml * fixes * Update playbook_04_00_00_db_install.yaml * Update playbook_04_00_00_db_install.yaml * troubleshooting skipping of observer_db node * Update playbook_04_00_00_db_install.yaml * update to cluster config * Update playbook_04_00_00_db_install.yaml * task naming simplification for accurate debugging * Update 1.18.0-set_runtime_facts.yml * Update 1.18.1-pre_checks.yml * Update 1.18.2-provision.yml * Update 1.18.2.0-cluster-Suse.yml * corosync template error * Update corosync.conf.j2 * variables port * Update main.yml * Update 1.18.2.0-cluster-Suse.yml * switch to using primary and secondary instance names * Update 1.18.2.0-cluster-RedHat.yml * add code for cluster configuration for SUSE post pacemaker configuration * Update 5.5.4.2-cluster-ScaleOut-Suse.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * add 20-saphana.j2 file * Update 20-saphana.j2 * Update 20-saphana.j2 * Update 20-saphana.j2 * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * fix for RHEL * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update main.yaml * Update 1.18.2.0-cluster-Suse.yml * Update 1.18.2.0-cluster-Suse.yml * update to 20-saphana sudoers file * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * rename scale out task file * bug fix * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * Update 5.5.4.0-clusterPrep-ScaleOut-Suse.yml * minor fix * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * Update 5.5.4.0-clusterPrep-ScaleOut-Suse.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.1-cluster-ScaleOut-Suse.yml * fix to SLES clustering code and installing SAPHANASr-Scaleout package for majority maker * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * minor fix for SAPHANAsr-multitarget subtask * fix for majority maker node not installing hook plugin * Added code for RHEL pacemaker configuration * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * MM-01 plugin issue bugfix * SAP HANA hook fix 2 * remove SCS references from 1.18 task as its specific to HANA scale out pacemaker creation * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.1-iSCSI.yml * Update 1.18.1.2-sbd.yaml * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update main.yml * minor fix for RHEL pacemaker * Update 1.18.3-post_provision_report.yml * add pacemaker code for RHEL * minor fix to hana basepaths * Update main.yaml * slight update to hana paths * Update main.yaml * syntax fix * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update 1.18.2.0-cluster-RedHat.yml * Update os-packages.yaml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * changes to packages for scale out * development test push * Update ansible-input-api.yaml * Update ansible-input-api.yaml * Update ansible-input-api.yaml * Update ansible-input-api.yaml * package fix for RHEL * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * change to how sap starts and stops * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * remove HTTPS prot flag in saphostagent * Update 5.5.3.1-SAPHanaSRMultiTarget.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update main.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * pacemaker fix * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update os-packages.yaml * Update os-packages.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * rhel pcs node attribute fix * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * replace availability variable replace db_high_availability with database_high_availability * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * post provision fix for scale out * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * Update ansible-input-api.yaml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * scale out cluster config overhaul * refactor, move scale out code seperate * scale out specific provision tasks * Update main.yml * Update main.yml * Update playbook_04_00_01_db_ha.yaml * Update 1.18.0-set_runtime_facts.yml * Update main.yml * Update main.yml * Update 5.5.2-pre_checks.yml * fix error on import vs include in 5.5 sub tasks * add majority maker constraints * post-provision code, minor code fixes for HANA ha config * Update 1.18.3-post_provision_report.yml * Update main.yaml * Update main.yaml * Update 1.18.3-post_provision_report.yml * Update 5.5.4.0-clusterPrep-ScaleOut-RedHat.yml * error code fix * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update 5.5.4.1-cluster-ScaleOut-RedHat.yml * Update main.yml * Update 5.5.3-SAPHanaSR.yml * changes to HANA Replication python hook configi * Update 20-saphana-rhel.j2 * Scale out code overhaul !!! * Update main.yaml * Update main.yaml * Update 5.8.4-provision-ScaleOut.yml * Update 5.8.4-provision-ScaleOut.yml

…g external NFS

…F Mount task

…ented out code

… code

…into scaleout-hsr

…ks output in anydb_node, kudmp disks and extensions in app_tier module

…into scaleout-hsr

* Fix the deployment using SPN * Ensure that custom_logical_volumes can be striped + have sensible stripesize default if a lvm consists of more than one pv. (#587) Use already established pattern from framework specific LVMs to define stripesize on custom logical volumes. * Perform az login before the az access token in ams provider creation steps * Revert "Perform az login before the az access token in ams provider creation" * Update AMS provider creation tasks in ansible playbook * Update deploy control plane pipeline to use service principal for authentication * chore: Temporarily set identity type to "SystemAssigned, UserAssigned" in app_service.tf * test for new /etc/hosts configuration for HSR scale out * Update hosts.j2 * Update hosts.j2 * Update hosts.j2 * Various Ansible fixes (#591) * Add Red Hat 8.10 repo to 1.3-repository vars * Create entries for Red Hat 8.10 in 1.4-packages vars * Add 'pam' to OS packages list for DB2 with state 'latest' Ensures that x86_64 package is updated, avoiding conflict with libpam.so.0 install, which requires i686 version of pam * Add "state: 'latest'" to loops * Correct cluster version check in 1.17-generic-pacemaker * Correct cluster version check in 5.6-scsers-pacemaker * Correct cluster version check in 5.5-hanadb-pacemaker * Create entries for Red Hat 8.10 in 1.17-generic-pacemaker --------- Co-authored-by: Csaba Daradics <[email protected]> * add code for scaleout - hook * fix to HSR code branch * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update 1.4.3-update-packages-RedHat.yaml * Update 1.4.3-update-packages-RedHat.yaml * chore: Adjust Azure fence agent packages and remove unused Azure Python packages from list when deploying on SLES 15 SP5 * chore: Override changed status for saptune_check and active_saptune_solution tasks in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Update New-SDAFDevopsWorkloadZone.ps1 script to fix variable group creation issue * Update main.yaml * Update playbook_04_00_01_db_ha.yaml * Update main.yaml * Update main.yaml * chore: Refactor saptune_solution_to_apply logic in 2.10.3.yaml * Set HDB schema name for ABAP and JAVA systems (#593) * Set HDB Schema Name task * fix command error and remove ignore_errors * Fix parsing error * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * Update main.yaml * chore: Comment out unnecessary role assignments in New-SDAFDevopsProject.ps1 script * Update main.yaml * exclude sapmnt from observer * bug fix, Scale out tasks not running on Majority maker node in playbook 4_01 * Update playbook_04_00_01_db_ha.yaml * replace pause with wait-for * replace pause with wait_for * enable AUTOMATED_REGISTER to true * Release testing (#597) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Rename quality check to quality assurance (#600) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * chore: Remove cacheable flag from 3.3 BoM Processing task * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * chore: move SAP on Azure quality checks after post configuration * chore: Update quality check paths to quality_assurance * chore: Update quality assurance file paths * chore: Refactor YAML files to improve code organization and readability * chore: Add cacheable flag to 3.3 BoM Processing task --------- Co-authored-by: dkSteBTh <[email protected]> * Release v3.11.0.3 (#599) * Bring in the manual updates * Fixing Merge conflicts * Fix Linting * Pacemaker changes, saptune and NAT Gateway (#583) ## Summary of Changes ### Authentication and Identity Management - **Web Application Authentication Configuration**: Repeated updates to refine and simplify the authentication configuration and identity management scripts. ### Repository and Package Management - **SLES Repositories**: Added repositories for SLES 15.3, 15.4, and 15.5. - **WAAgent Updates**: Updated WAAgent package, configuration checks, and systemd service reloads across multiple roles. ### Deployment Configuration - **NAT Gateway**: Added support for provisioning a NAT gateway, including configuration variables in Terraform files. - **AutoUpdate Configuration**: Updated AutoUpdate.Enabled settings and added Extensions.WaitForCloudInit across various roles. - **Oracle Simplification**: Simplified Oracle-related configurations, including ASM backup process and Data Guard tasks. - **SAP Deployment Playbooks**: Various updates to SAP deployment playbooks, including fixing conditions, resource flag settings, and systemd service paths. ### Infrastructure and Pipeline Enhancements - **Control Plane Pipeline**: Multiple fixes and improvements to error handling, logging, environment variables, and Azure AD authentication. - **Terraform and Ansible Versions**: Updated versions in deployment scripts to 1.7.5. - **Dotnet SDK**: Bumped dotnet SDK installation to the latest version. ### Miscellaneous - **Error Handling and Logging**: Improved error handling and logging across various deployment scripts and playbooks. - **Validation Fixes**: Fixed validation conditions for disk space, OS version checks, and cluster initialization commands. --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]> * Fix regex necessary to comment lines in /usr/sap/sapservices (#584) Co-authored-by: Csaba Daradics <[email protected]> * remove duplicate resource * Fix path * chore: Fix the count for the table resource * Misc fixes * Fix systemd service reload in 1.4 Packages role * Various Terraform code fixes (#586) * Fix typo in terraform-units/modules/sap_landscape/providers.tf * Remove duplicate of resource azurerm_network_security_rule/nsr_controlplane_storage In terraform-units/modules/sap_landscape/nsg.tf * Remove fourth argument from nat_gateway_name definition In terraform-units/modules/sap_landscape/variables_local.tf * Remove duplicate for database_kdump_disks In terraform-units/modules/sap_system/anydb_node/outputs.tf * Remove all duplicates from terraform-units/modules/sap_system/app_tier/vm-scs.tf * Remove duplicates in terraform-units/modules/sap_system/output_files/sap-parameters.tmpl --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Update count condition in dns.tf for local private DNS usage * chore: Update NAT Gateway public IP name format * chore: Update NAT Gateway public IP lifecycle configuration * chore: Update NAT Gateway provider to azureng * chore: Fix typo in azureng provider configuration alias in sap_landscape module * chore: Update NAT Gateway provider to azurerm.main * Update 01-deploy-control-plane.yaml * chore: Update app_service_plan name format in sap_deployer module * Update ARM_CLIENT_SECRET assignment in deploy control plane pipeline * Add the compliance extensions also to the deployers * Ensure that custom_logical_volumes can be striped + have sensible stripesize default if a lvm consists of more than one pv. (#587) Use already established pattern from framework specific LVMs to define stripesize on custom logical volumes. * Update AMS provider creation tasks in ansible playbook * Update deploy control plane pipeline to use service principal for authentication * chore: Temporarily set identity type to "SystemAssigned, UserAssigned" in app_service.tf * Various Ansible fixes (#591) * Add Red Hat 8.10 repo to 1.3-repository vars * Create entries for Red Hat 8.10 in 1.4-packages vars * Add 'pam' to OS packages list for DB2 with state 'latest' Ensures that x86_64 package is updated, avoiding conflict with libpam.so.0 install, which requires i686 version of pam * Add "state: 'latest'" to loops * Correct cluster version check in 1.17-generic-pacemaker * Correct cluster version check in 5.6-scsers-pacemaker * Correct cluster version check in 5.5-hanadb-pacemaker * Create entries for Red Hat 8.10 in 1.17-generic-pacemaker --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Adjust Azure fence agent packages and remove unused Azure Python packages from list when deploying on SLES 15 SP5 * chore: Override changed status for saptune_check and active_saptune_solution tasks in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Update New-SDAFDevopsWorkloadZone.ps1 script to fix variable group creation issue * chore: Refactor saptune_solution_to_apply logic in 2.10.3.yaml * Set HDB schema name for ABAP and JAVA systems (#593) * Set HDB Schema Name task * fix command error and remove ignore_errors * Fix parsing error * chore: Comment out unnecessary role assignments in New-SDAFDevopsProject.ps1 script * Release testing (#597) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Rename quality check to quality assurance (#600) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * chore: Remove cacheable flag from 3.3 BoM Processing task * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * chore: move SAP on Azure quality checks after post configuration * chore: Update quality check paths to quality_assurance * chore: Update quality assurance file paths * chore: Refactor YAML files to improve code organization and readability * chore: Add cacheable flag to 3.3 BoM Processing task --------- Co-authored-by: dkSteBTh <[email protected]> --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: daradicscsaba <[email protected]> Co-authored-by: Csaba Daradics <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: Nadeen Noaman <[email protected]> Co-authored-by: Steffen Bo Thomsen <[email protected]> Co-authored-by: Jesper Severinsen <[email protected]> * Bump Azure.Identity from 1.11.3 to 1.11.4 in /Webapp/SDAF (#594) Bumps [Azure.Identity](https://github.com/Azure/azure-sdk-for-net) from 1.11.3 to 1.11.4. - [Release notes](https://github.com/Azure/azure-sdk-for-net/releases) - [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Identity_1.11.3...Azure.Identity_1.11.4) --- updated-dependencies: - dependency-name: Azure.Identity dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Update os-packages.yaml for redhat8.6 * enable EPEL repositories for RedHat, enable unrar package to allow for newer database schemas * Update repos.yaml * restrict unrar to SCS tier only * Update bom_processor.yaml * Update os-packages.yaml * test "unar" no directory option to force files unpack to CD_EXPORT in flatten structure * experimental change to check for nested directories when running unar for Redhat on schema files * Update bom_processor.yaml * port RAR exe fix from windows to Linux 3.3 bom for testing * Update process_exe_archives.yaml * Update process_exe_archives.yaml * made a minor typo in path due to habitual CMD.exe user * Update process_exe_archives.yaml * Update process_exe_archives.yaml * Update process_exe_archives.yaml * Update process_exe_archives.yaml * chore: Create directories for SAP deployment automation * chore: Update authentication prompt for App Registration configuration * Update configuration_menu.sh * chore: Update PostBuildCleanup task to version 4 in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 for all stages in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 in deploy pipelines * chore: Update clusterPrep-RedHat.yml to avoid resource discovery during location constraints * Do not fail on saptune solution verify (#602) Set failed_when to false, so that saptune does not fail on N/A parameters. * chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists * Revert "chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists" This reverts commit e86dff14a149d8c866b2ce5b4570f2212959c062. * chore: Update accelerated networking configuration in Terraform modules, as enable_accelerated_networking is deprecated; new parameter is accelerated_networking_enabled * Ensure we are in the right context when getting access tokens and subsequently running the ps1 script, where we already have the trust setup for the SSH key. Not doing it this way, leads to either needing to manually create an SSH session inside pwsh with POSH-SSH to ensure the known_hosts entry is updated or having to update the quality check script upstream, to allow the -Force flag for the SSH session. (#603) * Fix for catching AHCO_INA_SERVICE delivery Unit import failure (#605) Co-authored-by: jasksingh <[email protected]> * Web App Component updates * Several (bug)fixes for RHEL deployments and deployments in general (#604) * Add fast_stop=no to pacemaker fileystem resources Pacemaker isn't respecting the stop timeout on filesystem resources due to the default setting fast_stop=yes. Without setting fencing will occur because if SAP (A)SCS / ERS isn't stopped in time processes will be terminated which are restarted by sapstartsrv and node will be fenced because fileystem can't be unmounted. https://www.suse.com/support/kb/doc/?id=000020860 https://access.redhat.com/solutions/4801371 * Distribute systemd services between SCS / ERS nodes and stop services Both (A)SCS and ERS systemd services should be present on SCS and ERS nodes otherwise pacemaker only handles SCS on the SCS node and ERS on the ERS node with the systemd integration. * Add resource clear for move contrainsts on (A)SCS resource group * Bugfix folders on local disks to be managed after mounting local disk * sid_private_key isn't required * Add sdu_secret prefix/suffix to manage custom Key Vault secret naming * Update Web App to support NAT Gateway * Hotfix release after testing with new RHEL image (#611) * update: added fixed encountered during RHEL94 testing * chore: Update Red Hat and SLES package versions for Red Hat 9.4 * update: add network rules to deployer diagnostic storage account * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Add ${SID}_virtual_machines.json to git if it exists * chore: Update sap_system module to use database_server_vm_resource_ids for database server VMs * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to use comma-separated database server VM resource IDs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap_system module to include application server VM resource IDs * chore: Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor use_spn variable to default to false for all SAP deployment modules * Refactor systemd process limit configuration for pacemaker cluster * Refactor systemd process limit configuration for pacemaker cluster * Update os-packages.yaml (#613) * chore: Refactor Azure Fencing Agent creation in 1.17 Generic Pacemaker role (#614) * Add SAP-CAL Integration for non-HA Installation (#608) * Add AVG support for Scale out scenarios (#577) * Add data and log volumes * Refactor AVG logic * Fix proximity_placement_group_id calculation in avg.tf * Refactor for_each condition in avg.tf * Refactor for_each condition in avg.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor volume creation logic in variables_local.tf * Refactor zone calculation logic in variables_local.tf * Refactor proximity_placement_group_id calculation in avg.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in vm-app.tf * Add dependency on azurerm_virtual_machine_data_disk_attachment.scs in infrastructure.tf * Refactor package update condition in 1.4.3-update-packages-RedHat.yaml --------- Co-authored-by: Kimmo Forss <[email protected]> * Update subnet_cidr_storage in sap-parameters.tmpl * Update hosts jinja for client subnet * Update SAP-specific configuration playbook for HANA database scale-out scenario * Version update * Simplify Web App Identity management * Update Azure package versions in SDAFWebApp.csproj * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script * Add SLES 15.3, 15.4, and 15.5 repositories * Update Web Application authentication configuration script and simplify Web App Identity management * Refactor Web App Identity management and update authentication configuration script * Update Web Application authentication configuration script * Update Web Application authentication configuration script and simplify Web App Identity management * Commented out SSH trust relationship checks in 1.17.2-provision.yml * Revert "Commented out SSH trust relationship checks in 1.17.2-provision.yml" This reverts commit 09cd30de6003a891b5c8c31b4c96b495b676aa9b. * ACSS updates * Oracle simplification * Add AutoUpdate.Enabled configuration in 1.1-swap role and enable package cache update in 1.4-packages role * Update deployment type configuration in OS and SAP specific playbooks * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Update WAAgent package and restart service in 1.1-swap role * Updated key_vault_sap_landscape.tf * Revert "Updated key_vault_sap_landscape.tf" * Update WAAgent package and restart service in 1.1-swap role * Add SAP CAL Integration * Update AutoUpdate.Enabled configuration in 1.1-swap role and add Extensions.WaitForCloudInit configuration * Revert "Add SAP CAL Integration" This reverts commit adae6662ba478d9f1d4d0de7f5c175e4f5da739b. * Update WAAgent package and restart service in 1.4-packages role * Update waagent configuration check in 1.4-packages role * Update waagent configuration check and systemd service reload in 1.4-packages role * Update AutoUpdate.Enabled configuration and add Extensions.WaitForCloudInit configuration in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update waagent configuration check and systemd service reload in 1.1-swap role * Update database_high_availability condition in playbook_04_00_01_db_ha.yaml * Add the ability to block app registration * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update web_instance_number and add web_sid variable in sap_system/transform.tf * Fix validation error message for web dispatcher sid in variables_global.tf * Remove chkconfig package from os-packages.yaml * Update systemd service file path in 5.6.7-config-systemd-sap-start.yml * Update OS version check for RHEL 8.2 and SLES 15 in 5.6.1-set_runtime_facts.yml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update Oracle ASM backup process and fix file permissions * Fix file path in 1.4.0-packages-RedHat-prep.yaml * Update OS version check for RHEL 9.0 or newer in 1.4.0-packages-RedHat-prep.yaml * Update file path and preserve file permissions in 1.4.0-packages-RedHat-prep.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Fix action values in playbook_04_00_01_db_ha.yaml and roles-db/4.1.3-ora-dg/tasks/main.yaml * Update wait time for StartService in 5.6 SCS/ERS Validation * Update Terraform version to 1.8.0 in deployment scripts and tfvar_variables.tf files * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Fix missing else statement in deploy control plane pipeline * Update virtual machine extension reference in vm.tf * Update virtual machine extension version to 1.0 in vm.tf * Fix missing else statement in deploy control plane pipeline * Update network interface and virtual machine counts in vm-observer.tf * Update database high availability configuration * Update use_spn property to false in LandscapeModel and SystemModel * Update Terraform and Ansible versions to 1.7.5 in deployment scripts and variables * Update Display value in SystemDetails.json * Fix validation condition in variables_global.tf * Add ORACLE Post Processing: Reboot after Enabling HugePages task * Fix typo in Oracle Data Guard - Observer: Change UID for Oracle user task * install passlib * Add patch_mode support * Update deployment playbook to set single_server fact based on host count * Update patch_mode configuration in Terraform files * Update file permissions in SAP deployment playbook * Update deployment playbooks to set single_server fact consistently * Fix waagent configuration in swap role * Fix indentation in swap role tasks/main.yaml * Fix cluster group move command in 5.6 SCS/ERS Validation playbook * Fix condition in 1.17-generic-pacemaker playbook to exclude node_tier 'hana' * Fix commented out corosync configuration in 1.17-generic-pacemaker playbook * Create the SID subfolder * Update verbosity level in 5.6.7-config-systemd-sap-start.yml * Add passlib * Simplify Python logic * Update app_bom_id variable in 5.3-app-install/tasks/main.yaml * Update passlib installation in Ansible playbooks * Update reboot timeout and post-reboot delay in 5.6.4.2-sap-resources-Suse.yml * Update swap role and package tasks * Fix condition in 1.17-generic-pacemaker playbook to exclude node_tier 'hana' * Fix failed_when condition in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * enable corosync and pacemaker on Suse * change from command to shell * Update verbosity level for debug message in 5.6.4.0-cluster-Suse.yml * Refactor command to shell in 5.6-scsers-pacemaker tasks * Refactor command to shell in 5.6-scsers-pacemaker tasks * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 5.6-scsers-pacemaker tasks * Refactor path in ora-dg-observer-setup.yaml to include sap_sid variable * Refactor cluster initialization commands in 5.6-scsers-pacemaker tasks and add SAP component installation check * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook * Refactor cluster initialization commands in 1.17-generic-pacemaker playbook and 5.6-scsers-pacemaker tasks * add missing quotes * Fix disk space validation in playbook_00_validate_parameters.yaml * Refactor SAP resource flag setting in Ansible playbooks * Refactor SAP component installation check in 5.6-scsers-pacemaker tasks * Refactor SAP resources installed message in 5.6-scsers-pacemaker tasks * Refactor SCS/ERS validation tasks in 5.6-scsers-pacemaker playbook * Refactor SAP resource flag setting in Ansible playbooks * Refactor ORACLE: Find MOPatch tasks in 4.1.0-ora-install playbook * support for pools with auto qos * support for pools with auto qos * support for pools with auto qos * provide a way to override the oracle user * Update Web Application Configuration documentation * Fix default value for SAP_installed in 5.6-scsers-pacemaker tasks * Fix default value for SAP_installed in 5.6-scsers-pacemaker tasks * Fix shell command in 5.6-scsers-pacemaker pre_checks.yml * Passwordless Web App * Passwordless * Update variable group creation in New-SDAFDevopsProject.ps1 script * Fix client_id reference in app_service.tf * Update packages * Update Web Application Configuration to use resource group scope for role assignments * Update Web Application Configuration documentation * Fix target_nodes value in 2.6.1-anf-mounts.yaml * Web App updates * Update enable_db_lb_deployment logic in variables_local.tf * Bump up the dotnet version * Remove PAT * Remove PAT * Fix TF_VAR_agent_pat assignment in deploy control plane pipeline * Fix PAT assignment in deploy control plane pipeline * Update TF_VAR_agent_pool assignment in deploy control plane pipeline * Add MSI registration * Fix typo * Update versionLabel to v3.11.0.2 in New-SDAFDevopsProject.ps1 * Fix typo in New-SDAFDevopsProject.ps1 + add PAT back for Control Plane * Update ANF mount paths in 2.6.1-anf-mounts.yaml * Fix PostBuildCleanup task in deploy control plane pipeline * Update PostBuildCleanup task to version 4 in deploy control plane pipeline * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update DEPLOYER folder and file validations in deploy control plane pipeline * Update deploy control plane pipeline with environment and location information * Update deploy control plane pipeline with Deployer TFvars variable * Update deploy control plane pipeline with Library TFvars variable * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update installer.sh to display parameter file and current directory * Update deploy control plane pipeline with Library and Deployer TFvars variables * Update SAP_AUTOMATION_REPO_PATH assignment in deploy control plane pipeline * Update PostBuildCleanup task to version 3 in deploy control plane pipeline * Update dotnet-sdk installation in configure_deployer.sh.tmpl * Update deploy control plane pipeline with TF_VAR_agent_pat variable * Update deploy control plane pipeline with Azure CLI version display * Update deploy control plane pipeline with Workload TFvars variable * Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * Update deploy/ansible/roles-db/4.1.3-ora-dg/tasks/ora-dg-setup-secondary.yaml to remove the 'recurse' option in the ansible.builtin.file task * Update deploy/ansible/roles-db/4.1.3-ora-dg/tasks/ora-dg-setup-secondary.yaml to fix failed_when condition in rman restore tasks * chore: Update app_service.tf to add WHICH_ENV variable * Update app_service.tf to allow specific app registrations * chore: Update NuGet.Packaging dependency to version 6.9.1 * chore: Update app_service.tf to remove unused app setting and add WHICH_ENV variable * chore: Update deploy control plane pipeline with removal of AZURE_DEVOPS_EXT_PAT environment variable * chore: Update AFS Mount task to exclude 'app' node tier * chore: Update hosts.j2 template to exclude virtual hosts for non-high availability scenarios * chore: Update New-SDAFDevopsProject.ps1 to improve App Registration creation process * Change the ID to add * chore: Update New-SDAFDevopsProject.ps1 to improve App Registration creation process * Add SAP-CAL Integration * Linting * chore: Update deploy control plane pipeline with necessary environment variables * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use idToken for ARM_CLIENT_SECRET * chore: Update deploy control plane pipeline to use System.AccessToken for AZURE_DEVOPS_EXT_PAT * chore: Update deploy control plane pipeline to remove unused agent pool check * chore: Remove unused agent pool check in deploy control plane pipeline * chore: Update deploy control plane pipeline to use $(PAT) for AZURE_DEVOPS_EXT_PAT * changes to ERS group * chore: Update deploy control plane pipeline to improve error handling and logging * chore: Update deploy control plane pipeline to enable Azure AD authentication * chore: Update deploy control plane pipeline to extract deployer_random_id from environment file * chore: Improve error handling and logging in deploy control plane pipeline * chore: Update deploy control plane pipeline to extract deployer_random_id from environment file * chore: Update deploy control plane pipeline to create variable group variables for key vault, terraform remote storage subscription, and deployer random ID seed * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to fix typo in ARM_USE_AZUREAD variable * chore: Update deploy control plane pipeline to use $(PAT) instead of $(System.AccessToken) for AZURE_DEVOPS_EXT_PAT * chore: Update deploy control plane pipeline to improve error handling and logging * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update deploy control plane pipeline to remove unnecessary Azure login * chore: Update bootstrap flag to false in sap_library module * chore: Update storage account network rules for tfstate and sapbits * chore: Update dotnet-sdk installation to version 8.0 * chore: Update dotnet-sdk installation to latest version * chore: Update HttpClient usage in RestHelper.cs and Azure SDK versions in SDAFWebApp.csproj * chore: Update random_id_b64 format in output.tf files * chore: Update RestHelper.cs to accept a type parameter in the constructor * chore: Ignore changes to app_settings in azurerm_windows_web_app resource * chore: Update random_id_b64 format in output.tf files * chore: Update RestHelper.cs to use HttpClient instead of HttpClientGH * chore: Add Build Service user to Build Administrators group * Add the ability to authenticate using PAT * chore: Update RestHelper.cs to use HttpClient instead of HttpClientGH * Update on devops login * chore: Update New-SDAFDevopsProject.ps1 to use tsv output for project creation * chore: Refactor RestHelper.cs to use HttpClient and support PAT authentication * Change module name * update: SAP ASCS/SCS/ERS start resources configuration for SUSE - ENSA1 and ENSA2 when using simple mount. This commit updates the configuration of SAP ASCS/SCS/ERS start resources for SUSE - ENSA1 and ENSA2. * chore: Update SAP Directories creation in ansible playbook This commit updates the ansible playbook to create SAP Directories. It modifies the tasks to create the directories "/usr/sap/trans" and "/sapmnt/{{ sap_sid | upper }}". These changes improve the handling of SAP Transport Filesystems in the deployment process. * feat: Add additional destination port ranges for NSG rules This commit updates the NSG rules in the `sap_landscape` module to include additional destination port ranges. The destination port ranges for the `nsr_controlplane_app`, `nsr_controlplane_web`, `nsr_controlplane_storage`, `nsr_controlplane_db`, and `nsr_controlplane_admin` rules have been expanded to include ports 2049 and 111. * Update error message * Update SAP ASCS/SCS/ERS start resources configuration for SUSE - ENSA1 and ENSA2 when using simple mount. * Add the MSI to the project * Added debug statement to playbook_sapcal_integration.yaml * Added debug statement to playbook_sapcal_integration.yaml * Revert "Added debug statement to playbook_sapcal_integration.yaml" This reverts commit 839170ef4c76cc1b50a020e4ca3d5d3b1b20b932. * Revert "Added debug statement to playbook_sapcal_integration.yaml" This reverts commit 5170d0b0eaa69964306c16541568bf5325403345. * Skip all BOM related tasks if enable_sap_cal is true * Updated the variable name for consistency * Ensured tasks run with appropriate privileges * Store SAP-CAL API response/file in the repository * Lint code and set default values * Use a secure tempfile --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: Devansh Jain <[email protected]> * Allow for splitting out the privatelink resources (#616) * Add the ability to split out the privatelink resources * feat: Add privatelinkdnsmanagement provider configuration * refactor: Update storage_accounts.tf to use var.dns_settings.dns_zone_names.table_dns_zone_name * refactor: Update DNS zone names in dns.tf and storage_accounts.tf * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names.table_dns_zone_name * refactor: Update DNS zone names in infrastructure.tf, key_vault.tf, and keyvault_endpoint.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in keyvault_endpoint.tf and storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update count condition in dns.tf to use local.use_local_privatelink_dns instead of negation of it * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf and sap_deployer/tfvar_variables.tf to use var.dns_settings.dns_zone_names * Add the ability to split out DNS records for privatelink resources * refactor: Update DNS zone names to use var.dns_settings.dns_zone_names * refactor: Add privatelink DNS resource group and subscription properties to LandscapeModel * refactor: Update DNS zone names in infrastructure.tf, key_vault.tf, and keyvault_endpoint.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in LandscapeDetails.json, storage_accounts.tf, infrastructure.tf, and transform.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in transform.tf to use var.dns_settings.dns_zone_names * refactor: Update DNS zone names in storage_accounts.tf to use var.dns_settings.dns_zone_names * Add register_virtual_network_to_dns attribute * Add the ability to control the patch mode * add vm_agent_platform_updates_enabled * refactor: Remove patch_mode from vm-scs.tf * refactor: Remove patch_mode from vm-anchor.tf * Add auto update of the extensions * refactor: Tweak the Windows patch mode * Windows update settings * Debug show SystemD version * refactor: Update SystemD version debug message in 1.17 Generic Pacemaker role * refactor: Update VM patch information in SystemModel and LandscapeModel * refactor: Update Process limit configuration in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for pacemaker version in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for systemd version in 1.17 Generic Pacemaker role * refactor: Update process limit configuration for systemd version in 1.17 Generic Pacemaker role * Remove the white space * fix: Associate the iSCSI subnet with the route table * refactor: Add python3-pip package for different node tiers in HA setup * refactor: remove the lower pipe from distro name * refactor: Split out OracleLinux tasks * refactor: Update iSCSI subnet association with route table * chore: Update NuGet.Packaging dependency to version 6.11.0 * TEswt if we can handle no read access scenarios to key vault * revert casing * refactor: Split out OracleLinux tasks * chore: Add condition to include custom repositories in 1.3 Repository tasks * refactor: Update 1.3 Repository tasks to include custom repositories for SUSE and RedHat * refactor: Remove unnecessary OracleLinux tasks and custom repositories * refactor: Update VM deployment configuration * Remove the token check * refactor: Add TF_VAR_agent_pat to control plane deployment pipeline * refactor: Fix private DNS zone ID in keyvault_endpoint.tf * Web App and version updates * Restore patch_mode * Web App updates * chore: Add System.Data.SqlClient package reference * refactor: Update 1.3 Repository tasks to include custom repositories for SUSE and RedHat * refactor: Update tfvar_variables.tf with new variables for tfstate storage account and deployer's tfstate file * Remove some of the python packages * Remove unnecessary python packages * refactor: Remove trailing spaces in LandscapeDetails.json and SystemDetails.json * refactor: Remove trailing spaces in LandscapeDetails.json and SystemDetails.json * Fix reboot on RHEL * refactor: Fix typo in DBLoad task names * refactor: Update cluster resource monitor intervals to 20 seconds * LINT fixes --------- Co-authored-by: Kimmo Forss <[email protected]> * Extend 'root' and 'tmp' logical volumes for SAP CAL integration on RHEL (#618) * Extend 'root' and 'tmp' logical volumes for SAP CAL integration on RHEL * chore: Set pipefail and Code Linting * feat: Add ability to split out privatelink resources * feat: Refactor subnet configuration to enforce private link endpoint network policies Refactor the subnet configuration in the `sap_landscape` module to enforce private link endpoint network policies. This change ensures that the private link endpoints have network policies enabled, as specified by the `var.use_private_endpoint` variable. Co-authored-by: Kimmo Forss <[email protected]> * fix: update the management dns subscription id to saplib sub id, pin azurerm version in lanscape, deployer (#619) * pin azurerm version in deployer and landscape * Extend 'root' and 'tmp' logical volumes for SAP CAL integration on RHEL * chore: Extend 'root' and 'tmp' logical volumes for SAP CAL integration on RHEL * Add iSCSI NSG rule * Change rule name * Add Expiry to workload zone key vault secrets * Provide a more robust way to source the deployer subscription * Add support for secret expiry * chore: Update keyvault set-policy command in sap-workload-zone.yaml Co-authored-by: Kimmo Forss <[email protected]> * feat: Add expiry date to secrets in key vault * chore: Disable cross-tenant replication for sapmnt storage account * chore: Update DNS configuration for sap_library module * chore: Update DNS configuration for sap_library module * chore: Update use_local_privatelink_dns condition in sap_library module * chore: Update DNS configuration for sap_library module * chore: Update private DNS configuration in sap_library module * Don't create route table if Firewall is used * chore: Update key_vault.tf for sap_library module * chore: Update private DNS configuration in sap_library module * chore: Update private endpoint configuration for sapmnt storage account * Bump up the TF version * Also add the DB Virtual Hostname * chore: Update private endpoint configuration for sapmnt storage account * chore: Update default value for "use_private_endpoint" to true * chore: remove extra line from 2.10.3.yaml * Update github-actions-ansible-lint.yml Update lint version * chore: Update yum to dnf for Oracle RPM package installation * chore: Update secret expiry date for SAP cluster and system passwords * chore: Update default value for "use_private_endpoint" to true * chore: Update secret expiry date for SAP cluster and system passwords * chore: Update secret expiry date format for SAP cluster and system passwords * chore: Update resource group name for private DNS zone virtual network link * chore: Update secret expiry date format for SAP cluster and system passwords * Set Expiry for all key vault secrets * chore: Fix typo * Feature: Add support for SLES 15 SP6 * chore: Remove redundant code for adding HA repositories for RHEL * Configure constraint HANA ANF mounts only when HANA mountpoints are used (#625) When using ANF for the deployment without data, log and shared mountpoint on ANF the constraint shouldn't be configured. * AND constraint logic * Add support for TGZ files * Fix the OR statement * chore: Update verbosity level for debugging BOM object creation * chore: Update reboot timeout and post-reboot delay for 5.6 SCSERS - RHEL instance * Run reboot as root * chore: Expand volumes for Red Hat OS family * chore: Update Test-SDAFReadiness.ps1 script * chore: Update PostBuildCleanup task version to 4 * chore: Update privatelink DNS configuration in transform.tf * Expand logical volumes and resize file systems * Expand logical volumes and resize file systems * Fix Linting * chore: update the check for the free size in VG * Expand logical volumes and resize file systems * chore: Update expand-volumes task to handle default values for sufficient_vg_space and sufficient_vg_space_db * Chore: Fix typo * chore: Clear the failed state of hosts during database installation playbook * Update free space check for SAPCAL * Update ANF_sapmnt variable description and default value * Update ANF_sapmnt variable description and default value * HSR changes * chore: Update Azure.ResourceManager package to version 1.13.0 * Update backup commands and capture backup results * Update backup commands and capture backup results * Update backup commands and capture backup results * Update backup commands to capture backup results * Update backup commands to capture backup results * V3.12.0.0 (#624) * Bring in the manual updates * Fixing Merge conflicts * Fix Linting * Pacemaker changes, saptune and NAT Gateway (#583) ## Summary of Changes ### Authentication and Identity Management - **Web Application Authentication Configuration**: Repeated updates to refine and simplify the authentication configuration and identity management scripts. ### Repository and Package Management - **SLES Repositories**: Added repositories for SLES 15.3, 15.4, and 15.5. - **WAAgent Updates**: Updated WAAgent package, configuration checks, and systemd service reloads across multiple roles. ### Deployment Configuration - **NAT Gateway**: Added support for provisioning a NAT gateway, including configuration variables in Terraform files. - **AutoUpdate Configuration**: Updated AutoUpdate.Enabled settings and added Extensions.WaitForCloudInit across various roles. - **Oracle Simplification**: Simplified Oracle-related configurations, including ASM backup process and Data Guard tasks. - **SAP Deployment Playbooks**: Various updates to SAP deployment playbooks, including fixing conditions, resource flag settings, and systemd service paths. ### Infrastructure and Pipeline Enhancements - **Control Plane Pipeline**: Multiple fixes and improvements to error handling, logging, environment variables, and Azure AD authentication. - **Terraform and Ansible Versions**: Updated versions in deployment scripts to 1.7.5. - **Dotnet SDK**: Bumped dotnet SDK installation to the latest version. ### Miscellaneous - **Error Handling and Logging**: Improved error handling and logging across various deployment scripts and playbooks. - **Validation Fixes**: Fixed validation conditions for disk space, OS version checks, and cluster initialization commands. --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: devanshjain <[email protected]> * Fix regex necessary to comment lines in /usr/sap/sapservices (#584) Co-authored-by: Csaba Daradics <[email protected]> * remove duplicate resource * Fix path * chore: Fix the count for the table resource * Misc fixes * Fix systemd service reload in 1.4 Packages role * Various Terraform code fixes (#586) * Fix typo in terraform-units/modules/sap_landscape/providers.tf * Remove duplicate of resource azurerm_network_security_rule/nsr_controlplane_storage In terraform-units/modules/sap_landscape/nsg.tf * Remove fourth argument from nat_gateway_name definition In terraform-units/modules/sap_landscape/variables_local.tf * Remove duplicate for database_kdump_disks In terraform-units/modules/sap_system/anydb_node/outputs.tf * Remove all duplicates from terraform-units/modules/sap_system/app_tier/vm-scs.tf * Remove duplicates in terraform-units/modules/sap_system/output_files/sap-parameters.tmpl --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Update count condition in dns.tf for local private DNS usage * chore: Update NAT Gateway public IP name format * chore: Update NAT Gateway public IP lifecycle configuration * chore: Update NAT Gateway provider to azureng * chore: Fix typo in azureng provider configuration alias in sap_landscape module * chore: Update NAT Gateway provider to azurerm.main * Update 01-deploy-control-plane.yaml * chore: Update app_service_plan name format in sap_deployer module * Update ARM_CLIENT_SECRET assignment in deploy control plane pipeline * Add the compliance extensions also to the deployers * Ensure that custom_logical_volumes can be striped + have sensible stripesize default if a lvm consists of more than one pv. (#587) Use already established pattern from framework specific LVMs to define stripesize on custom logical volumes. * Update AMS provider creation tasks in ansible playbook * Update deploy control plane pipeline to use service principal for authentication * chore: Temporarily set identity type to "SystemAssigned, UserAssigned" in app_service.tf * Various Ansible fixes (#591) * Add Red Hat 8.10 repo to 1.3-repository vars * Create entries for Red Hat 8.10 in 1.4-packages vars * Add 'pam' to OS packages list for DB2 with state 'latest' Ensures that x86_64 package is updated, avoiding conflict with libpam.so.0 install, which requires i686 version of pam * Add "state: 'latest'" to loops * Correct cluster version check in 1.17-generic-pacemaker * Correct cluster version check in 5.6-scsers-pacemaker * Correct cluster version check in 5.5-hanadb-pacemaker * Create entries for Red Hat 8.10 in 1.17-generic-pacemaker --------- Co-authored-by: Csaba Daradics <[email protected]> * chore: Adjust Azure fence agent packages and remove unused Azure Python packages from list when deploying on SLES 15 SP5 * chore: Override changed status for saptune_check and active_saptune_solution tasks in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Add condition to check if saptune_solution_enabled is defined in 2.10.3.yaml * chore: Update New-SDAFDevopsWorkloadZone.ps1 script to fix variable group creation issue * chore: Refactor saptune_solution_to_apply logic in 2.10.3.yaml * Set HDB schema name for ABAP and JAVA systems (#593) * Set HDB Schema Name task * fix command error and remove ignore_errors * Fix parsing error * chore: Comment out unnecessary role assignments in New-SDAFDevopsProject.ps1 script * Release testing (#597) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Rename quality check to quality assurance (#600) * script from main branch * chore: Add "nat_gateway" variable to global variables in sap_namegenerator module * chore: Update bom-register.yaml to use the correct path for the Microsoft supplied BOM archive * chore: Add debug task to bom-register.yaml for Microsoft supplied BOM archive * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Remove unnecessary code for extra parameters in DB and SAP installation pipeline * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * Refactor az logout command in sap-workload-zone.yaml * Refactor SSH command in 1.17 Generic Pacemaker provision playbook * chore: Remove cacheable flag from 3.3 BoM Processing task * Refactor bom-register.yaml to remove debug task and include correct path for Microsoft supplied BOM archive * Add SAP on Azure quality chekcs feature to the 05-DB-and-SAP-installation.yaml pipeline. * remove duplicate block * remove blank line * add mode to get_url downloaded file. * remove blank line from start of file. * Refactor sap-system-deployment.yaml to configure devops CLI extension * Refactor sap-system-deployment.yaml to configure devops CLI extension * chore: Update SDAF version to 3.11.0.3 * Refactor az logout command in sap-workload-zone.yaml * chore: move SAP on Azure quality checks after post configuration * chore: Update quality check paths to quality_assurance * chore: Update quality assurance file paths * chore: Refactor YAML files to improve code organization and readability * chore: Add cacheable flag to 3.3 BoM Processing task --------- Co-authored-by: dkSteBTh <[email protected]> * chore: Update os-packages.yaml for redhat8.6 * chore: Create directories for SAP deployment automation * chore: Update authentication prompt for App Registration configuration * chore: Update PostBuildCleanup task to version 4 in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 for all stages in 01-deploy-control-plane.yaml * chore: Update PostBuildCleanup task to version 4 in deploy pipelines * chore: Update clusterPrep-RedHat.yml to avoid resource discovery during location constraints * Do not fail on saptune solution verify (#602) Set failed_when to false, so that saptune does not fail on N/A parameters. * chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists * Revert "chore: Update New-SDAFDevopsProject.ps1 to use tsv output format for subscription and identity lists" This reverts commit e86dff14a149d8c866b2ce5b4570f2212959c062. * chore: Update accelerated networking configuration in Terraform modules, as enable_accelerated_networking is deprecated; new parameter is accelerated_networking_enabled * Ensure we are in the right context when getting access tokens and subsequently running the ps1 script, where we already have the trust setup for the SSH key. Not doing it this way, leads to either needing to manually create an SSH session inside pwsh with POSH-SSH to ensure the known_hosts entry is updated or having to update the quality check script upstream, to allow the -Force flag for the SSH session. (#603) * Fix for catching AHCO_INA_SERVICE delivery Unit import failure (#605) Co-authored-by: jasksingh <[email protected]> * Web App Component updates * Several (bug)fixes for RHEL deployments and deployments in general (#604) * Add fast_stop=no to pacemaker fileystem resources Pacemaker isn't respecting the stop timeout on filesystem resources due to the default setting fast_stop=yes. Without setting fencing will occur because if SAP (A)SCS / ERS isn't stopped in time processes will be terminated which are restarted by sapstartsrv and node will be fenced because fileystem can't be unmounted. https://www.suse.com/support/kb/doc/?id=000020860 https://access.redhat.com/solutions/4801371 * Distribute systemd services between SCS / ERS nodes and stop services Both (A)SCS and ERS systemd services should be present on SCS and ERS nodes otherwise pacemaker only handles SCS on the SCS node and ERS on the ERS node with the systemd integration. * Add resource clear for move contrainsts on (A)SCS resource group * Bugfix folders on local disks to be managed after mounting local disk * sid_private_key isn't required * Add sdu_secret prefix/suffix to manage custom Key Vault secret naming * Update Web App to support NAT Gateway * Hotfix release after testing with new RHEL image (#611) * update: added fixed encountered during RHEL94 testing * chore: Update Red Hat and SLES package versions for Red Hat 9.4 * update: add network rules to deployer diagnostic storage account * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Add ${SID}_virtual_machines.json to git if it exists * chore: Update sap_system module to use database_server_vm_resource_ids for database server VMs * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to use comma-separated database server VM resource IDs * chore: Update sap-vm-resources.tmpl to include additional server information * chore: Update sap_system module to include empty lists for SCS, application, and webdisp server VMs * chore: Update sap_system module to include application server VM resource IDs * chore: Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' for executing Azure CLI commands * Refactor cluster_group_location task in ACSS registration role * Refactor cluster_group_location task in ACSS registration role * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'ansible.builtin.shell' for executing Azure CLI commands * Refactor ACSS registration role to use 'ansible.builtin.command' instead of 'a…

Bring in commits from experimental --------- Co-authored-by: Steffen Bo Thomsen <[email protected]> Co-authored-by: devanshjain <[email protected]> Co-authored-by: Nadeen Noaman <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: daradicscsaba <[email protected]> Co-authored-by: Csaba Daradics <[email protected]> Co-authored-by: Jesper Severinsen <[email protected]> Co-authored-by: Devansh Jain <[email protected]> Co-authored-by: Jaskirat Singh <[email protected]> Co-authored-by: jasksingh <[email protected]> Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: Harm Jan Stam <[email protected]> Co-authored-by: Kimmo Forss <[email protected]>

* Refactor ANF volume group creation in HDB node module * Update provider configurations to use version 4.0 or higher * Refactor ANF volume group creation in HDB node module and update provider configurations * chore: replace db_scale_out with database_scale_out * Bump up the version * Add User creation * Set permissions on /sapmnt * Fix capitalization of source in providers.tf * Refactor NAT Gateway resource properties in outputs.tf * Refactor azapi provider source in providers.tf * Refactor private endpoint network policies in subnet resources * For scaleout use the admin subnet ID * TF 4.0 support * For scale out make the admin nic the primary * Refactor Variable class to remove nullable value property Update Azure.ResourceManager.Network package to version 1.9.0 * Refactor inventory.tf to support scale-out for SAP system * Add site information * Refactor inventory.tf to include scale_out variable * Refactor vm-hdb.tf to include site information for scale-out databases * Refactor HANA computer and secondary DNS names for scale-out databases --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]>

commit 0961f18 Author: Kimmo Forss <[email protected]> Date: Sat Sep 14 14:34:48 2024 +0300 Refactor VM name generation for scale-out databases with zonal markers Don 't remove hana shared from fstab Refactor ANF mount logic for HANA shared volume based on site configuration Refactor inventory.tf to include scale_out variable Update provider configurations to use version 4.0 or higher Refactor ANF volume group creation in HDB node module

* Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module and update provider configurations * Update provider configurations * Update provider configurations to use version 4.0 or higher * chore: replace db_scale_out with database_scale_out * Bump up the version * Refactor NAT Gateway resource properties in outputs.tf * Refactor azapi provider source in providers.tf * Refactor address space concatenation in NSG rule * For scaleout use the admin subnet ID * TF 4.0 support * Refactor provider configuration for SAP library * For scale out make the admin nic the primary * Update Azure.ResourceManager.Network package to version 1.9.0 * Refactor inventory.tf to support scale-out for SAP system * Add site information * Refactor vm-hdb.tf to include site information for scale-out databases * Refactor HANA shared volume output for improved zonal ordering * Refactor VM name generation for scale-out databases with zonal markers * Refactor ANF mount logic for HANA shared volume based on site configuration * Don 't remove hana shared from fstab * Refactor SAP user and group creation for generic installation --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]>

* Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module * Refactor ANF volume group creation in HDB node module and update provider configurations * Refactor ANF volume group creation in HDB node module and add dependencies * Refactor ANF volume group creation in HDB node module and update provider configurations * Update provider configurations * Update provider configurations to use version 4.0 or higher * Refactor ANF volume group creation in HDB node module and update provider configurations * chore: replace db_scale_out with database_scale_out * Bump up the version * Add User creation * Set permissions on /sapmnt * Fix capitalization of source in providers.tf * Refactor NAT Gateway resource properties in outputs.tf * Refactor azapi provider source in providers.tf * Refactor private endpoint network policies in subnet resources * Refactor private endpoint network policies in subnet resources * Refactor private endpoint network policies in subnet resources * Refactor address space concatenation in NSG rule * Refactor address space concatenation in NSG rule * Refactor address space concatenation in NSG rule * Refactor address space concatenation in NSG rule * For scaleout use the admin subnet ID * TF 4.0 support * Refactor SAP user and group creation for generic installation * More verbose debugging --------- Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]>

* Refactor ANF volume group creation in HDB node module and update provider configurations * chore: replace db_scale_out with database_scale_out * Bump up the version * Add User creation * Set permissions on /sapmnt * Fix capitalization of source in providers.tf * Refactor NAT Gateway resource properties in outputs.tf * Refactor azapi provider source in providers.tf * Refactor private endpoint network policies in subnet resources * Refactor address space concatenation in NSG rule * For scaleout use the admin subnet ID * TF 4.0 support * Refactor provider configuration for SAP library * For scale out make the admin nic the primary * Refactor Variable class to remove nullable value property Update Azure.ResourceManager.Network package to version 1.9.0 * Refactor inventory.tf to support scale-out for SAP system * Add site information variable * Refactor HANA and secondary DNS names for scale-out databases * Refactor HANA shared volume output for improved zonal ordering * Refactor VM name generation for scale-out databases with zonal markers * Refactor ANF mount logic for HANA shared volume based on site configuration * Don 't remove hana shared from fstab * Refactor VM name generation for scale-out databases with zonal markers * Linting fixes * More verbose debugging * Move observer to client NIC * Misc fixes * Jinja updates * Update main.yaml - correct subnet entry for [system_replication_hostname_resolution] to point to HSR subnet for scale out - HSR based deployment * Update 5.5.4.0-clusterPrep-RedHat.yml * fix for HANA scale out - HSR not correctly populating FS resource params * Update 5.8.4.0-clusterPrep-ScaleOut-RedHat.yml removing bind in NFS options as its causing mount failure * Start HANA using sapcontrol commands * Fix HANA HSR start command in clusterPrep-ScaleOut-RedHat.yml * Task naming * Update HANA HSR start command in clusterPrep-ScaleOut-RedHat.yml * Refactor HANA Pacemaker: Remove unused code and update helper variables * chore: Update environment variables in provision-ScaleOut.yml * Refactor HANA Pacemaker: Remove unused code and update helper variables * Refactor HANA Pacemaker: Update HANA start task to use include_tasks instead of import_tasks * Refactor sap_namegenerator module: Update computer and VM names for HA configuration * Update github-actions-ansible-lint.yml Use commit hashes instead of versions Signed-off-by: hdamecharla <[email protected]> * Create scorecard.yml create ossf scorecard workflow Signed-off-by: hdamecharla <[email protected]> * Update scorecard.yml Only default branch is supported Signed-off-by: hdamecharla <[email protected]> * Create ossf-scorecard.yml create ossf-scorecard.yml in main Signed-off-by: hdamecharla <[email protected]> * chore: Update NFS filesystem resource configuration for HSR sites and general cluster properties * Refactor cluster preparation task to display cluster configuration status * Refactor cluster preparation task to remove unnecessary debug statements * Add rescue action * Refactor cluster preparation task to remove unnecessary debug statements and add rescue action * HDB start * Update main.yaml * Change the load balancer backend pool for scale out scenarios * Remove obsolete files --------- Signed-off-by: hdamecharla <[email protected]> Co-authored-by: Kimmo Forss <[email protected]> Co-authored-by: hdamecharla <[email protected]> Co-authored-by: Shekhar Sorot ( MSFT ) <[email protected]> Co-authored-by: Shekhar Sorot ( MSFT ) <[email protected]> Co-authored-by: hdamecharla <[email protected]>

…-hsr

Add support for HANA Scaleout with HSR #637

Add support for HANA Scaleout with HSR #637

Commits on Mar 6, 2024

Commits on Mar 7, 2024

Commits on Mar 8, 2024

Commits on Mar 9, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

Commits on Mar 14, 2024

Commits on Mar 15, 2024

Commits on Mar 16, 2024

Commits on Mar 19, 2024

Commits on Mar 25, 2024

Commits on Mar 26, 2024

Commits on Mar 27, 2024

Commits on Mar 28, 2024

Commits on Mar 29, 2024

Commits on Apr 2, 2024

Commits on Apr 3, 2024

Commits on Apr 7, 2024

Commits on Apr 8, 2024

Commits on Apr 10, 2024

Commits on Apr 11, 2024

Commits on Apr 15, 2024

Commits on Apr 16, 2024

Commits on Apr 17, 2024

Commits on Apr 18, 2024

Commits on Apr 19, 2024

Commits on Apr 27, 2024

Commits on May 17, 2024

Commits on May 19, 2024

Commits on May 25, 2024

Commits on May 26, 2024

Commits on May 27, 2024

Commits on May 28, 2024

Commits on May 29, 2024

Commits on May 30, 2024

Commits on May 31, 2024

Commits on Jun 1, 2024

Commits on Sep 11, 2024

Commits on Sep 13, 2024

Commits on Sep 14, 2024

Commits on Sep 15, 2024

Commits on Sep 23, 2024

Commits on Sep 24, 2024