Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RHEL 8.8 #473

Merged
merged 1 commit into from
Aug 7, 2023
Merged

Add RHEL 8.8 #473

merged 1 commit into from
Aug 7, 2023

Conversation

hjstam
Copy link
Contributor

@hjstam hjstam commented Aug 1, 2023

Problem

Red Hat Enterprise Linux 8.8 is missing causing this error: The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'redhat8.8'. Red Hat 8.8 can be used for db2 deployments.

Solution

Define RedHat 8.8 in repos and os-packages

@hjstam hjstam requested a review from a team as a code owner August 1, 2023 17:31
@hjstam hjstam requested a review from ajaygit158 August 1, 2023 17:31
@hjstam
Copy link
Contributor Author

hjstam commented Aug 7, 2023

@KimForss @hdamecharla please review this PR. We would like to test the DB2 deployments with RHEL 8.8.

Copy link
Member

@hdamecharla hdamecharla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

@hjstam
Copy link
Contributor Author

hjstam commented Aug 7, 2023

approved

Could you merge it as well?

@hdamecharla hdamecharla merged commit a1b1c2f into Azure:experimental Aug 7, 2023
2 checks passed
hdamecharla pushed a commit that referenced this pull request Aug 25, 2023
@hjstam hjstam deleted the patch/add-rhel-8.8 branch September 6, 2023 19:36
KimForss added a commit that referenced this pull request Sep 29, 2023
* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Added README.md file

* Added file CONTRIBUTING.md

* Updated README.md

* Updated CONTRIBUTING.md

* Added .gitignore

* Adding owners.txt containing ServiceTree admins

* v3.6 release

* Updated owners.txt

* Current code for ansible 3.7 that is being tested by ACSS.

* Added a delay before validating the SCS ERS cluster.

* Removed the BOM-catalog folder.

* Merged PR 8320960: Merging sap-automation main into this repo's main branch.

This is done as part of the 3.8.3 parity task. This is an attempt to bring the Co-Dev model into effect so that all ansible development can now happen through the SDAF-ASCS repo.

Related work items: #24291278

* Merged PR 8350970: Fix Logical Volume creation

Fix Logical Volume creation:
Description:

In the volume group vg_sap, we need to create several logical volumes which include for lv_sapmnt, lv_usrsapinstall, lv_usrsap. The goal is to allocate a predetermined amount of space to lv_sapmnt, lv_usersap and then allocate the rest of the space to lv_usersapinstall. But the order in which this allocation was being done was in reverse which resulted in lv_usersapinstall consuming all the space in vg_sap therefore, blocking the creation of other logical volumes. The ordering has been fixed in this PR.

This is the test proof for the same:

Query to view the results for createinfra test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDQQyG7%2fsrwpwqbGFZQUGpILXCHvzAFXoQD2kmtaO7M0sm21Lwxzur0A8qzCFD3id5k8wR6wdWcTRbs9fsGzYrFoZnYXKRX13LtWLbwQ1YVNb0H5VFeT4uLsblJRTF1e8724F1v4gkrlMXfGXBeRhVXlk8NoepuCdeOIZeiJOagld0PoJBuwpkdpq71NfHgXsz8yBfyyZsHrFl8w6TCZipcDJX%2baVgVOlJe%2bH%2f4aeOBQcDaSod8MGgqXsiZsvW5GDu0TUpGgx2Ej6Z9GQb%2bVHNmUiQabD85yiHmmXtiG9J3drptrL5wZBD%2fJHAHPb51CqIZYHF9kh5cgWMdJ1lP74BzVW1AQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "CreateInfrastructure"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

Query to view the results for install test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDMRCG7%2fsrhpwqbGFZQUGpIFphD37gCj2Ih2ky2mg2WSazWwr%2beBOFflAhhwnv%2b%2bSdmSwQ23sStno%2bkpfiG9YrYoInJm0jvdiOWsGuhyswKCTpPqmr%2bnRanU3rc6iqi99zsgXbYRk1215s8I0B62HSeCH26PaluCOeKYaBNSW3Dl7Q%2bggKzSpotfXcplwfM%2feqFoG%2f3l1YP2BH6g1mM1CNj4LOZcUFNP9zjz0x5uw0kGQy96baQWsiQ0aVoO7QulTl3noOn6TlaBHlwZtz5sA3wdBfMyW0xKPVdK3FjlY2jSn35sv1RwJL2OkpKrAhhuXmwHn0ARj1ZVH8AGqZJGKwAQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "InstallWorkload"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

1 SCS failure observed which was transient and succeeded on retry
These are the other errors observed which are not related to ansible:
InstallHostUnreachable
InstallServiceError
AzureVMIsNotInSupportedProvisioningState

Related work items: #24199283

* Merged PR 8352697: fix logical volume order with vg_sap

fix logical volume order with vg_sap

* Update owners.txt

* Remediating Azure Artifacts Configuration Issues in your Repository

* Merged PR 8383025: Getting latest release code from github repo into main branch.

Pulling in the changes made to the experimental branch so that the hotfix release (3.8.3.2) made in github public repo is present here as well

* Merged PR 8497623: Syncing SDAF Public main branch to SDAF-ACSS main branch

Syncing SDAF Public main branch to SDAF-ACSS main branch

Related work items: #24650568

* Merged PR 8543963: Added checks to see if dynamic BOM download has succeeded.

In /deploy/ansible/roles-sap/3.3-bom-processing/tasks/bom_processor.yaml,
added checks to see if:
For every media element defined in a BOM
- Dynamic BOM download has succeeded.
- Whether archive property is defined.

* Resolved merge conflict.

* Adding some files to gitignore.

* Removed Webapp/.npmrc

* Removed the Nuget.config file.

---------

Co-authored-by: Rob Chappelle <[email protected]>
Co-authored-by: PIE Easy Start <[email protected]>
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: MerlinBot <>
Co-authored-by: MerlinBot <MerlinBot>

* Update CONTRIBUTING.md (#481)

Add note to submit PRs against experimental branch instead of main.

---------

Co-authored-by: Jennifer Hajduk <[email protected]>
Co-authored-by: Rob Chappelle <[email protected]>
Co-authored-by: PIE Easy Start <[email protected]>
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* Merging changes from sdaf-acss main after creation of v3.8.3.5 (#482) (#483)

Problem
When ansible code failed during the PAS installation with message server or database offline, then error message doesn't have the proper information on the error.

Solution
Added the error detailing in the custom_filter.py with regards to PAS installation failures

Tests
This has already been tested in E2E and Loop Tests

Notes
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.5 (#482) (#483)

Problem
When ansible code failed during the PAS installation with message server or database offline, then error message doesn't have the proper information on the error.

Solution
Added the error detailing in the custom_filter.py with regards to PAS installation failures

Tests
This has already been tested in E2E and Loop Tests

Notes
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* Experimental (#456)

* Added error categorization changes.

* PR for pushing commits in SDAF-ASCS prod-ansible-3.7.0 into sap-automation.

* Added names to the block.

* Fixed the spacing for variable expansion,

* Fixed the rescue indentation.

* Resolved merge conflicts for changes to the deploy/ansible/roles-db/4.0.0-hdb-install/tasks/main.yaml.

* Resolving Hemanth's comments and Reviews.

* Fixing linting issues.

* Fixed linting issues.

---------

Co-authored-by: shayaksarkar <[email protected]>, hdamecharla <[email protected]>

* 3.8.2 release (#452)

* Add the database instance number

* Added a failed when condition

* Add pool & account IDs to outputs

* countrol output of transport volume

* add a try to the transport volume name to facilitate resource imports

* uppercase ansible_os_family

* Add OracleLinux

* Start with AVG support

* Add skip-broken

* change the dir where we search for sapcontrol

* restore AVG

* Add the EPEL repo

* Add the node_tier to dictionary

* Corrected attribute name

* removed the extra when statement

* remove the epel repo

* Use subscription-manager to enable repos

* Use DNF to register repo

* temporary disable subscription-manager repos

* adding libknet1-compress-lz4-plugin manually

* subscription-manager

* disable_gpg_check

* removed test package

* Add AVG later

* put cluster in maintenance mode after generic resource creation on RHEL

* update repo state

* add install azure-cli to configure_deployer.sh

* put cluster in maintenance mode after generic resource creation on SUSE

* update the code for managing Azure CLI installs

* Don't ise subscription-manager

* cluster maintenance mode changes for RHEL & SUSE

* only upgrade the packages once

* Add OL repo

* removed root

* Rationalize login code, provide ARM_SUBSCRIPTION_ID as output

* change order of when condition

* Add azapi

* removed the failed_when

* Don't install ERS if SCS fails

* Remove the extra when statement

* removed the when

* add failed_when conditions

* azapi

* local not locals

* removed extra variables

* Logon using SPN comparison

* Don't use the variable

* use the pool as data source

* Remove AVG for now

* changes to cluster components

* Only set root owner if SAP users have not been created

* Look for sap sid

* update ERS HA install rescue action

* change inventory_hostname to ansible_hostname in 5.6.4.0-cluster-Suse.yaml

* also check for installationSuccesfullyFinished file

* Fix the search path

* check if installation output folder exists before doing find

* remove the state attribute from stat

* verify output directory exists before searching

* don't recurse

* Bring in the Fixes to get Ha SCS installation working (#448)

* Control ini file cleanup

* check the installation file

* Add More debugging info

* Testing the 3.7 codebase

* Use code from 3.7

* Ensure primary node is online

* don't put cluster in maintenance mode yet

* run as root

* feng Shui

* Linting fixes

* Indentation fixes

---------

Co-authored-by: Kimmo Forss <[email protected]>

* comment out the additional call to create SAP resources.

* comment the maintenance mode code for RHEL as well.

* Introduce AVG

* Add AVG variable to dictionary

* add additional data source

* simplify the data sources

* update the disctionary

* Set defaults for ppg and avset

* Use the correct dictionary variable

* add log & shared

* Remove the count for the data source

* add the avg resources

* Add the account ID

* Make the rule naming more tolerant

* simplify the data source

* naming update

* Use the account ID

* Add a try statement for ANF account name

* Add a try statement for ANF pool name

* Fix for empty arm_id

* create empty dictionary

* change uuid

* Change networkFeatures

* provide empty dictionary

* Fix HANA log

* fix creationToken

* set unixPermissions

* unixPermissions

* removed unix

* add try for capacityPoolResourceId

* serviceLevel             = "Premium"

* set kerberos5xReadOnly to false

* service_level

* Add Zonal support

* xompact the volumes

* improve readability

* AVG updates

* Use the second ppg

* zonality fixes for avg

* If AVGs are used don't create standalone volumes

* correct spec

* AVG ppg logic

* Updated the creationToken

* Create the volumes

* Add dependencies

* Check the output

* Use output

* Use the AVG name

* Support updating the AVG

* location is not needed

* User standard azurerm

* remove azapi

* simplify enumerator

* remove azapi

* added resource_group_name =

* enumerator

* change provider

* HANA data data source

* HANA shared output

* Add secondary sapmnt

* Incorrect function

* make sapmnt multi value

* Don't have sap_mnt as list

* Don't look for HANA shared if it is provided by ANF

* Default to no shared disk

* HANA shared disk logic

* hana_shared_mountpoint logic

* missing if

* Debug adding shared disk

* Removed the hana shared disk from main dictionary

* Add zones to AVG volumes

* No zonal support

* Use Ansible's magic variables for mounts

* Don't show unneeded files from hosts

* Corrected the task names

* Web App semicolon fix

* Don't constrain NFS info

* Don't call AFS tasks for ANF usage

* Add a try statement to handle upgrades from older versions

* add the missing variables

* Don't use SCS cluster name

* Don't use the HA template for non HA deployments

* Add an if statement to control the jinja template name

* Don't link the VNEts if no private endpoints

* Create bom folders as orchestration user

* Add quotes for the web dispatcher instance number

* Don't fail for local DNS

* 3.8.1 release

* Add support for SPN login for control plane

* Set pipefail

* Update the linter and skip the octal value rule

* Linting updates

* Linting fixes

* remove whitespace

* update sshkey playbook

* Fix the pipefails

* password jinja updates

* set -o pipefail &&

* Don't lint chrony

* Fix lint for chrony

* Add the filter folder

* Use the correct instance numbers

* rename web_instance_number variable in sap-parameters.yml.tmpl to wd_instance_number.

* push changes

* fix web dispatcher variables

* Add Logon using SPN code

* Add SPN logon to removal pipeline

* set 20GB swap for web dispatcher

* disable selinux for web tier

* Add support for marketplace images with plan

* remove the set pipefail

* swap the attributes in plan

* swap plan components

* Add RHEL 9 support for packages, repositories and services

* Make python3 the default and only set python2 for SLES12

* Use the internal DNS when using private endpoints

* Upgrade az cli

* sudo upgrade

* Removed the connection_String setting

* Change to use MSI

* Don't assign specific permission for web app

* Provide DB SKU override

* Add the SKU override in the Web App

* Add the ability to not deploy the Microsoft.AzureCAT.AzureEnhancedMonitoring extension

* Correct the variable name

* Add RHEL 90

* Add Kernel parameters for RHEL 9

* removed the package libssh2

* removed libpng12

* removed oddjobd service

* Add RHEL9 distro

* 3119751 only mentions RHEL8

* update packages

* Set NIC name for private endpoint for control plane

* Upgrade Azure CLI & Terraform when re-running

* Bump terrafom version

* removed  compat-sap-c++-*

* Dont register Microsoft keys

* USe CLI version 2.47

* Lock 0n 2.47 on Ubuntu 20.04

* Add llogging for CLI

* Show what version got installed

* Use grep to get the cli version

* Simplify re-install

* Ad more debug info on which agent is in use

* SELinux for RHEL 9

* Allow rerun for manual deployer

* Fix virtualenv on RHEL

* RHEL deployer updates

* Save the random ID value

* missing $

* Corrected the output variable

* use $$ in template

* show YAML version

* debug HANA on RHEL 9

* testing HANA install

* debugging HANA installation

* Add package chkconfig

* only install chkconfig on hana

* move flush handler

* Use azurerm backend if reinstall

* remove extra password parameter

* test password setting for HANA installation

* add password for

* use the correct password copy

* Only delete ini file if installation is successful

* moved chkconfig to 9.0 dictionary

* Add debugging

* Improve re-install experience

* reset firewall

* storage account fire wall

* export deployer_kv_user_arm_id

* update variable name

* Don't rerun the secrets setting

* Remove Oracle repos

* Add more images for RHEL

* update BoM Validator

* Added installation of devops extension in the main pipelines

* Provide the ability to control zonal ANF

---------

Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* delete samples and training materials as they are not part of the latest code set.

* 3.8.3 Release (#454)

* naming update

* Use the account ID

* Add a try statement for ANF account name

* Add a try statement for ANF pool name

* Fix for empty arm_id

* create empty dictionary

* change uuid

* Change networkFeatures

* provide empty dictionary

* Fix HANA log

* fix creationToken

* set unixPermissions

* unixPermissions

* removed unix

* add try for capacityPoolResourceId

* serviceLevel             = "Premium"

* set kerberos5xReadOnly to false

* service_level

* Add Zonal support

* xompact the volumes

* improve readability

* AVG updates

* Use the second ppg

* zonality fixes for avg

* If AVGs are used don't create standalone volumes

* correct spec

* AVG ppg logic

* Updated the creationToken

* Create the volumes

* Add dependencies

* Check the output

* Use output

* Use the AVG name

* Support updating the AVG

* location is not needed

* User standard azurerm

* remove azapi

* simplify enumerator

* remove azapi

* added resource_group_name =

* enumerator

* change provider

* HANA data data source

* HANA shared output

* Add secondary sapmnt

* Incorrect function

* make sapmnt multi value

* Don't have sap_mnt as list

* Don't look for HANA shared if it is provided by ANF

* Default to no shared disk

* HANA shared disk logic

* hana_shared_mountpoint logic

* missing if

* Debug adding shared disk

* Removed the hana shared disk from main dictionary

* Add zones to AVG volumes

* No zonal support

* Use Ansible's magic variables for mounts

* Don't show unneeded files from hosts

* Corrected the task names

* Web App semicolon fix

* Don't constrain NFS info

* Don't call AFS tasks for ANF usage

* Add a try statement to handle upgrades from older versions

* add the missing variables

* Don't use SCS cluster name

* Don't use the HA template for non HA deployments

* Add an if statement to control the jinja template name

* Don't link the VNEts if no private endpoints

* Create bom folders as orchestration user

* Add quotes for the web dispatcher instance number

* Don't fail for local DNS

* 3.8.1 release

* Add support for SPN login for control plane

* Set pipefail

* Update the linter and skip the octal value rule

* Linting updates

* Linting fixes

* remove whitespace

* update sshkey playbook

* Fix the pipefails

* password jinja updates

* set -o pipefail &&

* Don't lint chrony

* Fix lint for chrony

* Add the filter folder

* Use the correct instance numbers

* rename web_instance_number variable in sap-parameters.yml.tmpl to wd_instance_number.

* push changes

* fix web dispatcher variables

* Add Logon using SPN code

* Add SPN logon to removal pipeline

* set 20GB swap for web dispatcher

* disable selinux for web tier

* Add support for marketplace images with plan

* remove the set pipefail

* swap the attributes in plan

* swap plan components

* Add RHEL 9 support for packages, repositories and services

* Make python3 the default and only set python2 for SLES12

* Use the internal DNS when using private endpoints

* Upgrade az cli

* sudo upgrade

* Removed the connection_String setting

* Change to use MSI

* Don't assign specific permission for web app

* Provide DB SKU override

* Add the SKU override in the Web App

* Add the ability to not deploy the Microsoft.AzureCAT.AzureEnhancedMonitoring extension

* Correct the variable name

* Add RHEL 90

* Add Kernel parameters for RHEL 9

* removed the package libssh2

* removed libpng12

* removed oddjobd service

* Add RHEL9 distro

* 3119751 only mentions RHEL8

* update packages

* Set NIC name for private endpoint for control plane

* Upgrade Azure CLI & Terraform when re-running

* Bump terrafom version

* removed  compat-sap-c++-*

* Dont register Microsoft keys

* USe CLI version 2.47

* Lock 0n 2.47 on Ubuntu 20.04

* Add llogging for CLI

* Show what version got installed

* Use grep to get the cli version

* Simplify re-install

* Ad more debug info on which agent is in use

* SELinux for RHEL 9

* Allow rerun for manual deployer

* Fix virtualenv on RHEL

* RHEL deployer updates

* Save the random ID value

* missing $

* Corrected the output variable

* use $$ in template

* show YAML version

* debug HANA on RHEL 9

* testing HANA install

* debugging HANA installation

* Add package chkconfig

* only install chkconfig on hana

* move flush handler

* Use azurerm backend if reinstall

* remove extra password parameter

* test password setting for HANA installation

* add password for

* use the correct password copy

* Only delete ini file if installation is successful

* moved chkconfig to 9.0 dictionary

* Add debugging

* Improve re-install experience

* reset firewall

* storage account fire wall

* export deployer_kv_user_arm_id

* update variable name

* Don't rerun the secrets setting

* Remove Oracle repos

* Add more images for RHEL

* update BoM Validator

* Added installation of devops extension in the main pipelines

* Provide the ability to control zonal ANF

* Add zonal validation for HANA log and shared

* Update zonality for sapmnt

* Add VMSS-Flex support

* Don't create PPG if VMSS-Flex is used

* Use the PPG ID and not the full object

* typo

* Fix AVG and scale set

* Don't create AVG if VMSS-Flex is used

* change resource name

* Update the provider

* 1.3 Repos: Add the HA repositories for RHEL

* 1.3 Repos: Add the HA repositories for RHEL

* app disk zonal logic

* add support for RHEL 9

* Add RHEL 9 to clustering

* Add documentation to output variables

* Introduce resource lock variables

* Control the secondary DNS registration

* Control the output

* DNS outputs

* Add dependencies and a wait

* Use the server counters

* Add resource locks

* don't save the randomizer value

* set the connection string secret via terraform

* update resource name

* update secret name

* shorten the names for the virtual hosts

* remove duplicate

* corrected local variable

* Check for key vault existence

* correct variable for vnet

* Update the NIC name for the sapmnt private endpoint

* Add scaleset and lock support to web app

* Use both identities

* simplify keyvault existence checks

* Shorten the virtual hostname for anydb

* The IP address are in different order in SLES and RHEL

* Check the hosts code

* debugging hosts

* change play_hosts to ansible_play_hosts

* update

* debug hosts file

* Remove the serialization

* Add support for premium storage v2

* Fixed caching

* Change endpoint name

* Premium storage V2

* Identity assignment

* HANA sizing

* disable SELinux on RHEL9

* Add a dependency on blob DNS link

* add check to verify If library deployment failed

* Add the scope

* Add the mandatory scope parameter

* Fix typo in command

* Add the scope to the extension code

* pass the variables for exports when the PAS and SCS are on the same server.

* Add the missing 's'

* Add AVG to Web App

* Add the correct packages

* Install compat-libcap1 package manually

* move the yum command

* typo in package name

* disable_gpg_check on the compat-libcap1

* Use builtin package handling

* Add distro info to HANA cluster tasks

* Add the permissions

* It is not a directory but a file

* removed duplicate package

* Add the SBP fixes

* Only do the ifup eth0 on RHEL

* ASM updates

* Misc updates

* SYBASE and RHEL

* Fix the profile name to sap-netweaver

* Add the virtual host for observer

* Experimental (#456)

* Added error categorization changes.

* PR for pushing commits in SDAF-ASCS prod-ansible-3.7.0 into sap-automation.

* Added names to the block.

* Fixed the spacing for variable expansion,

* Fixed the rescue indentation.

* Resolved merge conflicts for changes to the deploy/ansible/roles-db/4.0.0-hdb-install/tasks/main.yaml.

* Resolving Hemanth's comments and Reviews.

* Fixing linting issues.

* Fixed linting issues.

---------

Co-authored-by: shayaksarkar <[email protected]>, hdamecharla <[email protected]>

---------

Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: hdamecharla <[email protected]>
Co-authored-by: ShayakSarkar <[email protected]>

* Update configure_deployer.sh (#464)

Removed duplicate "dotnet" argument in dotnet-sdk alias command for Ubuntu and SLES.

Was resulting in error message: "error: too many arguments for command" in testing on Ubuntu 22.04 LTS.

* Fixing Logical Volume creation order (#466)

Fix Logical Volume creation:
Description:

In the volume group vg_sap, we need to create several logical volumes which include for lv_sapmnt, lv_usrsapinstall, lv_usrsap. The goal is to allocate a predetermined amount of space to lv_sapmnt, lv_usersap and then allocate the rest of the space to lv_usersapinstall. But the order in which this allocation was being done was in reverse which resulted in lv_usersapinstall consuming all the space in vg_sap therefore, blocking the creation of other logical volumes. The ordering has been fixed in this PR.

Co-authored-by: Jaskirat Singh <[email protected]>

* Provide several fixes for Terraform and Ansible (#465)

* Provide several fixes for Terraform code

* Fix internet check

* Fix owner of BOM directories

* Ignore PyCache directories

* Remove become from SDAF deployer check

* Bugfix dns_info_loadbalancers output variable for app_tier

* Add become to App installation block for Linux

* Create compact-sap-c++-10 symlink owner and group on the symlink, not the src file

The /opt/rh/SAP/lib64/compat-sap-c++-10.so src is provided by a Red Hat RPM. The owner / group should stay the same.

* sapmnt shouldn't be created on HANA systems

---------

Co-authored-by: Marges, RSY (Rick) <[email protected]>
Co-authored-by: Harm Jan Stam <[email protected]>

* Add RHEL 8.8 (#473)

* Several fixes while testing with Ansible 2.14.6 (#474)

* Remove unsupported warn parameter from tasks

The warn parameter isn't supported anymore with newer Ansible versions.

* Bugfix grub config file content

The lookup module is executed on the controller and not on the remote host. Replace the lookup with a slurp.

* Bugfix BOM aggregate only find local bom folders

* Refactor curl command for Azure instance metadata to Ansible uri module

The Ansible uri module recognizes the json output returned by the azure metadata instance api. No converting to json is therefore needed anymore.

* Refactor anydb_node dns_info_vms output

* Remove how file

* Cast db_sid for db2sid user and add abap_connect_user variable (#476)

* Fix DB2 for installation to lowercase db2sid folder

* Add input API variable for ABAP connect user

* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.5 (#482) (#483)

Problem
When ansible code failed during the PAS installation with message server or database offline, then error message doesn't have the proper information on the error.

Solution
Added the error detailing in the custom_filter.py with regards to PAS installation failures

Tests
This has already been tested in E2E and Loop Tests

Notes
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Added README.md file

* Added file CONTRIBUTING.md

* Updated README.md

* Updated CONTRIBUTING.md

* Added .gitignore

* Adding owners.txt containing ServiceTree admins

* v3.6 release

* Updated owners.txt

* Current code for ansible 3.7 that is being tested by ACSS.

* Added a delay before validating the SCS ERS cluster.

* Removed the BOM-catalog folder.

* Merged PR 8320960: Merging sap-automation main into this repo's main branch.

This is done as part of the 3.8.3 parity task. This is an attempt to bring the Co-Dev model into effect so that all ansible development can now happen through the SDAF-ASCS repo.

Related work items: #24291278

* Merged PR 8350970: Fix Logical Volume creation

Fix Logical Volume creation:
Description:

In the volume group vg_sap, we need to create several logical volumes which include for lv_sapmnt, lv_usrsapinstall, lv_usrsap. The goal is to allocate a predetermined amount of space to lv_sapmnt, lv_usersap and then allocate the rest of the space to lv_usersapinstall. But the order in which this allocation was being done was in reverse which resulted in lv_usersapinstall consuming all the space in vg_sap therefore, blocking the creation of other logical volumes. The ordering has been fixed in this PR.

This is the test proof for the same:

Query to view the results for createinfra test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDQQyG7%2fsrwpwqbGFZQUGpILXCHvzAFXoQD2kmtaO7M0sm21Lwxzur0A8qzCFD3id5k8wR6wdWcTRbs9fsGzYrFoZnYXKRX13LtWLbwQ1YVNb0H5VFeT4uLsblJRTF1e8724F1v4gkrlMXfGXBeRhVXlk8NoepuCdeOIZeiJOagld0PoJBuwpkdpq71NfHgXsz8yBfyyZsHrFl8w6TCZipcDJX%2baVgVOlJe%2bH%2f4aeOBQcDaSod8MGgqXsiZsvW5GDu0TUpGgx2Ej6Z9GQb%2bVHNmUiQabD85yiHmmXtiG9J3drptrL5wZBD%2fJHAHPb51CqIZYHF9kh5cgWMdJ1lP74BzVW1AQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "CreateInfrastructure"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

Query to view the results for install test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDMRCG7%2fsrhpwqbGFZQUGpIFphD37gCj2Ih2ky2mg2WSazWwr%2beBOFflAhhwnv%2b%2bSdmSwQ23sStno%2bkpfiG9YrYoInJm0jvdiOWsGuhyswKCTpPqmr%2bnRanU3rc6iqi99zsgXbYRk1215s8I0B62HSeCH26PaluCOeKYaBNSW3Dl7Q%2bggKzSpotfXcplwfM%2feqFoG%2f3l1YP2BH6g1mM1CNj4LOZcUFNP9zjz0x5uw0kGQy96baQWsiQ0aVoO7QulTl3noOn6TlaBHlwZtz5sA3wdBfMyW0xKPVdK3FjlY2jSn35sv1RwJL2OkpKrAhhuXmwHn0ARj1ZVH8AGqZJGKwAQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "InstallWorkload"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

1 SCS failure observed which was transient and succeeded on retry
These are the other errors observed which are not related to ansible:
InstallHostUnreachable
InstallServiceError
AzureVMIsNotInSupportedProvisioningState

Related work items: #24199283

* Merged PR 8352697: fix logical volume order with vg_sap

fix logical volume order with vg_sap

* Update owners.txt

* Remediating Azure Artifacts Configuration Issues in your Repository

* Merged PR 8383025: Getting latest release code from github repo into main branch.

Pulling in the changes made to the experimental branch so that the hotfix release (3.8.3.2) made in github public repo is present here as well

* Merged PR 8497623: Syncing SDAF Public main branch to SDAF-ACSS main branch

Syncing SDAF Public main branch to SDAF-ACSS main branch

Related work items: #24650568

* Merged PR 8543963: Added checks to see if dynamic BOM download has succeeded.

In /deploy/ansible/roles-sap/3.3-bom-processing/tasks/bom_processor.yaml,
added checks to see if:
For every media element defined in a BOM
- Dynamic BOM download has succeeded.
- Whether archive property is defined.

* Resolved merge conflict.

* Adding some files to gitignore.

* Removed Webapp/.npmrc

* Removed the Nuget.config file.

---------

Co-authored-by: Rob Chappelle <[email protected]>
Co-authored-by: PIE Easy Start <[email protected]>
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: MerlinBot <>
Co-authored-by: MerlinBot <MerlinBot>

* Update CONTRIBUTING.md (#481)

Add note to submit PRs against experimental branch instead of main.

---------

Co-authored-by: Jennifer Hajduk <[email protected]>
Co-authored-by: Rob Chappelle <[email protected]>
Co-authored-by: PIE Easy Start <[email protected]>
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Added README.md file

* Added file CONTRIBUTING.md

* Updated README.md

* Updated CONTRIBUTING.md

* Added .gitignore

* Adding owners.txt containing ServiceTree admins

* v3.6 release

* Updated owners.txt

* Current code for ansible 3.7 that is being tested by ACSS.

* Added a delay before validating the SCS ERS cluster.

* Removed the BOM-catalog folder.

* Merged PR 8320960: Merging sap-automation main into this repo's main branch.

This is done as part of the 3.8.3 parity task. This is an attempt to bring the Co-Dev model into effect so that all ansible development can now happen through the SDAF-ASCS repo.

Related work items: #24291278

* Merged PR 8350970: Fix Logical Volume creation

Fix Logical Volume creation:
Description:

In the volume group vg_sap, we need to create several logical volumes which include for lv_sapmnt, lv_usrsapinstall, lv_usrsap. The goal is to allocate a predetermined amount of space to lv_sapmnt, lv_usersap and then allocate the rest of the space to lv_usersapinstall. But the order in which this allocation was being done was in reverse which resulted in lv_usersapinstall consuming all the space in vg_sap therefore, blocking the creation of other logical volumes. The ordering has been fixed in this PR.

This is the test proof for the same:

Query to view the results for createinfra test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDQQyG7%2fsrwpwqbGFZQUGpILXCHvzAFXoQD2kmtaO7M0sm21Lwxzur0A8qzCFD3id5k8wR6wdWcTRbs9fsGzYrFoZnYXKRX13LtWLbwQ1YVNb0H5VFeT4uLsblJRTF1e8724F1v4gkrlMXfGXBeRhVXlk8NoepuCdeOIZeiJOagld0PoJBuwpkdpq71NfHgXsz8yBfyyZsHrFl8w6TCZipcDJX%2baVgVOlJe%2bH%2f4aeOBQcDaSod8MGgqXsiZsvW5GDu0TUpGgx2Ej6Z9GQb%2bVHNmUiQabD85yiHmmXtiG9J3drptrL5wZBD%2fJHAHPb51CqIZYHF9kh5cgWMdJ1lP74BzVW1AQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "CreateInfrastructure"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

Query to view the results for install test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDMRCG7%2fsrhpwqbGFZQUGpIFphD37gCj2Ih2ky2mg2WSazWwr%2beBOFflAhhwnv%2b%2bSdmSwQ23sStno%2bkpfiG9YrYoInJm0jvdiOWsGuhyswKCTpPqmr%2bnRanU3rc6iqi99zsgXbYRk1215s8I0B62HSeCH26PaluCOeKYaBNSW3Dl7Q%2bggKzSpotfXcplwfM%2feqFoG%2f3l1YP2BH6g1mM1CNj4LOZcUFNP9zjz0x5uw0kGQy96baQWsiQ0aVoO7QulTl3noOn6TlaBHlwZtz5sA3wdBfMyW0xKPVdK3FjlY2jSn35sv1RwJL2OkpKrAhhuXmwHn0ARj1ZVH8AGqZJGKwAQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "InstallWorkload"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

1 SCS failure observed which was transient and succeeded on retry
These are the other errors observed which are not related to ansible:
InstallHostUnreachable
InstallServiceError
AzureVMIsNotInSupportedProvisioningState

Related work items: #24199283

* Merged PR 8352697: fix logical volume order with vg_sap

fix logical volume order with vg_sap

* Update owners.txt

* Remediating Azure Artifacts Configuration Issues in your Repository

* Merged PR 8383025: Getting latest release code from github repo into main branch.

Pulling in the changes made to the experimental branch so that the hotfix release (3.8.3.2) made in github public repo is present here as well

* Merged PR 8497623: Syncing SDAF Public main branch to SDAF-ACSS main branch

Syncing SDAF Public main branch to SDAF-ACSS main branch

Related work items: #24650568

* Merged PR 8543963: Added checks to see if dynamic BOM download has succeeded.

In /deploy/ansible/roles-sap/3.3-bom-processing/tasks/bom_processor.yaml,
added checks to see if:
For every media element defined in a BOM
- Dynamic BOM download has succeeded.
- Whether archive property is defined.

* Resolved merge conflict.

* Adding some files to gitignore.

* Removed Webapp/.npmrc

* Removed the Nuget.config file.

---------

Co-authored-by: Rob Chappelle <[email protected]>
Co-authored-by: PIE Easy Start <[email protected]>
Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: sedusch <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>
Co-authored-by: MerlinBot <>
Co-authored-by: MerlinBot <MerlinBot>

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Added checks to see if dynamic BOM download has succeeded.

In /deploy/ansible/roles-sap/3.3-bom-processing/tasks/bom_processor.yaml,
added checks to see if:
For every media element defined in a BOM
- Dynamic BOM download has succeeded.
- Whether archive property is defined.

* Resolved merge conflict.

* Adding some files to gitignore.

---------

Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: Shayak Sarkar <[email protected]>
Co-authored-by: Jaskirat Singh <[email protected]>

* Experimental (#456)

* Added error categorization changes.

* PR for pushing commits in SDAF-ASCS prod-ansible-3.7.0 into sap-automation.

* Added names to the block.

* Fixed the spacing for variable expansion,

* Fixed the rescue indentation.

* Resolved merge conflicts for changes to the deploy/ansible/roles-db/4.0.0-hdb-install/tasks/main.yaml.

* Resolving Hemanth's comments and Reviews.

* Fixing linting issues.

* Fixed linting issues.

---------

Co-authored-by: shayaksarkar <[email protected]>, hdamecharla <[email protected]>

* 3.8.2 release (#452)

* Add the database instance number

* Added a failed when condition

* Add pool & account IDs to outputs

* countrol output of transport volume

* add a try to the transport volume name to facilitate resource imports

* uppercase ansible_os_family

* Add OracleLinux

* Start with AVG support

* Add skip-broken

* change the dir where we search for sapcontrol

* restore AVG

* Add the EPEL repo

* Add the node_tier to dictionary

* Corrected attribute name

* removed the extra when statement

* remove the epel repo

* Use subscription-manager to enable repos

* Use DNF to register repo

* temporary disable subscription-manager repos

* adding libknet1-compress-lz4-plugin manually

* subscription-manager

* disable_gpg_check

* removed test package

* Add AVG later

* put cluster in maintenance mode after generic resource creation on RHEL

* update repo state

* add install azure-cli to configure_deployer.sh

* put cluster in maintenance mode after generic resource creation on SUSE

* update the code for managing Azure CLI installs

* Don't ise subscription-manager

* cluster maintenance mode changes for RHEL & SUSE

* only upgrade the packages once

* Add OL repo

* removed root

* Rationalize login code, provide ARM_SUBSCRIPTION_ID as output

* change order of when condition

* Add azapi

* removed the failed_when

* Don't install ERS if SCS fails

* Remove the extra when statement

* removed the when

* add failed_when conditions

* azapi

* local not locals

* removed extra variables

* Logon using SPN comparison

* Don't use the variable

* use the pool as data source

* Remove AVG for now

* changes to cluster components

* Only set root owner if SAP users have not been created

* Look for sap sid

* update ERS HA install rescue action

* change inventory_hostname to ansible_hostname in 5.6.4.0-cluster-Suse.yaml

* also check for installationSuccesfullyFinished file

* Fix the search path

* check if installation output folder exists before doing find

* remove the state attribute from stat

* verify output directory exists before searching

* don't recurse

* Bring in the Fixes to get Ha SCS installation working (#448)

* Control ini file cleanup

* check the installation file

* Add More debugging info

* Testing the 3.7 codebase

* Use code from 3.7

* Ensure primary node is online

* don't put cluster in maintenance mode yet

* run as root

* feng Shui

* Linting fixes

* Indentation fixes

---------

Co-authored-by: Kimmo Forss <[email protected]>

* comment out the additional call to create SAP resources.

* comment the maintenance mode code for RHEL as well.

* Introduce AVG

* Add AVG variable to dictionary

* add additional data source

* simplify the data sources

* update the disctionary

* Set defaults for ppg and avset

* Use the correct dictionary variable

* add log & shared

* Remove the count for the data source

* add the avg resources

* Add the account ID

* Make the rule naming more tolerant

* simplify the data source

* naming update

* Use the account ID

* Add a try statement for ANF account name

* Add a try statement for ANF pool name

* Fix for empty arm_id

* create empty dictionary

* change uuid

* Change networkFeatures

* provide empty dictionary

* Fix HANA log

* fix creationToken

* set unixPermissions

* unixPermissions

* removed unix

* add try for capacityPoolResourceId

* serviceLevel             = "Premium"

* set kerberos5xReadOnly to false

* service_level

* Add Zonal support

* xompact the volumes

* improve readability

* AVG updates

* Use the second ppg

* zonality fixes for avg

* If AVGs are used don't create standalone volumes

* correct spec

* AVG ppg logic

* Updated the creationToken

* Create the volumes

* Add dependencies

* Check the output

* Use output

* Use the AVG name

* Support updating the AVG

* location is not needed

* User standard azurerm

* remove azapi

* simplify enumerator

* remove azapi

* added resource_group_name =

* enumerator

* change provider

* HANA data data source

* HANA shared output

* Add secondary sapmnt

* Incorrect function

* make sapmnt multi value

* Don't have sap_mnt as list

* Don't look for HANA shared if it is provided by ANF

* Default to no shared disk

* HANA shared disk logic

* hana_shared_mountpoint logic

* missing if

* Debug adding shared disk

* Removed the hana shared disk from main dictionary

* Add zones to AVG volumes

* No zonal support

* Use Ansible's magic variables for mounts

* Don't show unneeded files from hosts

* Corrected the task names

* Web App semicolon fix

* Don't constrain NFS info

* Don't call AFS tasks for ANF usage

* Add a try statement to handle upgrades from older versions

* add the missing variables

* Don't use SCS cluster name

* Don't use the HA template for non HA deployments

* Add an if statement to control the jinja template name

* Don't link the VNEts if no private endpoints

* Create bom folders as orchestration user

* Add quotes for the web dispatcher instance number

* Don't fail for local DNS

* 3.8.1 release

* Add support for SPN login for control plane

* Set pipefail

* Update the linter and skip the octal value rule

* Linting updates

* Linting fixes

* remove whitespace

* update sshkey playbook

* Fix the pipefails

* password jinja updates

* set -o pipefail &&

* Don't lint chrony

* Fix lint for chrony

* Add the filter folder

* Use the correct instance numbers

* rename web_instance_number variable in sap-parameters.yml.tmpl to wd_instance_number.

* push changes

* fix web dispatcher variables

* Add Logon using SPN code

* Add SPN logon to removal pipeline

* set 20GB swap for web dispatcher

* disable selinux for web tier

* Add support for marketplace images with plan

* remove the set pipefail

* swap the attributes in plan

* swap plan components

* Add RHEL 9 support for packages, repositories and services

* Make python3 the default and only set python2 for SLES12

* Use the internal DNS when using private endpoints

* Upgrade az cli

* sudo upgrade

* Removed the connection_String setting

* Change to use MSI

* Don't assign specific permission for web app

* Provide DB SKU override

* Add the SKU override in the Web App

* Add the ability to not deploy the Microsoft.AzureCAT.AzureEnhancedMonitoring extension

* Correct the variable name

* Add RHEL 90

* Add Kernel parameters for RHEL 9

* removed the package libssh2

* removed libpng12

* removed oddjobd service

* Add RHEL9 distro

* 3119751 only mentions RHEL8

* update packages

* Set NIC name for private endpoint for control plane

* Upgrade Azure CLI & Terraform when re-running

* Bump terrafom version

* removed  compat-sap-c++-*

* Dont register Microsoft keys

* USe CLI version 2.47

* Lock 0n 2.47 on Ubuntu 20.04

* Add llogging for CLI

* Show what version got installed

* Use grep to get the cli version

* Simplify re-install

* Ad more debug info on which agent is in use

* SELinux for RHEL 9

* Allow rerun for manual deployer

* Fix virtualenv on RHEL

* RHEL deployer updates

* Save the random ID value

* missing $

* Corrected the output variable

* use $$ in template

* show YAML version

* debug HANA on RHEL 9

* testing HANA install

* debugging HANA installation

* Add package chkconfig

* only install chkconfig on hana

* move flush handler

* Use azurerm backend if reinstall

* remove extra password parameter

* test password setting for HANA installation

* add password for

* use the correct password copy

* Only delete ini file if installation is successful

* moved chkconfig to 9.0 dictionary

* Add debugging

* Improve re-install experience

* reset firewall

* storage account fire wall

* export deployer_kv_user_arm_id

* update variable name

* Don't rerun the secrets setting

* Remove Oracle repos

* Add more images for RHEL

* update BoM Validator

* Added installation of devops extension in the main pipelines

* Provide the ability to control zonal ANF

---------

Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: hdamecharla <[email protected]>

* 3.8.3 Release (#454)

* naming update

* Use the account ID

* Add a try statement for ANF account name

* Add a try statement for ANF pool name

* Fix for empty arm_id

* create empty dictionary

* change uuid

* Change networkFeatures

* provide empty dictionary

* Fix HANA log

* fix creationToken

* set unixPermissions

* unixPermissions

* removed unix

* add try for capacityPoolResourceId

* serviceLevel             = "Premium"

* set kerberos5xReadOnly to false

* service_level

* Add Zonal support

* xompact the volumes

* improve readability

* AVG updates

* Use the second ppg

* zonality fixes for avg

* If AVGs are used don't create standalone volumes

* correct spec

* AVG ppg logic

* Updated the creationToken

* Create the volumes

* Add dependencies

* Check the output

* Use output

* Use the AVG name

* Support updating the AVG

* location is not needed

* User standard azurerm

* remove azapi

* simplify enumerator

* remove azapi

* added resource_group_name =

* enumerator

* change provider

* HANA data data source

* HANA shared output

* Add secondary sapmnt

* Incorrect function

* make sapmnt multi value

* Don't have sap_mnt as list

* Don't look for HANA shared if it is provided by ANF

* Default to no shared disk

* HANA shared disk logic

* hana_shared_mountpoint logic

* missing if

* Debug adding shared disk

* Removed the hana shared disk from main dictionary

* Add zones to AVG volumes

* No zonal support

* Use Ansible's magic variables for mounts

* Don't show unneeded files from hosts

* Corrected the task names

* Web App semicolon fix

* Don't constrain NFS info

* Don't call AFS tasks for ANF usage

* Add a try statement to handle upgrades from older versions

* add the missing variables

* Don't use SCS cluster name

* Don't use the HA template for non HA deployments

* Add an if statement to control the jinja template name

* Don't link the VNEts if no private endpoints

* Create bom folders as orchestration user

* Add quotes for the web dispatcher instance number

* Don't fail for local DNS

* 3.8.1 release

* Add support for SPN login for control plane

* Set pipefail

* Update the linter and skip the octal value rule

* Linting updates

* Linting fixes

* remove whitespace

* update sshkey playbook

* Fix the pipefails

* password jinja updates

* set -o pipefail &&

* Don't lint chrony

* Fix lint for chrony

* Add the filter folder

* Use the correct instance numbers

* rename web_instance_number variable in sap-parameters.yml.tmpl to wd_instance_number.

* push changes

* fix web dispatcher variables

* Add Logon using SPN code

* Add SPN logon to removal pipeline

* set 20GB swap for web dispatcher

* disable selinux for web tier

* Add support for marketplace images with plan

* remove the set pipefail

* swap the attributes in plan

* swap plan components

* Add RHEL 9 support for packages, repositories and services

* Make python3 the default and only set python2 for SLES12

* Use the internal DNS when using private endpoints

* Upgrade az cli

* sudo upgrade

* Removed the connection_String setting

* Change to use MSI

* Don't assign specific permission for web app

* Provide DB SKU override

* Add the SKU override in the Web App

* Add the ability to not deploy the Microsoft.AzureCAT.AzureEnhancedMonitoring extension

* Correct the variable name

* Add RHEL 90

* Add Kernel parameters for RHEL 9

* removed the package libssh2

* removed libpng12

* removed oddjobd service

* Add RHEL9 distro

* 3119751 only mentions RHEL8

* update packages

* Set NIC name for private endpoint for control plane

* Upgrade Azure CLI & Terraform when re-running

* Bump terrafom version

* removed  compat-sap-c++-*

* Dont register Microsoft keys

* USe CLI version 2.47

* Lock 0n 2.47 on Ubuntu 20.04

* Add llogging for CLI

* Show what version got installed

* Use grep to get the cli version

* Simplify re-install

* Ad more debug info on which agent is in use

* SELinux for RHEL 9

* Allow rerun for manual deployer

* Fix virtualenv on RHEL

* RHEL deployer updates

* Save the random ID value

* missing $

* Corrected the output variable

* use $$ in template

* show YAML version

* debug HANA on RHEL 9

* testing HANA install

* debugging HANA installation

* Add package chkconfig

* only install chkconfig on hana

* move flush handler

* Use azurerm backend if reinstall

* remove extra password parameter

* test password setting for HANA installation

* add password for

* use the correct password copy

* Only delete ini file if installation is successful

* moved chkconfig to 9.0 dictionary

* Add debugging

* Improve re-install experience

* reset firewall

* storage account fire wall

* export deployer_kv_user_arm_id

* update variable name

* Don't rerun the secrets setting

* Remove Oracle repos

* Add more images for RHEL

* update BoM Validator

* Added installation of devops extension in the main pipelines

* Provide the ability to control zonal ANF

* Add zonal validation for HANA log and shared

* Update zonality for sapmnt

* Add VMSS-Flex support

* Don't create PPG if VMSS-Flex is used

* Use the PPG ID and not the full object

* typo

* Fix AVG and scale set

* Don't create AVG if VMSS-Flex is used

* change resource name

* Update the provider

* 1.3 Repos: Add the HA repositories for RHEL

* 1.3 Repos: Add the HA repositories for RHEL

* app disk zonal logic

* add support for RHEL 9

* Add RHEL 9 to clustering

* Add documentation to output variables

* Introduce resource lock variables

* Control the secondary DNS registration

* Control the output

* DNS outputs

* Add dependencies and a wait

* Use the server counters

* Add resource locks

* don't save the randomizer value

* set the connection string secret via terraform

* update resource name

* update secret name

* shorten the names for the virtual hosts

* remove duplicate

* corrected local variable

* Check for key vault existence

* correct variable for vnet

* Update the NIC name for the sapmnt private endpoint

* Add scaleset and lock support to web app

* Use both identities

* simplify keyvault existence checks

* Shorten the virtual hostname for anydb

* The IP address are in different order in SLES and RHEL

* Check the hosts code

* debugging hosts

* change play_hosts to ansible_play_hosts

* update

* debug hosts file

* Remove the serialization

* Add support for premium storage v2

* Fixed caching

* Change endpoint name

* Premium storage V2

* Identity assignment

* HANA sizing

* disable SELinux on RHEL9

* Add a dependency on blob DNS link

* add check to verify If library deployment failed

* Add the scope

* Add the mandatory scope parameter

* Fix typo in command

* Add the scope to the extension code

* pass the variables for exports when the PAS and SCS are on the same server.

* Add the missing 's'

* Add AVG to Web App

* Add the correct packages

* Install compat-libcap1 package manually

* move the yum command

* typo in package name

* disable_gpg_check on the compat-libcap1

* Use builtin package handling

* Add distro info to HANA cluster tasks

* Add the permissions

* It is not a directory but a file

* removed duplicate package

* Add the SBP fixes

* Only do the ifup eth0 on RHEL

* ASM updates

* Misc updates

* SYBASE and RHEL

* Fix the profile name to sap-netweaver

* Add the virtual host for observer

* Experimental (#456)

* Added error categorization changes.

* PR for pushing commits in SDAF-ASCS prod-ansible-3.7.0 into sap-automation.

* Added names to the block.

* Fixed the spacing for variable expansion,

* Fixed the rescue indentation.

* Resolved merge conflicts for changes to the deploy/ansible/roles-db/4.0.0-hdb-install/tasks/main.yaml.

* Resolving Hemanth's comments and Reviews.

* Fixing linting issues.

* Fixed linting issues.

---------

Co-authored-by: shayaksarkar <[email protected]>, hdamecharla <[email protected]>

---------

Co-authored-by: Kimmo Forss <[email protected]>
Co-authored-by: hdamecharla <[email protected]>
Co-authored-by: ShayakSarkar <[email protected]>

* Provide several fixes for Terraform and Ansible (#465)

* Provide several fixes for Terraform code

* Fix internet check

* Fix owner of BOM directories

* Ignore PyCache directories

* Remove become from SDAF deployer check

* Bugfix dns_info_loadbalancers output variable for app_tier

* Add become to App installation block for Linux

* Create compact-sap-c++-10 symlink owner and group on the symlink, not the src file

The /opt/rh/SAP/lib64/compat-sap-c++-10.so src is provided by a Red Hat RPM. The owner / group should stay the same.

* sapmnt shouldn't be created on HANA systems

---------

Co-authored-by: Marges, RSY (Rick) <[email protected]>
Co-authored-by: Harm Jan Stam <[email protected]>

* Several fixes while testing with Ansible 2.14.6 (#474)

* Remove unsupported warn parameter from tasks

The warn parameter isn't supported anymore with newer Ansible versions.

* Bugfix grub config file content

The lookup module is executed on the controller and not on the remote host. Replace the lookup with a slurp.

* Bugfix BOM aggregate only find local bom folders

* Refactor curl command for Azure instance metadata to Ansible uri module

The Ansible uri module recognizes the json output returned by the azure metadata instance api. No converting to json is therefore needed anymore.

* Refactor anydb_node dns_info_vms output

* Remove how file

* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Revert "Merging changes made in v3.8.3.4 into experimental branch. (#477)"

This reverts commit e0f896055443237e7c102e9d61a66d47875cbe5e.

* removing unnecessary characters from japan region codes (#471)

* Merging changes made in v3.8.3.4 into experimental branch. (#477)

* removing unnecessary characters from japan region codes (#471)

* Merging changes from sdaf-acss main after creation of v3.8.3.4 (#475)

* Added README.md file

* Added file CONTRIBUTING.md

* Updated README.md

* Updated CONTRIBUTING.md

* Added .gitignore

* Adding owners.txt containing ServiceTree admins

* v3.6 release

* Updated owners.txt

* Current code for ansible 3.7 that is being tested by ACSS.

* Added a delay before validating the SCS ERS cluster.

* Removed the BOM-catalog folder.

* Merged PR 8320960: Merging sap-automation main into this repo's main branch.

This is done as part of the 3.8.3 parity task. This is an attempt to bring the Co-Dev model into effect so that all ansible development can now happen through the SDAF-ASCS repo.

Related work items: #24291278

* Merged PR 8350970: Fix Logical Volume creation

Fix Logical Volume creation:
Description:

In the volume group vg_sap, we need to create several logical volumes which include for lv_sapmnt, lv_usrsapinstall, lv_usrsap. The goal is to allocate a predetermined amount of space to lv_sapmnt, lv_usersap and then allocate the rest of the space to lv_usersapinstall. But the order in which this allocation was being done was in reverse which resulted in lv_usersapinstall consuming all the space in vg_sap therefore, blocking the creation of other logical volumes. The ordering has been fixed in this PR.

This is the test proof for the same:

Query to view the results for createinfra test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDQQyG7%2fsrwpwqbGFZQUGpILXCHvzAFXoQD2kmtaO7M0sm21Lwxzur0A8qzCFD3id5k8wR6wdWcTRbs9fsGzYrFoZnYXKRX13LtWLbwQ1YVNb0H5VFeT4uLsblJRTF1e8724F1v4gkrlMXfGXBeRhVXlk8NoepuCdeOIZeiJOagld0PoJBuwpkdpq71NfHgXsz8yBfyyZsHrFl8w6TCZipcDJX%2baVgVOlJe%2bH%2f4aeOBQcDaSod8MGgqXsiZsvW5GDu0TUpGgx2Ej6Z9GQb%2bVHNmUiQabD85yiHmmXtiG9J3drptrL5wZBD%2fJHAHPb51CqIZYHF9kh5cgWMdJ1lP74BzVW1AQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "CreateInfrastructure"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

Query to view the results for install test:

https://waasservices.eastus.kusto.windows.net/WaaSKustoDB?query=H4sIAAAAAAAEAG2RTUsDMRCG7%2fsrhpwqbGFZQUGpIFphD37gCj2Ih2ky2mg2WSazWwr%2beBOFflAhhwnv%2b%2bSdmSwQ23sStno%2bkpfiG9YrYoInJm0jvdiOWsGuhyswKCTpPqmr%2bnRanU3rc6iqi99zsgXbYRk1215s8I0B62HSeCH26PaluCOeKYaBNSW3Dl7Q%2bggKzSpotfXcplwfM%2feqFoG%2f3l1YP2BH6g1mM1CNj4LOZcUFNP9zjz0x5uw0kGQy96baQWsiQ0aVoO7QulTl3noOn6TlaBHlwZtz5sA3wdBfMyW0xKPVdK3FjlY2jSn35sv1RwJL2OkpKrAhhuXmwHn0ARj1ZVH8AGqZJGKwAQAA&web=0

WaaSMetricEvent
| where PreciseTimeStamp > datetime(2023-06-27 00:00:00)
| where SubscriptionId in (InternalSubscriptions)
| where ResourceId contains "adhoc"
| where Dimensions["WorkflowName"] == "InstallWorkload"
| where Dimensions["OperationState"] in ("Succeeded", "Failed")
| project PreciseTimeStamp, Dimensions["ErrorCodeName"], ServiceActivityId, ResourceId, Region, ActivityId
| order by ResourceId, PreciseTimeStamp asc;

1 SCS failure observed which was transient and succeeded on retry
These are the other errors observed which are not related to ansible:
InstallHostUnreachable
InstallServiceError
AzureVMIsNotInSupportedProvisioningState

Related work items: #24199283

* Merged PR 8352697: fix logical volume order with vg_sap

fix logical volume order with vg_sap

* Update owners.txt

* Remediating Azure Artifacts Configuration Issues in your Repository

* Merged PR 8383025: Getting latest release code from github repo into main branch.

Pulling in the changes made to the experimental branch so that the hotfix release (3.8.3.2) made in github public repo is present here as well

* Merged PR 8497623: Syncing SDAF Public main branch to SDAF-ACSS main branch

Syncing SDAF Public main branch to SDAF-ACSS main branch

Related work …
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants