Merge pull request #19 from stackhpc/os_images_merge

os_images role

roles/os_images/README.md

@@ -0,0 +1,150 @@
+OpenStack Images
+================
+
+This role generates guest instance images using disk-image-builder
+and uploads them to OpenStack using the `openstack.cloud.image` module.
+
+Requirements
+------------
+
+The OpenStack APIs should be accessible from the target host.
+Client credentials should have been set in the environment, or
+using the `clouds.yaml` format.
+
+You must use a virtualenv with system site packages enabled
+as this role relies on python packages installed by the package
+manager, e.g:
+
+```
+virtualenv --system-site-packages ~/venvs/dib
+```
+
+Role Variables
+--------------
+
+`os_images_package_dependencies_extra`: List of additional packages to install
+on the build host.
+
+`os_images_cache`: a path to a directory in which to cache build artefacts.
+It defaults to `~/disk_images`
+`NOTE`: new images will NOT be built, even if changes are made in config, if an image is already cached.
+Use `force_rebuild` flag in order to apply new config changes.
+
+`os_images_auth_type`: OpenStack authentication endpoint and credentials.
+Defaults to `password`.
+
+`os_images_auth`: OpenStack authentication endpoint and credentials.  For
+example, a dict of the form:
+* `auth_url`: Keystone auth endpoint URL.  Defaults to `OS_AUTH_URL`.
+* `project`: OpenStack tenant/project.  Defaults to `OS_TENANT_NAME`.
+* `username`: OpenStack username.  Defaults to `OS_USERNAME`.
+* `password`: OpenStack password.  Defaults to `OS_PASSWORD`.
+
+`os_images_region`: Define a region to upload the images.  Default is None.
+
+`os_images_cacert` is an optional path to a CA certificate bundle.
+
+`os_images_interface` is the endpoint URL type to fetch from the service
+catalog. Maybe be one of `public`, `admin`, or `internal`.
+
+`os_images_list` is a list of YAML dicts, where `elements` and `image_url` are
+mutually exclusive where each contain:
+* `name`: the image name to use in OpenStack.
+* `elements`: a list of diskimage-builder elements to incorporate into the image.
+* `image_url`: the URL to image location on the Internet.
+* `checksum`: Checksum to validate a downloaded image. Format: <algorithm>:<checksum|url>.
+* `env`: (optional) environment variables to define for diskimage-builder parameters.
+  This is a dict of the form of `KEY: VALUE`.
+* `packages`: (optional) list of packages to install in the image.
+* `size`: (optional) size to make the image filesystem.
+* `properties`: (optional) dict of properties to set on the glance image.
+  Common image properties are available
+  [here](https://docs.openstack.org/glance/latest/user/common-image-properties.html).
+* `type`: (optional) image type. Default in DIB is qcow2. Image formats are
+  available [here](https://docs.openstack.org/glance/latest/user/formats.html).
+* `force_rebuild`: (optional) boolean flag indicating whether or not the image should always
+  be built (even if an existing image that name has been built before). The images on glance
+  will be replaced if `os_images_upload` is set to `True`. This defaults to
+  `os_images_force_rebuild`if left unset.
+* `is_public`: (optional) whether the image should be set as visible to all
+  projects or kept private.
+* `owner`: (optional) ID of the project that should own the uploaded image.
+
+`os_images_common`: A set of elements to include in every image listed.
+Defaults to `cloud-init enable-serial-console stable-interface-names`.
+
+`os_images_dib_pkg_name`: Optionally customise the name parameter passed 
+to the ansible.builtin.pip module when installing diskimage-builder. This can
+be used to install diskimage-builder from version control.
+
+`os_images_dib_version`: Optionally set a version of diskimage-builder to install.
+By default this is not constrained.
+
+`os_images_git_elements`: An optional list of elements to pull from github, deploy
+locally for incorporation into the images.  Supply a list of dicts with the
+following parameters:
+* `repo`: URL to a git repo for cloning (if not already present)
+* `local`: local path for git cloning
+* `version`: optional git reference (branch, tag, hash) for cloning.  Defaults
+  to `HEAD`
+* `elements_path`: optional relative path to elements within the repository.
+
+`os_images_elements`: An optional list of paths for site-specific DIB elements.
+
+`os_images_upload`: Whether to upload built images to Glance. Defaults to `True`.
+
+`os_images_force_rebuild`: Whether or not to force a rebuild of the DIB image. The images on Glance
+will be replaced with the newly built image if `os_images_upload` is set to `True`. Defaults to
+`False`.
+
+`os_images_public`: Whether uploaded images are public. Defaults to `True` - note this requires admin permissions.
+
+`os_images_venv`: Path to virtualenv in which to install python dependencies to upload images.
+
+`os_images_dib_venv`: Path to virtualenv in which to install DIB to build images.
+
+`os_images_promote`: Whether or not to retire old and promote new images. Defaults to `False`.
+
+`os_images_build`: Whether or not to build the images.
+
+`os_images_name_suffix`: Image suffix which would be removed during image promotion, for exmple: -rc, -dev, -test etc. Mandatory for promotion functionality. Empty by default.
+
+`os_images_hide`: Whether or not to hide the images in Glance list. Hiding images is available as an option in image retirement/promotion process. Defaults to `False`.
+
+Dependencies
+------------
+
+Example Playbook
+----------------
+
+The following playbook generates a guest image and uploads it to OpenStack:
+
+    ---
+    - name: Generate guest image and upload
+      hosts: localhost
+      roles:
+        - role: stackhpc.openstack.os_images
+          os_images_auth:
+            auth_url:     "{{ lookup('env','OS_AUTH_URL') }}"
+            username:     "{{ lookup('env','OS_USERNAME') }}"
+            password:     "{{ lookup('env','OS_PASSWORD') }}"
+            project_name: "{{ lookup('env','OS_TENANT_NAME') }}"
+          os_images_list:
+          - name: FedoraCore
+            elements:
+              - fedora
+              - selinux-permissive
+              - alaska-extras
+            env:
+              DIB_ALASKA_DELETE_REPO: "y"
+              DIB_ALASKA_PKGLIST: "pam-python pam-keystone"
+          - name: FedoraAtomic27
+            image_url: https://ftp.icm.edu.pl/pub/Linux/dist/fedora-alt/atomic/stable/Fedora-Atomic-27-20180326.1/CloudImages/x86_64/images/Fedora-Atomic-27-20180326.1.x86_64.qcow2
+            properties:
+              os_distro: fedora-atomic
+            type: qcow2
+
+Author Information
+------------------
+
+- Stig Telfer (<[email protected]>)

roles/os_images/defaults/main.yml

@@ -0,0 +1,97 @@
+---
+# List of additional host package dependencies to install.
+os_images_package_dependencies_extra: []
+# Path to virtualenv in which to install python dependencies to upload images.
+os_images_venv:
+# Path to virtualenv in which to install DIB to build images.
+os_images_dib_venv: "{{ os_images_venv }}"
+
+# State of python dependencies.
+os_images_package_state: present
+
+# Upper constraints file for installation of python dependencies to upload
+# images.
+#
+# Use Antelope upper constraints as openstacksdk 1.0.1 and openstackclient 6.2.0 are required
+os_images_upper_constraints_file: https://releases.openstack.org/constraints/upper/2023.1
+
+# Upper constraints file for installation of DIB to build images.
+os_images_dib_upper_constraints_file: ""
+
+# Path to a directory in which to cache build artefacts.
+os_images_cache: "{{ lookup('env', 'HOME') }}/disk_images"
+
+# List of paths of site-specific diskimage-builder elements
+# os_images_elements: [ "{{ playbook_dir }}/stackhpc-image-elements" ]
+os_images_elements: []
+# Common elements to include in all images
+os_images_common: cloud-init enable-serial-console stable-interface-names
+
+# Example image definition:
+# os_images_list:
+#   - name: CentOS7-OFED4
+#     elements:
+#       - centos7
+#       - mlnx-ofed
+#     size: 3GB
+#     properties:
+#       os_distro: centos
+#     env:
+#       DIB_XXX: yyy
+#     packages:
+#       - biosdevname
+#     type: qcow2
+#   - name: FedoraAtomic27
+#     image_url: https://ftp.icm.edu.pl/pub/Linux/dist/fedora-alt/atomic/stable/Fedora-Atomic-27-20180326.1/CloudImages/x86_64/images/Fedora-Atomic-27-20180326.1.x86_64.qcow2
+#     properties:
+#       os_distro: fedora-atomic
+#     type: qcow2
+os_images_list: []
+# Common properties to apply to all glance images.
+os_images_common_properties:
+  cpu_arch: x86_64
+
+# OpenStack authentication type: passed to the os_image Ansible module
+os_images_auth_type: password
+
+# OpenStack authentication credentials:
+# For example, take OpenStack authentication credentials from the environment
+# os_images_auth:
+#   auth_url:     "{{ lookup('env','OS_AUTH_URL') }}"
+#   username:     "{{ lookup('env','OS_USERNAME') }}"
+#   password:     "{{ lookup('env','OS_PASSWORD') }}"
+#   project_name: "{{ lookup('env','OS_TENANT_NAME') }}"
+os_images_auth:
+# Endpoint URL type to fetch from the service catalog. Maybe be one of:
+# public, admin, or internal.
+os_images_interface:
+# Default region for OpenStack endpoint.
+os_images_region:
+# Pin to a specific version of diskimage-builder if required
+os_images_dib_version:
+# Customise the name parameter passed to the ansible.builtin.pip module.
+os_images_dib_pkg_name: diskimage-builder
+
+# List of git repositories containing site-specific diskimage-builder elements.
+# Each item should be a dict containing 'repo', 'local', and optionally,
+# 'version' items.
+os_images_git_elements: []
+# Whether to upload built images to Glance.
+os_images_upload: true
+
+# Whether or not the images should be rebuilt if they already exist
+os_images_force_rebuild: false
+
+# Whether images should be public (requires admin rights)
+os_images_public: true
+
+# Whether or not should old images be retired and new images be promoted
+os_images_promote: false
+
+# Whether or not to build the images
+os_images_build: true
+
+# Image suffix which would be removed during image promotion for exmple: -rc, -dev, -test
+os_images_name_suffix:
+# Whether or not to hide the images in Glance list
+os_images_hide: false

roles/os_images/meta/main.yml

@@ -0,0 +1,11 @@
+---
+dependencies:
+  - role: stackhpc.openstack.os_openstacksdk
+    os_openstacksdk_venv: "{{ os_images_venv }}"
+    os_openstacksdk_state: "{{ os_images_package_state }}"
+    os_openstacksdk_upper_constraints_file: "{{ os_images_upper_constraints_file }}"
+
+  - role: stackhpc.openstack.os_openstackclient
+    os_openstackclient_venv: "{{ os_images_venv }}"
+    os_openstackclient_state: "{{ os_images_package_state }}"
+    os_openstackclient_upper_constraints_file: "{{ os_images_upper_constraints_file }}"