Skip to content

Latest commit

 

History

History
155 lines (108 loc) · 6.03 KB

README.md

File metadata and controls

155 lines (108 loc) · 6.03 KB
Raw

A first look at mirror registry

This document provides information on how to use the new mirror-registry designed to provide a lightweight Quay for disconnected use case.

Environment

  • Any box where podman can run. We used CentOS8Stream

How To use

  1. Gather the binary at https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/mirror-registry/1.2.3/mirror-registry.tar.gz and untar it:

    curl -s -L https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/mirror-registry/1.2.3/mirror-registry.tar.gz | tar xvz -C /usr/bin

  2. Use it in a simple way:

    ℹ️ This will use an auto generated cert stored in /etc/quay-install and create a random password for the init user. You will have to use Podman with insecure mode or trust said certificate in order to interact with the resulting registry.

    /usr/bin/mirror-registry install

  3. You can also be more specific:

    • REGISTRY_NAME is the fqdn of your box
    • SSL_CERT is the path of a valid CA (as generated by openssl)
    • SSL_KEY is the path of the corresponding ssl key

    1. Generate the self-signed certificates:

      REGISTRY_NAME=$(hostname -f)
sudo mkdir -p /etc/quay-install/certs/
sudo chown -R $(id -u):$(id -g) /etc/quay-install

openssl req -x509 -newkey rsa:4096 -keyout /etc/quay-install/certs/registry-key.pem -nodes -out /etc/quay-install/certs/registry-cert.pem -sha256 -days 9999 -subj "/CN=${REGISTRY_NAME}" -addext "subjectAltName = DNS:${REGISTRY_NAME}"

    2. Run the registry install:

      ℹ️ The command below will initialize the registry with the user admin and the password n0tTh4tS3cur3.

      SSH_KEY="${HOME}/.ssh/id_rsa"
REGISTRY_USER=admin
REGISTRY_PASSWORD=n0tTh4tS3cur3
/usr/bin/mirror-registry install --quayHostname ${REGISTRY_NAME} --sslCert /etc/quay-install/certs/registry-cert.pem --sslKey /etc/quay-install/certs/registry-key.pem --initUser ${REGISTRY_USER} --initPassword ${REGISTRY_PASSWORD} --ssh-key ${SSH_KEY}

  4. Profit. One can validate the installation with the following command:

    ⚠️ If you don't have the self-signed certificate in your trust store you may need to add --tls-verify=false to the podman login command below.

    podman login -u ${REGISTRY_USER} -p ${REGISTRY_PASSWORD} ${REGISTRY_NAME}:8443

Development

In order to modify the registry, you will need:

  • git
  • make
  • podman
  • valid credentials for registry.redhat.io

The repository for the code of the registry is stored at https://github.com/quay/mirror-registry, one can create its own modified image by:

  1. Authenticating to registry.redhat.io from a valid pull secret

    PULL_SECRET="/root/openshift_pull.json"
REDHAT_CREDS=$(cat $OPENSHIFT_PULL_JSON | jq .auths.\"registry.redhat.io\".auth -r | base64 -d)
RHN_USER=$(echo $REDHAT_CREDS | cut -d: -f1)
RHN_PASSWORD=$(echo $REDHAT_CREDS | cut -d: -f2)
podman login -u "$RHN_USER" -p "$RHN_PASSWORD" registry.redhat.io

  2. creating a tar with all the contents(the dependency images get bundled):

    make build-online-zip

Limitations/Additional Notes

  • The command can also target a remote host, although we didn't see too much benefit to going with such approach.

  • IPv6 isn't currently supported by quay. For the time being, the folder IPv6 contains assets allowing to deploy an extra container in an IPv6 context. This can be deployed with the following instructions:

    ⚠️ Using HAProxy for providing IPv6 support on Quay is not supported by Red Hat, use at your own risk.

    cp quay_haproxy.cfg /etc/quay-install/haproxy.cfg
cp quay_haproxy.service /usr/lib/systemd/system
systemctl daemon-reload
systemctl enable --now quay-haproxy

Mirroring content with oc-mirror

One of the main use cases of a mirror registry is enabling OpenShift disconnected installations.

Installing the tool

As we mentioned, the tool is still not released and as such we need to build it ourselves.

  1. Get the latest stable build:

    curl -s -L https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/oc-mirror.tar.gz | sudo tar xvz -C /usr/bin

Configuring the mirror config file

The tool uses a yaml configuration file where you describe what you want to mirror, you have several examples in the repository.

Our configuration file looks as follows:

apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
storageConfig:
  registry:
    imageURL: registry.example.com:8443/ocp4/metadata:latest  # Registry where we will mirror the images
    skipTLS: true
mirror:
  platform:
    channels: # OCP Versions to mirror
      - name: stable-4.10
        minVersion: 4.10.10
        maxVersion: 4.10.20
    graph: true
  operators: # Operators we want to mirror
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.10
      packages:
        - name: local-storage-operator
        - name: advanced-cluster-management
        - name: openshift-gitops-operator
        - name: ocs-operator
        - name: multicluster-engine

Running the mirroring

Now that we have the config file ready we just need to run the mirror command. This command expects the user to be logged in the different registries that will be used. If you have a pull secret you can get it copied to ${XDG_RUNTIME_DIR}/containers/auth.json, and it will be consumed by the tool.

/usr/bin/cp -f pull_secret.json ${XDG_RUNTIME_DIR}/containers/auth.json

oc-mirror --config imageset-config.yaml docker://registry.example.com:8443 --dest-skip-tls

After a while we will have the content mirrored on our Quay registry!