Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Docker impl #127

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

birkland
Copy link

Draft PR to demonstrate Islandora 8 in Docker

Creates and runs two Docker containers:

  • An ansible container that, upon startup, runs ansible to install Islandora 8 to the vm container
  • A vm container that plays the role of an empty Centos 7 box upon which Islandora 8 is installed via the ansible container.

Motivation

  • Providing the start of a development environment that would allow modeling of "services deployed on separate hosts" as containers.
    • Right now, though, everything is installed to a single container whose role is analogous to the Vagrant VM. It's a start, though.
  • Because using Ansible + Vagrant + Virtualbox in Windows is possible, but impractically awkward

To Test

  • Make sure Docker is installed, as well as docker-compose (usually comes with the Docker distribution, e.g on Mac and Windows)
  • Make sure the Islandora ports are free on your machine (e.g. there isn't another Islandora running on the same ports using Vagrant)
  • Check out this PR, and in the repository's directory (claw-playbook/) do docker-compose build
  • Do docker-compose up -d
  • Watch the logs of the ansible container via docker logs -f ansible. That's the ansible output!
  • At this point, the only way to know it's done/ready is to look at the logs. So watch the logs until Ansible is finished
    • Once the Ansible stops, the container will just run infinitely tailing /dev/null. The container won't exit if it succeeds or fails. It runs infinitely because that has been helpful for me in debugging Ansible issues.
  • If Ansible succeeds, try logging into Islandora at the usual address localhost:8000, and poke around!

Tips and tricks

  • You can hop onto any container while it's running by docker exec -it /bin/bash. For example, docker exec -it vm /bin/bash
  • docker-compose stop will stop all containers, but keep state. You can later on do a docker-compose up vm just to start the Islandora "vm" container.
    • I haven't yet put the persisted state into volumes, so files/state are just written in the container. As a result, all state disappears in a poof once the containers are destroyed, e.g. via docker-compose down. docker-compose stop preserves state.

See if this works for you @dannylamb, or anybody else interested. Have fun!

@dannylamb
Copy link
Member

@bseeger ^^

You seein' this? Looks like a viable way to re-use our ansible provisioners and not have to write a dockerfile for every role we've already developed.

@bseeger
Copy link
Member

bseeger commented Aug 5, 2019

Wow! Cool! Thanks, @birkland, for getting this ball rolling. @dwk2, check this out.

@dwk2
Copy link

dwk2 commented Aug 5, 2019

@birkland : Woot! I'm so excited to see your thinking here! Dockerizing I8 is scheduled to be part of the next Islandora webinar on Aug 13th. Would you be interested in joining a planning meeting with @dannylamb, @bseeger + others? (I will reach out to you via email.) Also, we have reached the halfway mark in our crowdfunding goal to refactor ISLE for use with I8. ISLE's committers would love to talk I8 with you. (ISLE has solved issues of data persistence using bind mounts and docker volumes.)

@dannylamb
Copy link
Member

@birkland Yep, totally works. I have a CENTOS container running Islandora 8 using our Ansible provisioner!

@DonRichards
Copy link
Member

Removed my comments, looks like my issue is related to Islandora/documentation#1245

@attiks
Copy link

attiks commented Jan 24, 2020

I tried running this for a new project, but I always end up with a composer error

Your requirements could not be resolved to an installable set of packages. Problem 1 - Installation request for islandora/islandora_defaults 1.0.0 -> satisfiable by islandora/islandora_defaults[1.0.0]. - islandora/islandora_defaults 1.0.0 requires islandora/islandora 1.0.0 -> no matching package found

I've tried the following, but still the same error

drupal_composer_dependencies:
  - "islandora/islandora:dev-8.x-1.x"
  - "islandora/islandora_defaults:dev-8.x-1.x"

@attiks
Copy link

attiks commented Jan 24, 2020

Last output

ansible    | changed: [default] => (item=islandora/islandora_defaults:dev-8.x-1.x)
ansible    | ok: [default] => (item=islandora/carapace:1.0.0)
ansible    | 	to retry, use: --limit @/root/playbook/playbook.retry
ansible    | 
ansible    | PLAY RECAP *********************************************************************
ansible    | default                    : ok=121  changed=1    unreachable=0    failed=1   
ansible    | 
ansible    | Friday 24 January 2020  15:13:37 +0000 (0:06:24.148)       0:11:40.685 ******** 
ansible    | =============================================================================== 
ansible    | geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 384.15s
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 227.68s
ansible    | geerlingguy.php-mysql : Install PHP MySQL dependencies (RedHat). ------- 17.58s
ansible    | install extra packages centos ------------------------------------------- 6.85s
ansible    | gather facts ------------------------------------------------------------ 4.22s
ansible    | Gathering Facts --------------------------------------------------------- 4.03s
ansible    | Gathering Facts --------------------------------------------------------- 4.00s
ansible    | Gathering Facts --------------------------------------------------------- 3.95s
ansible    | geerlingguy.repo-remi : Import remi GPG key. ---------------------------- 1.79s
ansible    | geerlingguy.solr : Ensure dependencies are installed. ------------------- 1.66s
ansible    | geerlingguy.php-versions : Define PHP variables. ------------------------ 1.60s
ansible    | geerlingguy.git : Ensure git is installed (RedHat). --------------------- 1.29s
ansible    | geerlingguy.mysql : Ensure MySQL is started and enabled on boot. -------- 1.10s
ansible    | geerlingguy.mysql : Copy my.cnf global MySQL configuration. ------------- 1.07s
ansible    | geerlingguy.apache : Ensure Apache has selected state and enabled on boot. --- 0.98s
ansible    | geerlingguy.solr : Ensure solr is started and enabled on boot if configured. --- 0.86s
ansible    | geerlingguy.solr : Apply Solr configuration changes. -------------------- 0.81s
ansible    | geerlingguy.repo-remi : Install remi repo. ------------------------------ 0.76s
ansible    | geerlingguy.solr : Ensure solr_user exists. ----------------------------- 0.75s
ansible    | geerlingguy.mysql : Remove MySQL test database. ------------------------- 0.66s

http://localhost:8000/core/install.php is accessible but complains about

  • Configuration directory: sync
  • File system
  • The Settings file is not writable.

@birkland
Copy link
Author

@attiks I can try to reproduce this in a little while. This PR is against master, which hasn't changed in a while. It's unclear to me what sources of instability can affect a playbook's ability to make a repeatable build.

In any case, I can see if this can be reproduced on my end. I also have a branch against dev that I could open a PR for, if you wanted to build against that. That being said, on the tech call, folks were mentioning some issues with the tip of dev preventing successful builds at the moment, so I'm not sure if it's working now or not.

@attiks
Copy link

attiks commented Jan 24, 2020

I've made some progress

Changes

diff --git a/docker/ansible/Dockerfile b/docker/ansible/Dockerfile
index 1ab5c73..281a536 100644
--- a/docker/ansible/Dockerfile
+++ b/docker/ansible/Dockerfile
@@ -6,7 +6,9 @@ ENV ISLANDORA_DISTRO="centos/7" \
 WORKDIR /root/playbook
 
 RUN yum -y install epel-release && \
-    yum -y install git ansible openssh-clients
+    yum -y install git openssh-clients && \
+    yum -y install python-pip python-dev && \
+    pip install ansible==2.7.10
 
 COPY entrypoint.sh /bin/
 
diff --git a/inventory/docker/group_vars/webserver/drupal.yml b/inventory/docker/group_vars/webserver/drupal.yml
index 80018b3..5a83aca 100644
--- a/inventory/docker/group_vars/webserver/drupal.yml
+++ b/inventory/docker/group_vars/webserver/drupal.yml
@@ -14,9 +14,11 @@ drupal_composer_dependencies:
   - "drupal/content_browser:^1.0@alpha"
   - "drupal/matomo:^1.7"
   - "drupal/pdf:1.x-dev"
+  - "islandora/openseadragon:dev-8.x-1.x"
+  - "islandora/islandora:dev-8.x-1.x"
+  - "islandora/islandora_defaults:dev-8.x-1.x"
   - "islandora/carapace:1.0.0"
-  - "islandora/islandora_defaults:1.0.0"
-drupal_composer_project_package: "islandora/drupal-project:8.6.10"
+drupal_composer_project_package: "islandora/drupal-project:dev-no-auto-scaffold"
 drupal_composer_project_options: "--prefer-dist --stability dev --no-interaction"
 drupal_core_path: "{{ drupal_composer_install_dir }}/web"
 drupal_db_user: drupal8
diff --git a/inventory/vagrant/group_vars/webserver/drupal.yml b/inventory/vagrant/group_vars/webserver/drupal.yml
index 80018b3..e5b2533 100644
--- a/inventory/vagrant/group_vars/webserver/drupal.yml
+++ b/inventory/vagrant/group_vars/webserver/drupal.yml
@@ -14,9 +14,11 @@ drupal_composer_dependencies:
   - "drupal/content_browser:^1.0@alpha"
   - "drupal/matomo:^1.7"
   - "drupal/pdf:1.x-dev"
+  - "islandora/openseadragon:dev-8.x-1.x"
+  - "islandora/islandora:dev-8.x-1.x
+  - "islandora/islandora_defaults:dev-8.x-1.x"
   - "islandora/carapace:1.0.0"
-  - "islandora/islandora_defaults:1.0.0"
-drupal_composer_project_package: "islandora/drupal-project:8.6.10"
+drupal_composer_project_package: "islandora/drupal-project:8.8.1"
 drupal_composer_project_options: "--prefer-dist --stability dev --no-interaction"
 drupal_core_path: "{{ drupal_composer_install_dir }}/web"
 drupal_db_user: drupal8
diff --git a/webserver.yml b/webserver.yml
index f799879..6288785 100644
--- a/webserver.yml
+++ b/webserver.yml
@@ -5,6 +5,7 @@
 
   vars:
     php_version: "7.2"
+    php_memory_limit: "2G"
 
   roles:
     - name: geerlingguy.repo-remi

Changed roles/external/Islandora-Devops.drupal-openseadragon/defaults/main.yml so it uses

openseadragon_composer_item: "islandora/openseadragon:dev-8.x-1.x"

Fails now on

ansible    | TASK [webserver-app : Import features] *****************************************
ansible    | Friday 24 January 2020  16:50:23 +0000 (0:00:00.456)       0:12:06.941 ******** 
ansible    | fatal: [default]: FAILED! => {"changed": true, "cmd": ["/var/www/html/drupal/vendor/drush/drush/drush", "--root", "/var/www/html/drupal/web", "-y", "fim", "islandora_core_feature,controlled_access_terms_defaults"], "delta": "0:00:00.486601", "end": "2020-01-24 16:50:24.658819", "msg": "non-zero return code", "rc": 1, "start": "2020-01-24 16:50:24.172218", "stderr": "\n                                 \n  Command \"fim\" is not defined.  \n                                 \n  Did you mean this?             \n      cim                        \n                                 ", "stderr_lines": ["", "                                 ", "  Command \"fim\" is not defined.  ", "                                 ", "  Did you mean this?             ", "      cim                        ", "                                 "], "stdout": "", "stdout_lines": []}
ansible    | 	to retry, use: --limit @/root/playbook/playbook.retry
ansible    | 
ansible    | PLAY RECAP *********************************************************************
ansible    | default                    : ok=161  changed=12   unreachable=0    failed=1   
ansible    | 
ansible    | Friday 24 January 2020  16:50:24 +0000 (0:00:00.873)       0:12:07.815 ******** 
ansible    | =============================================================================== 
ansible    | geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 355.21s
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 226.04s
ansible    | Islandora-Devops.drupal-openseadragon : Install Openseadragon with composer -- 22.87s
ansible    | geerlingguy.php-mysql : Install PHP MySQL dependencies (RedHat). ------- 13.92s
ansible    | install extra packages centos ------------------------------------------ 13.35s
ansible    | gather facts ------------------------------------------------------------ 4.75s
ansible    | Gathering Facts --------------------------------------------------------- 4.14s
ansible    | Gathering Facts --------------------------------------------------------- 4.11s
ansible    | Islandora-Devops.drupal-openseadragon : Unarchive seadragon library ----- 3.99s
ansible    | Gathering Facts --------------------------------------------------------- 3.95s
ansible    | Islandora-Devops.drupal-openseadragon : Enable modules ------------------ 3.51s
ansible    | webserver-app : Uninstall core search module ---------------------------- 2.76s
ansible    | webserver-app : Enable Carapace if not enabled -------------------------- 2.36s
ansible    | geerlingguy.solr : Ensure dependencies are installed. ------------------- 1.52s
ansible    | geerlingguy.php-versions : Define PHP variables. ------------------------ 1.50s
ansible    | geerlingguy.repo-remi : Import remi GPG key. ---------------------------- 1.45s
ansible    | geerlingguy.solr : Ensure dependencies are installed. ------------------- 1.43s
ansible    | webserver-app : Set theme to carapace ----------------------------------- 1.37s
ansible    | geerlingguy.mysql : Ensure MySQL is started and enabled on boot. -------- 1.25s
ansible    | webserver-app : Get Carapace status ------------------------------------- 1.02s

http://localhost:8000/ displays the drupal site

@dannylamb
Copy link
Member

That's a problem with the build we're experiencing on dev right now. Totally orthogonal to docker.

TL;DR we have patch features. I'm testing it now.

@attiks
Copy link

attiks commented Jan 25, 2020

A mbit of pregress

diff --git a/inventory/vagrant/group_vars/webserver/drupal.yml b/inventory/vagrant/group_vars/webserver/drupal.yml
index 80018b3..ce74178 100644
--- a/inventory/vagrant/group_vars/webserver/drupal.yml
+++ b/inventory/vagrant/group_vars/webserver/drupal.yml
@@ -7,6 +7,7 @@ drupal_composer_dependencies:
   - "zaporylie/composer-drupal-optimizations:^1.0"
   - "drupal/console:~1.0"
   - "drupal/devel:^2.0"
+  - "drupal/features:^3.8"
   - "drupal/rdfui:^1.0-beta1"
   - "drupal/restui:^1.16"
   - "drupal/search_api_solr:^2.0"
@@ -14,9 +15,12 @@ drupal_composer_dependencies:
   - "drupal/content_browser:^1.0@alpha"
   - "drupal/matomo:^1.7"
   - "drupal/pdf:1.x-dev"
+  - "islandora/openseadragon:dev-8.x-1.x"
+  - "islandora/islandora:dev-8.x-1.x
+  - "islandora/islandora_defaults:dev-8.x-1.x"
+  - "islandora/controlled_access_terms:dev-8.x-1.x"
   - "islandora/carapace:1.0.0"
-  - "islandora/islandora_defaults:1.0.0"
-drupal_composer_project_package: "islandora/drupal-project:8.6.10"
+drupal_composer_project_package: "islandora/drupal-project:8.8.1"
 drupal_composer_project_options: "--prefer-dist --stability dev --no-interaction"
 drupal_core_path: "{{ drupal_composer_install_dir }}/web"
 drupal_db_user: drupal8
@@ -28,6 +32,7 @@ drupal_site_name: "Islandora 8"
 drupal_install_profile: standard
 drupal_account_name: admin
 drupal_enable_modules:
+  - features
   - rdf
   - responsive_image
   - syslog
@@ -43,8 +48,9 @@ drupal_enable_modules:
   - content_browser
   - matomo
   - pdf
-  - islandora_core_feature
   - controlled_access_terms_defaults
+  - controlled_access_terms
+  - islandora_core_feature
 drupal_trusted_hosts:
   - ^localhost$
 drupal_trusted_hosts_file: "{{ drupal_core_path }}/sites/default/settings.php"

Output

ansible    | TASK [webserver-app : Import features] *****************************************
ansible    | Saturday 25 January 2020  12:37:16 +0000 (0:00:00.354)       0:10:52.107 ****** 
ansible    | fatal: [default]: FAILED! => {"changed": true, "cmd": ["/var/www/html/drupal/vendor/drush/drush/drush", "--root", "/var/www/html/drupal/web", "-y", "fim", "islandora_core_feature,controlled_access_terms_defaults"], "delta": "0:00:01.548341", "end": "2020-01-25 12:37:18.793714", "msg": "non-zero return code", "rc": 1, "start": "2020-01-25 12:37:17.245373", "stderr": "\nIn FeaturesCommands.php line 763:\n                                                                  \n  No such feature is installed: controlled_access_terms_defaults  \n                                                                  \n\nfeatures:import [--bundle [BUNDLE]] [--force [FORCE]] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-d|--debug] [-y|--yes] [--no] [--remote-host REMOTE-HOST] [--remote-user REMOTE-USER] [-r|--root ROOT] [-l|--uri URI] [--simulate] [--pipe] [-D|--define DEFINE] [--notify [NOTIFY]] [--xh-link XH-LINK] [--druplicon] [--] <command> <feature>", "stderr_lines": ["", "In FeaturesCommands.php line 763:", "                                                                  ", "  No such feature is installed: controlled_access_terms_defaults  ", "                                                                  ", "", "features:import [--bundle [BUNDLE]] [--force [FORCE]] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-d|--debug] [-y|--yes] [--no] [--remote-host REMOTE-HOST] [--remote-user REMOTE-USER] [-r|--root ROOT] [-l|--uri URI] [--simulate] [--pipe] [-D|--define DEFINE] [--notify [NOTIFY]] [--xh-link XH-LINK] [--druplicon] [--] <command> <feature>"], "stdout": "\n // Do you really want to import islandora_core_feature :                       \n // taxonomy.vocabulary.islandora_media_use?: yes.                              \n\n // Do you really want to import islandora_core_feature :                       \n // taxonomy.vocabulary.islandora_models?: yes.                                 \n\n // Do you really want to import islandora_core_feature :                       \n // views.view.display_media?: yes.                                             \n\n // Do you really want to import islandora_core_feature : views.view.media_of?: \n // yes.                                                                        ", "stdout_lines": ["", " // Do you really want to import islandora_core_feature :                       ", " // taxonomy.vocabulary.islandora_media_use?: yes.                              ", "", " // Do you really want to import islandora_core_feature :                       ", " // taxonomy.vocabulary.islandora_models?: yes.                                 ", "", " // Do you really want to import islandora_core_feature :                       ", " // views.view.display_media?: yes.                                             ", "", " // Do you really want to import islandora_core_feature : views.view.media_of?: ", " // yes.                                                                        "]}
ansible    | 	to retry, use: --limit @/root/playbook/playbook.retry
ansible    | 
ansible    | PLAY RECAP *********************************************************************
ansible    | default                    : ok=154  changed=1    unreachable=0    failed=1   
ansible    | 
ansible    | Saturday 25 January 2020  12:37:18 +0000 (0:00:01.866)       0:10:53.973 ****** 
ansible    | =============================================================================== 
ansible    | geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 342.56s
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 191.36s
ansible    | Islandora-Devops.drupal-openseadragon : Install Openseadragon with composer -- 21.58s
ansible    | geerlingguy.php-mysql : Install PHP MySQL dependencies (RedHat). ------- 12.73s
ansible    | install extra packages centos ------------------------------------------- 8.87s
ansible    | gather facts ------------------------------------------------------------ 4.21s
ansible    | Gathering Facts --------------------------------------------------------- 3.78s
ansible    | Gathering Facts --------------------------------------------------------- 3.62s
ansible    | Gathering Facts --------------------------------------------------------- 3.57s
ansible    | webserver-app : Import features ----------------------------------------- 1.87s
ansible    | geerlingguy.php-versions : Define PHP variables. ------------------------ 1.42s
ansible    | geerlingguy.solr : Ensure dependencies are installed. ------------------- 1.25s
ansible    | geerlingguy.repo-remi : Import remi GPG key. ---------------------------- 1.25s
ansible    | geerlingguy.mysql : Ensure MySQL is started and enabled on boot. -------- 1.17s
ansible    | geerlingguy.solr : Ensure dependencies are installed. ------------------- 1.16s
ansible    | geerlingguy.git : Ensure git is installed (RedHat). --------------------- 0.97s
ansible    | webserver-app : Test if theme is Carapace ------------------------------- 0.95s
ansible    | geerlingguy.mysql : Copy my.cnf global MySQL configuration. ------------- 0.90s
ansible    | webserver-app : Get Carapace status ------------------------------------- 0.89s
ansible    | webserver-app : Uninstall core search module ---------------------------- 0.87s

@attiks
Copy link

attiks commented Jan 25, 2020

More progress, but a bit of cheating

Diff

diff --git a/docker/ansible/Dockerfile b/docker/ansible/Dockerfile
index 1ab5c73..281a536 100644
--- a/docker/ansible/Dockerfile
+++ b/docker/ansible/Dockerfile
@@ -6,7 +6,9 @@ ENV ISLANDORA_DISTRO="centos/7" \
 WORKDIR /root/playbook
 
 RUN yum -y install epel-release && \
-    yum -y install git ansible openssh-clients
+    yum -y install git openssh-clients && \
+    yum -y install python-pip python-dev && \
+    pip install ansible==2.7.10
 
 COPY entrypoint.sh /bin/
 
diff --git a/inventory/docker/group_vars/webserver/drupal.yml b/inventory/docker/group_vars/webserver/drupal.yml
index 80018b3..4500410 100644
--- a/inventory/docker/group_vars/webserver/drupal.yml
+++ b/inventory/docker/group_vars/webserver/drupal.yml
@@ -7,6 +7,7 @@ drupal_composer_dependencies:
   - "zaporylie/composer-drupal-optimizations:^1.0"
   - "drupal/console:~1.0"
   - "drupal/devel:^2.0"
+  - "drupal/features:^3.8"
   - "drupal/rdfui:^1.0-beta1"
   - "drupal/restui:^1.16"
   - "drupal/search_api_solr:^2.0"
@@ -14,9 +15,12 @@ drupal_composer_dependencies:
   - "drupal/content_browser:^1.0@alpha"
   - "drupal/matomo:^1.7"
   - "drupal/pdf:1.x-dev"
+  - "islandora/openseadragon:dev-8.x-1.x"
+  - "islandora/islandora:dev-8.x-1.x"
+  - "islandora/islandora_defaults:dev-8.x-1.x"
+  - "islandora/controlled_access_terms:dev-8.x-1.x"
   - "islandora/carapace:1.0.0"
-  - "islandora/islandora_defaults:1.0.0"
-drupal_composer_project_package: "islandora/drupal-project:8.6.10"
+drupal_composer_project_package: "islandora/drupal-project:dev-no-auto-scaffold"
 drupal_composer_project_options: "--prefer-dist --stability dev --no-interaction"
 drupal_core_path: "{{ drupal_composer_install_dir }}/web"
 drupal_db_user: drupal8
@@ -28,6 +32,7 @@ drupal_site_name: "Islandora 8"
 drupal_install_profile: standard
 drupal_account_name: admin
 drupal_enable_modules:
+  - features
   - rdf
   - responsive_image
   - syslog
@@ -43,8 +48,9 @@ drupal_enable_modules:
   - content_browser
   - matomo
   - pdf
-  - islandora_core_feature
   - controlled_access_terms_defaults
+  - controlled_access_terms
+  - islandora_core_feature
 drupal_trusted_hosts:
   - ^localhost$
 drupal_trusted_hosts_file: "{{ drupal_core_path }}/sites/default/settings.php"
diff --git a/inventory/vagrant/group_vars/webserver/drupal.yml b/inventory/vagrant/group_vars/webserver/drupal.yml
index 80018b3..ce74178 100644
--- a/inventory/vagrant/group_vars/webserver/drupal.yml
+++ b/inventory/vagrant/group_vars/webserver/drupal.yml
@@ -7,6 +7,7 @@ drupal_composer_dependencies:
   - "zaporylie/composer-drupal-optimizations:^1.0"
   - "drupal/console:~1.0"
   - "drupal/devel:^2.0"
+  - "drupal/features:^3.8"
   - "drupal/rdfui:^1.0-beta1"
   - "drupal/restui:^1.16"
   - "drupal/search_api_solr:^2.0"
@@ -14,9 +15,12 @@ drupal_composer_dependencies:
   - "drupal/content_browser:^1.0@alpha"
   - "drupal/matomo:^1.7"
   - "drupal/pdf:1.x-dev"
+  - "islandora/openseadragon:dev-8.x-1.x"
+  - "islandora/islandora:dev-8.x-1.x
+  - "islandora/islandora_defaults:dev-8.x-1.x"
+  - "islandora/controlled_access_terms:dev-8.x-1.x"
   - "islandora/carapace:1.0.0"
-  - "islandora/islandora_defaults:1.0.0"
-drupal_composer_project_package: "islandora/drupal-project:8.6.10"
+drupal_composer_project_package: "islandora/drupal-project:8.8.1"
 drupal_composer_project_options: "--prefer-dist --stability dev --no-interaction"
 drupal_core_path: "{{ drupal_composer_install_dir }}/web"
 drupal_db_user: drupal8
@@ -28,6 +32,7 @@ drupal_site_name: "Islandora 8"
 drupal_install_profile: standard
 drupal_account_name: admin
 drupal_enable_modules:
+  - features
   - rdf
   - responsive_image
   - syslog
@@ -43,8 +48,9 @@ drupal_enable_modules:
   - content_browser
   - matomo
   - pdf
-  - islandora_core_feature
   - controlled_access_terms_defaults
+  - controlled_access_terms
+  - islandora_core_feature
 drupal_trusted_hosts:
   - ^localhost$
 drupal_trusted_hosts_file: "{{ drupal_core_path }}/sites/default/settings.php"
diff --git a/roles/internal/webserver-app/tasks/drupal.yml b/roles/internal/webserver-app/tasks/drupal.yml
index 608b035..067412f 100644
--- a/roles/internal/webserver-app/tasks/drupal.yml
+++ b/roles/internal/webserver-app/tasks/drupal.yml
@@ -60,7 +60,7 @@
     group: "{{ webserver_app_user }}"
 
 - name: Import features
-  command: "{{ drush_path }} --root {{ drupal_core_path }} -y fim islandora_core_feature,controlled_access_terms_defaults"
+  command: "{{ drush_path }} --root {{ drupal_core_path }} -y fim islandora_core_feature"
 
 # masonry library is required by content_browser and not installed by composer due to issue 2971165.
 - name: Create drupal library directory.
@@ -82,17 +82,6 @@
   args:
     creates: "{{ drupal_external_libraries_directory }}/masonry"
 
-# Set Matomo Settings
-- name: Set Matomo site id.
-  command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set matomo.settings site_id 1"
-  register: set_search_api_config
-  changed_when: "'Do you want to update site_id' in set_search_api_config.stdout"
-
-- name: Set Matmo sever url.
-  command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set matomo.settings url_http http://localhost:8000/matomo/"
-  register: set_search_api_config
-  changed_when: "'Do you want to update site_id' in set_search_api_config.stdout"
-
 # pdf.js library
 - name: ensure pdf.js directory exists
   file:
diff --git a/roles/internal/webserver-app/tasks/solr.yml b/roles/internal/webserver-app/tasks/solr.yml
index cae8a22..66fb78a 100644
--- a/roles/internal/webserver-app/tasks/solr.yml
+++ b/roles/internal/webserver-app/tasks/solr.yml
@@ -1,9 +1,9 @@
 ---
 
-- name: Set default solr server to point to CLAW core
-  command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set search_api.server.default_solr_server backend_config.connector_config.core CLAW"
-  register: set_search_api_config
-  changed_when: "'Do you want to update' in set_search_api_config.stdout"
+#- name: Set default solr server to point to CLAW core
+#  command: "{{ drush_path }} --root {{ drupal_core_path }} -y config-set search_api.server.default_solr_server backend_config.connector_config.core CLAW"
+#  register: set_search_api_config
+#  changed_when: "'Do you want to update' in set_search_api_config.stdout"
 
 - name: Get solr config files to copy
   command: "find {{ webserver_document_root }}/drupal/web/modules/contrib/search_api_solr/solr-conf/7.x -type f"
diff --git a/webserver.yml b/webserver.yml
index f799879..6288785 100644
--- a/webserver.yml
+++ b/webserver.yml
@@ -5,6 +5,7 @@
 
   vars:
     php_version: "7.2"
+    php_memory_limit: "2G"
 
   roles:
     - name: geerlingguy.repo-remi

Changes in untracked files

roles/external/Islandora-Devops.drupal-openseadragon/defaults/main.yml
openseadragon_composer_item: "islandora/openseadragon:dev-8.x-1.x"

roles/external/Islandora-Devops.blazegraph/tasks/download.yml
url: https://repo1.maven.org/maven2/com/blazegraph/bigdata-war/{{ blazegraph_version }}/bigdata-war-{{ blazegraph_version }}.war

Output

ansible    | TASK [Islandora-Devops.alpaca : Add Alpaca Repos] ******************************
ansible    | Saturday 25 January 2020  16:11:52 +0000 (0:00:00.051)       0:26:18.353 ****** 
ansible    | failed: [default] (item=mvn:org.apache.camel.karaf/apache-camel/2.20.4/xml/features) => {"changed": false, "item": "mvn:org.apache.camel.karaf/apache-camel/2.20.4/xml/features", "msg": "client: Ignoring predefined value for KARAF_HOME\nAdding feature url mvn:org.apache.camel.karaf/apache-camel/2.20.4/xml/features\nError executing command: Error resolving artifact org.apache.camel.karaf:apache-camel:xml:features:2.20.4: [Could not transfer artifact org.apache.camel.karaf:apache-camel:xml:features:2.20.4 from/to central (http://repo1.maven.org/maven2/): Failed to transfer file: http://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/2.20.4/apache-camel-2.20.4-features.xml. Return code is: 501 , ReasonPhrase:HTTPS Required., Could not find artifact org.apache.camel.karaf:apache-camel:xml:features:2.20.4 in spring.ebr.release (http://repository.springsource.com/maven/bundles/release/), Could not find artifact org.apache.camel.karaf:apache-camel:xml:features:2.20.4 in spring.ebr.external (http://repository.springsource.com/maven/bundles/external/), Could not transfer artifact org.apache.camel.karaf:apache-camel:xml:features:2.20.4 from/to gemini (http://zodiac.springsource.com/maven/bundles/release/): zodiac.springsource.com: Name or service not known] : mvn:org.apache.camel.karaf/apache-camel/2.20.4/xml/features\n"}
ansible    | failed: [default] (item=mvn:org.apache.activemq/activemq-karaf/5.15.0/xml/features) => {"changed": false, "item": "mvn:org.apache.activemq/activemq-karaf/5.15.0/xml/features", "msg": "client: Ignoring predefined value for KARAF_HOME\nAdding feature url mvn:org.apache.activemq/activemq-karaf/5.15.0/xml/features\nError executing command: Error resolving artifact org.apache.activemq:activemq-karaf:xml:features:5.15.0: [Could not transfer artifact org.apache.activemq:activemq-karaf:xml:features:5.15.0 from/to central (http://repo1.maven.org/maven2/): Failed to transfer file: http://repo1.maven.org/maven2/org/apache/activemq/activemq-karaf/5.15.0/activemq-karaf-5.15.0-features.xml. Return code is: 501 , ReasonPhrase:HTTPS Required., Could not find artifact org.apache.activemq:activemq-karaf:xml:features:5.15.0 in spring.ebr.release (http://repository.springsource.com/maven/bundles/release/), Could not find artifact org.apache.activemq:activemq-karaf:xml:features:5.15.0 in spring.ebr.external (http://repository.springsource.com/maven/bundles/external/), Could not transfer artifact org.apache.activemq:activemq-karaf:xml:features:5.15.0 from/to gemini (http://zodiac.springsource.com/maven/bundles/release/): zodiac.springsource.com] : mvn:org.apache.activemq/activemq-karaf/5.15.0/xml/features\n"}
ansible    | failed: [default] (item=mvn:ca.islandora.alpaca/islandora-karaf/1.0.1/xml/features) => {"changed": false, "item": "mvn:ca.islandora.alpaca/islandora-karaf/1.0.1/xml/features", "msg": "client: Ignoring predefined value for KARAF_HOME\nAdding feature url mvn:ca.islandora.alpaca/islandora-karaf/1.0.1/xml/features\nError executing command: Error resolving artifact ca.islandora.alpaca:islandora-karaf:xml:features:1.0.1: [Could not transfer artifact ca.islandora.alpaca:islandora-karaf:xml:features:1.0.1 from/to central (http://repo1.maven.org/maven2/): Failed to transfer file: http://repo1.maven.org/maven2/ca/islandora/alpaca/islandora-karaf/1.0.1/islandora-karaf-1.0.1-features.xml. Return code is: 501 , ReasonPhrase:HTTPS Required., Could not find artifact ca.islandora.alpaca:islandora-karaf:xml:features:1.0.1 in spring.ebr.release (http://repository.springsource.com/maven/bundles/release/), Could not find artifact ca.islandora.alpaca:islandora-karaf:xml:features:1.0.1 in spring.ebr.external (http://repository.springsource.com/maven/bundles/external/), Could not transfer artifact ca.islandora.alpaca:islandora-karaf:xml:features:1.0.1 from/to gemini (http://zodiac.springsource.com/maven/bundles/release/): zodiac.springsource.com] : mvn:ca.islandora.alpaca/islandora-karaf/1.0.1/xml/features\n"}
ansible    | 	to retry, use: --limit @/root/playbook/playbook.retry
ansible    | 
ansible    | PLAY RECAP *********************************************************************
ansible    | default                    : ok=348  changed=67   unreachable=0    failed=1   
ansible    | 
ansible    | Saturday 25 January 2020  16:12:01 +0000 (0:00:08.498)       0:26:26.852 ****** 
ansible    | =============================================================================== 
ansible    | geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 382.73s
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 304.74s
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 217.63s
ansible    | Islandora-Devops.crayfish : Install requisite packages ---------------- 145.26s
ansible    | Islandora-Devops.grok : Make grok -------------------------------------- 57.87s
ansible    | Islandora-Devops.crayfish : Build crayfish code including dependencies -- 45.95s
ansible    | Islandora-Devops.grok : Install Grok dependencies (yum) ---------------- 43.34s
ansible    | Islandora-Devops.drupal-openseadragon : Install Openseadragon with composer -- 20.52s
ansible    | Islandora-Devops.karaf : Install Java packages ------------------------- 18.88s
ansible    | Islandora-Devops.cantaloupe : Install Cantaloupe ----------------------- 17.73s
ansible    | Islandora-Devops.blazegraph : Wait for blazegraph to expand ------------ 17.55s
ansible    | geerlingguy.php-mysql : Install PHP MySQL dependencies (RedHat). ------- 16.83s
ansible    | Islandora-Devops.blazegraph : Wait for Blazegraph to come up ----------- 16.21s
ansible    | Islandora-Devops.activemq : Download and unarchive --------------------- 10.47s
ansible    | install extra packages centos ------------------------------------------ 10.24s
ansible    | Islandora-Devops.alpaca : Add Alpaca Repos ------------------------------ 8.50s
ansible    | Islandora-Devops.grok : Clone Grok -------------------------------------- 7.59s
ansible    | geerlingguy.solr : restart solr ----------------------------------------- 7.56s
ansible    | Islandora-Devops.blazegraph : Download blazegraph war ------------------- 7.31s
ansible    | gather facts ------------------------------------------------------------ 6.38s

@attiks
Copy link

attiks commented Jan 26, 2020

I've destroyed the container and restarted from scratch, but Drush 10 is being installed by https://github.com/Islandora/drupal-project/blob/8.x-1.x/composer.json#L26 but this breaks features, https://www.drupal.org/project/features/issues/3091278

Notes at https://gist.github.com/attiks/96b4e0802130b6876f1cfee466f6f3d7

@attiks
Copy link

attiks commented Jan 27, 2020

Drush 10 problem fixed by specifying Drush 9 in composer, https://gist.github.com/attiks/96b4e0802130b6876f1cfee466f6f3d7#file-diff-patch-L24

Will also be fixed once islandora-deprecated/drupal-project#35 is merged

Still fails on Maven repositories, tried adding settings but it seems they don't work or aren't copied to the right account, https://gist.github.com/attiks/96b4e0802130b6876f1cfee466f6f3d7#file-roles__external__islandora-devops-alpaca__templates__maven-xml-j2

@dannylamb
Copy link
Member

@attiks Thanks, I can add drush 9 to the other PR to sort that out.

If your roles don't seem to be taking hold, you have to clear out the roles/external directory. Also, heads up, sometimes they can wind up in roles/internal too, which takes priority over roles/external. I've been bitten by that more than a few times my friend 🙇‍♂️

@attiks
Copy link

attiks commented Jan 29, 2020

I started from scratch using minimal changes

diff --git a/docker/ansible/Dockerfile b/docker/ansible/Dockerfile
index 1ab5c73..281a536 100644
--- a/docker/ansible/Dockerfile
+++ b/docker/ansible/Dockerfile
@@ -6,7 +6,9 @@ ENV ISLANDORA_DISTRO="centos/7" \
 WORKDIR /root/playbook
 
 RUN yum -y install epel-release && \
-    yum -y install git ansible openssh-clients
+    yum -y install git openssh-clients && \
+    yum -y install python-pip python-dev && \
+    pip install ansible==2.7.10
 
 COPY entrypoint.sh /bin/
 
diff --git a/webserver.yml b/webserver.yml
index f799879..6288785 100644
--- a/webserver.yml
+++ b/webserver.yml
@@ -5,6 +5,7 @@
 
   vars:
     php_version: "7.2"
+    php_memory_limit: "2G"
 
   roles:
     - name: geerlingguy.repo-remi

and it fails on Installation request for islandora/islandora_defaults 1.0.0 -> satisfiable by islandora/islandora_defaults[1.0.0]. - islandora/islandora_defaults 1.0.0 requires islandora/islandora 1.0.0

ansible    | failed: [default] (item=islandora/islandora_defaults:1.0.0) => {"changed": false, "item": "islandora/islandora_defaults:1.0.0", "msg": "./composer.json has been updated > DrupalProject\\composer\\ScriptHandler::checkComposerVersion Loading composer repositories with package information Updating dependencies (including require-dev) Your requirements could not be resolved to an installable set of packages. Problem 1 - Installation request for islandora/islandora_defaults 1.0.0 -> satisfiable by islandora/islandora_defaults[1.0.0]. - islandora/islandora_defaults 1.0.0 requires islandora/islandora 1.0.0 -> no matching package found. Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details. - It's a private package and you forgot to add a custom repository to find it Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems. Installation failed, reverting ./composer.json to its original content.", "stdout": "./composer.json has been updated\n> DrupalProject\\composer\\ScriptHandler::checkComposerVersion\nLoading composer repositories with package information\nUpdating dependencies (including require-dev)\nYour requirements could not be resolved to an installable set of packages.\n\n  Problem 1\n    - Installation request for islandora/islandora_defaults 1.0.0 -> satisfiable by islandora/islandora_defaults[1.0.0].\n    - islandora/islandora_defaults 1.0.0 requires islandora/islandora 1.0.0 -> no matching package found.\n\nPotential causes:\n - A typo in the package name\n - The package is not available in a stable-enough version according to your minimum-stability setting\n   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.\n - It's a private package and you forgot to add a custom repository to find it\n\nRead <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.\n\nInstallation failed, reverting ./composer.json to its original content.\n", "stdout_lines": ["./composer.json has been updated", "> DrupalProject\\composer\\ScriptHandler::checkComposerVersion", "Loading composer repositories with package information", "Updating dependencies (including require-dev)", "Your requirements could not be resolved to an installable set of packages.", "", "  Problem 1", "    - Installation request for islandora/islandora_defaults 1.0.0 -> satisfiable by islandora/islandora_defaults[1.0.0].", "    - islandora/islandora_defaults 1.0.0 requires islandora/islandora 1.0.0 -> no matching package found.", "", "Potential causes:", " - A typo in the package name", " - The package is not available in a stable-enough version according to your minimum-stability setting", "   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.", " - It's a private package and you forgot to add a custom repository to find it", "", "Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.", "", "Installation failed, reverting ./composer.json to its original content."]}
ansible    | 
ansible    | RUNNING HANDLER [geerlingguy.apache : restart apache] **************************
ansible    | Wednesday 29 January 2020  13:25:37 +0000 (0:06:01.071)       0:23:04.710 ***** 
ansible    | 
ansible    | RUNNING HANDLER [geerlingguy.php : restart webserver] **************************
ansible    | Wednesday 29 January 2020  13:25:37 +0000 (0:00:00.004)       0:23:04.715 ***** 
ansible    | 	to retry, use: --limit @/root/playbook/playbook.retry
ansible    | 
ansible    | PLAY RECAP *********************************************************************
ansible    | default                    : ok=142  changed=49   unreachable=0    failed=1   
ansible    | 
ansible    | Wednesday 29 January 2020  13:25:37 +0000 (0:00:00.001)       0:23:04.716 ***** 
ansible    | =============================================================================== 
ansible    | geerlingguy.php : Ensure PHP packages are installed. ------------------ 491.42s
ansible    | geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 361.07s
ansible    | geerlingguy.drupal : Generate Drupal project with composer package in /tmp/composer-project (this may take a while). - 174.47s
ansible    | install extra packages centos ------------------------------------------ 65.96s
ansible    | geerlingguy.apache : Ensure Apache is installed on RHEL. --------------- 47.68s
ansible    | geerlingguy.java : Ensure Java is installed. --------------------------- 35.48s
ansible    | geerlingguy.solr : Download Solr. -------------------------------------- 24.76s
ansible    | geerlingguy.git : Ensure git is installed (RedHat). -------------------- 21.91s
ansible    | geerlingguy.php-mysql : Install PHP MySQL dependencies (RedHat). ------- 15.07s
ansible    | geerlingguy.solr : Run Solr installation script. ----------------------- 13.75s
ansible    | geerlingguy.mysql : Ensure MySQL packages are installed. --------------- 10.03s
ansible    | geerlingguy.mysql : restart mysql --------------------------------------- 7.89s
ansible    | geerlingguy.solr : restart solr ----------------------------------------- 7.53s
ansible    | geerlingguy.solr : Expand Solr. ----------------------------------------- 6.75s
ansible    | geerlingguy.solr : Ensure solr is started and enabled on boot if configured. --- 5.72s
ansible    | geerlingguy.mysql : Ensure MySQL is started and enabled on boot. -------- 5.20s
ansible    | Gathering Facts --------------------------------------------------------- 5.11s
ansible    | gather facts ------------------------------------------------------------ 4.91s
ansible    | Gathering Facts --------------------------------------------------------- 4.44s
ansible    | geerlingguy.solr : Create configured cores. ----------------------------- 4.09s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants