We should support the Atomic host natively

[purpleidea]

This requires some fixes to the sync dir, and we should detect and do this automatically if the box name starts with 'atomic-'

[purpleidea]

This feature depends on #68

[purpleidea]

This requires screen in the atomic host. This was added in https://git.fedorahosted.org/cgit/fedora-atomic.git/commit/?id=268fabe30ae6640293bc0c1a57372e0c83f504c4 but is not yet available in the builds I have tested. If there is a new vagrant-libvirt box build to test, please leave the link here.

james@computer:~/code/oh-my-vagrant/vagrant (feat/atomic)$ vscreen omv1
bash: screen: command not found
Connection to 192.168.121.176 closed.

[purpleidea]

This requires the root user setup correctly as expected by vagrant. It requires both the password set to 'vagrant' as well as the insecure private key added. I have reported this to the Atomic folks, but I'm not sure if it was fixed yet. If someone has an up-to-date vagrant-libvirt box with the correct fixes, please let me know.

james@computer:~/code/oh-my-vagrant/vagrant (feat/atomic)$ vscreen root@omv1
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[purpleidea]

The docker group does not exist. Ideally it should be added. If there is a reason this should be different from every other host that uses docker, then the upstream vagrant docker provisioner needs to be patched to detect atomic, and work around these weird quirks.

Example:

james@computer:~/code/oh-my-vagrant/vagrant (feat/atomic)$ time vup omv1
Bringing machine 'omv1' up with 'libvirt' provider...
==> omv1: Uploading base box image as volume into libvirt storage...
==> omv1: Creating image (snapshot of base box volume).
==> omv1: Creating domain with the following settings...
==> omv1:  -- Name:              omv_omv1
==> omv1:  -- Domain type:       kvm
==> omv1:  -- Cpus:              1
==> omv1:  -- Memory:            512M
==> omv1:  -- Base box:          atomic-rhel-7.1
==> omv1:  -- Storage pool:      default
==> omv1:  -- Image:             /var/lib/libvirt/images/omv_omv1.img
==> omv1:  -- Volume Cache:      default
==> omv1:  -- Kernel:            
==> omv1:  -- Initrd:            
==> omv1:  -- Graphics Type:     vnc
==> omv1:  -- Graphics Port:     5900
==> omv1:  -- Graphics IP:       127.0.0.1
==> omv1:  -- Graphics Password: Not defined
==> omv1:  -- Video Type:        cirrus
==> omv1:  -- Video VRAM:        9216
==> omv1:  -- Keymap:            en-us
==> omv1:  -- Command line : 
==> omv1: Starting domain.
==> omv1: Waiting for domain to get an IP address...
==> omv1: Waiting for SSH to become available...
==> omv1: Starting domain.
==> omv1: Waiting for domain to get an IP address...
==> omv1: Waiting for SSH to become available...
==> omv1: Creating shared folders metadata...
==> omv1: Setting hostname...
==> omv1: Rsyncing folder: /home/james/code/oh-my-vagrant/vagrant/ => /home/vagrant/sync
==> omv1: Configuring and enabling network interfaces...
==> omv1: Updating /etc/hosts file on active guest machines...
==> omv1: Running provisioner: shell...
    omv1: Running: inline script
==> omv1: Running provisioner: docker...
    omv1: Configuring Docker to autostart containers...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

usermod -a -G docker vagrant

Stdout from the command:



Stderr from the command:

usermod: group 'docker' does not exist


real    1m9.228s
user    0m33.127s
sys 0m2.970s

[purpleidea]

You'll find a feature branch for atomic support in omv located here: https://github.com/purpleidea/oh-my-vagrant/tree/feat/atomic

It is based on the https://github.com/purpleidea/oh-my-vagrant/tree/feat/hostmanager branch which is not currently in git master because of the upstream issues in #68.

Once the other issues mentioned in this tracker are addressed then we can continue testing this branch.

Cheers!

[purpleidea]

To work around the docker issues and continue debugging to see if there are other issues with the atomic host, you can do the following workaround:

time vup omv1 --no-provision
[snip]
vp --provision-with shell
[snip]

At the end of this second command, the following error is seen:

==> omv1: Running provisioner: shell...
    omv1: Running: inline script
==> omv1: F0418 05:54:34.392419   12310 get.go:151] 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
==> omv1: I0418 05:54:34.436621   12318 restclient.go:146] Waiting for completion of operation 1
==> omv1: F0418 05:54:36.442779   12318 create.go:75] 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

This is happening because the kubernetes setup seems to be somehow not working correctly. This is either due to an issue with the kubernetes version installed on the machine, a mis-match in the way it should be configured, or some other issue. As a result, this needs investigation from a kubernetes expert since this works fine for other operating systems.

An example of the issue can be seen by running:

$ vssh omv1
[vagrant@omv1 ~]$ kubectl get minions 
F0418 05:55:31.610708   12361 get.go:151] 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]

Normally we see the other hosts as "NotReady", but the command returns an exit code of zero.

[purpleidea]

@goern and https://github.com/cgwalters hopefully the above information is helpful to you. Until some of those blockers are resolved, I'm at a dead end for the moment.

Cheers,
James

[purpleidea]

This was posted to atomic-devel here: https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2015-April/msg00046.html

[cgwalters]

RE: docker group, see https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2015-January/msg00034.html

[purpleidea]

@cgwalters thanks for the background on docker group! I guess there are three possibilities:

1. Add the docker group to the atomic host so it exists but don't add anyone to it.
2. Add the docker group to the vagrant image
3. Patch upstream vagrant docker provisioner to not modify group settings

I recommend doing at least #2 and #3. #1 is up to the atomic devel folks.
In parallel if there are enough of these quirks in atomic, I'd recommend someone starting a vagrant os type for atomic.

[purpleidea]

(In the meantime, I think the code itself is ready for git master, even if the boxes aren't...)
So, in order to make maintaining a bunch of unmerged feature branches, I'm dumping this into git master. 85d3b22

[purpleidea]

I added this patch to hopefully preempt any issues with subscription manager + atomic + omv not working: f3e2e6a

[purpleidea]

After talking with https://github.com/rwmjones it appears two bugs are currently blocking atomic+virt-builder and these should be fixed so that developers can hack on atomic more easily, along with many more use cases. The two bugs are:

https://bugzilla.redhat.com/show_bug.cgi?id=1102241
https://bugzilla.redhat.com/show_bug.cgi?id=1177460

Beverages, eternal fame or other prizes will be awarded to whoever gets these fixed :)

[cgwalters]

Note that wrt virt-customize, anything you can do there can also be accomplished by using kickstart + ImageFactory, which I'll be the first to say is nowhere near as easy to use. On the plus side, kickstart files can also apply to bare metal installations, and it's the same technology used by release engineering, so any improvements to it benefit rel-eng.

[purpleidea]

Agreed... Actually I was just having this conversation with imcleod about how I'd use the other tools if they were easy to use and had the features... So atm, the goal of a tool that's easy for a user to use and take home is virt-builder...

Virt-builder can do metal. It can build an image which you can dd to a drive or anywhere...

[purpleidea]

After a discussion with @cgwalters, he suggested a workaround for the docker group issue, and it turns out it works great! It's definitely a hack, but it's good enough for oh-my-vagrant until something better comes along, and it's good enough for git master: d8d1cb1

Great idea @cgwalters !

In parallel, someone who wants vagrant+atomic to work long term should really patch this upstream so future atomic quirks don't break vagrant. CoreOS does this for example: https://github.com/mitchellh/vagrant/tree/master/plugins/guests/coreos

I'm going to close this bug for now. Atomic now WFM (with the right image), and this is key, because a good number of the available images out there are still broken. (lack of screen, lack of root/vagrant users setup correctly, etc...)

[pschiffe]

When using atomic host, I see that the second network interface in the guest (configured by omv) won't come up automatically after boot. Are you using some kind of workaround for it?

[purpleidea]

@pschiffe It should come up... If it's not, please open a new ticket, including vagrant versions, more info, etc...