Genome::Sys::Lock::backends error #162
Comments
|
It looks like locking (with respect to 'scope') was recently reworked (Jan/2015) by Mark Burnett. It possible that there is some expectation/environment related to this that is not being met in the standalone environment now. |
|
@mkiwala you mentioned the nessy vs. old split. Is there a way to configure this to work with the old locking, since nessy isn't ready yet? |
|
In The concept of scopes was introduced recently after it was discovered that some code was requesting a lock on a local filesystem path called There is also the concept of locking backends Michael
This email message is a private communication. The information transmitted, including attachments, is intended only for the person or entity to which it is addressed and may contain confidential, privileged, and/or proprietary material. Any review, duplication, retransmission, distribution, or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is unauthorized by the sender and is prohibited. If you have received this message in error, please contact the sender immediately by return email and delete the original message from all computer systems. Thank you. |
|
One possible fix. Create a new site module and place necessary setup in there. For example to configure the 'backends' it could contain the following. Genome/Site/SGMS.pm package Genome::Site::SGMS;
use strict;
use warnings;
BEGIN {
unless ($INC{'Genome/Sys/Lock.pm'}) {
die 'must load Genome::Sys::Lock first';
}
};
require Genome::Sys::Lock::FileBackend;
Genome::Sys::Lock->add_backend('site',
Genome::Sys::Lock::FileBackend->new(is_mandatory => 1,
parent_dir => $ENV{GENOME_LOCK_DIR}));
Genome::Sys::Lock->add_backend('unknown',
Genome::Sys::Lock::FileBackend->new(is_mandatory => 1,
parent_dir => '/'));
1;Then update: Change: To: |
|
One thing that confuses me still. How does Within However, in the standalone GMS we won't know what the domain is going to be... So how do we ensure that this stuff gets loaded... On possibility is that instead of updating if ($ENV{GENOME_SYS_ID} && $ENV{GENOME_SYS_ID} ne 'GMS1') {
use Genome::Site::SGMS;
}OR... maybe we can take advantage of the sub import {
if (my $config = $ENV{GENOME_CONFIG}) {
require_module($config);
}
else {
load_host_config();
}
}
Depending on how |
|
I agree that the change to I’m not sure what would make the most sense. I think it would work to just add the Perhaps a better approach would be to introduce a new optional environment variable in master which overrides the hostname parsing. The purpose of the environment variable would be to set the site name. The variable could be called GENOME_INSTALL_SITE with a value of ‘StandaloneGMS’. The presence of the variable with this value could cause Genome/Site.pm to load the StandaloneGMS site code.
This email message is a private communication. The information transmitted, including attachments, is intended only for the person or entity to which it is addressed and may contain confidential, privileged, and/or proprietary material. Any review, duplication, retransmission, distribution, or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is unauthorized by the sender and is prohibited. If you have received this message in error, please contact the sender immediately by return email and delete the original message from all computer systems. Thank you. |
|
is $GENOME_CONFIG supposed to do something similar to what @mkiwala describes ? |
|
I agree, it seems that perhaps This search: Yields this result: Perhaps it was added with this idea in mind a while back, but never wound up being used. If that's the case I say we try it out now. |
|
Genome::Sys::Lock.pm has some scopes hardcoded,
|
|
So Genome/Site.pm was supposed to have generic/external defaults, and Genome/Site/TGI.pm was supposed to override them. The question was then how does the system automatically figure out that it is a TGI system? Because parsing the hostname leads to edu/wustl/gsc.pm, this is the TGI-specific entry point. So, can we register the default handlers in Genome/Site.pm, and let Site/TGI.pm override those with Nessy? Or is it problematic to even ever set-up the default handlers? |
At one point there was another subnet that was also active at TGI and it, like edu/wustl/gsc.pm also used Site/TGI.pm. |
|
A couple of pull requests addressing this was made to genome/gms and genome/genome. We basically set $GENOME_CONFIG=Genome::Site::SGMS and configure two backends in Site/SGMS.pm |
gatoravi
changed the title
|
this still needs to be merged into master. merged into the SGMS branch so that we can test this. |
|
this was addressed in master here and the same changes are in the SGMS branches, genome/genome@97226f1 and |
|
closing since it looks like the backends stuff was resolved. |
|
Doing a fresh install of gms... we seem to have encountered this issue anew. Here's a log: sudo bash -l -c 'source /etc/genome.conf; /usr/bin/perl setup/prime-disk-allocations.pl' |
|
This is really strange since as you have mentioned there hasn't been an update to the gms-pub branch. We definitely expect issues with using the master branch for this project and hence would not recommend that, there is code unique to both of these branches. The Two questions,
If this is still an issue we'd be glad to take a look at this and the other issue, could you email me on the email address on my profile! Thanks, |
|
cc'ing @GrubLord just in case this old issue gets drowned amidst notifications. |
|
Thanks kindly, Avi... this issue is rather odd to be sure. Generally speaking we've found GMS to behave rather unexpectedly when it comes to handling storage... and this was an outgrowth of that - perhaps caused my trying to do a fresh GMS setup over the drives of our old instance. While I have done as fresh an install as possible, with new drives, and managed to circumvent this - it's the other issues we have reported recently that have been real show-stoppers - particularly the fact that without us making direct changes to the "Creator.pm" code to invoke Volume.pm's "sync_total_kb()" method, our disk is considered to run out of space long before the drive actually fills up... It was this issue that caused us to have to rebuild our GMS setup altogether, along with issues around re-priming, and an unfortunate problem where GMS started writing its temporary files to "/home/ubuntu" rather than /tmp (due to an accessibility problem caused by replacing /tmp with a symlink, which we then reverted). Despite re-creating the original setup as best we could, and various attempts using environment variables and poking around in the code, we were unable to get GMS to write to /tmp any more, and every job would fail due to lack of space in /home/ubuntu. While I think we can consider this issue closed, I would very much appreciate your assistance with these other two - particularly as your colleague appears to have misunderstood our setup (we've been using an 80 Gb root drive, and two additional drives of 4 TB and 7 TB each, mounted at /tmp and /opt respectively, which we understand to be your recommended setup). Also, the replacement of /tmp with a symlink was done because we would keep getting disk space errors, and believed that perhaps consolidating to a single larger /opt disk and symlinking /tmp onto this drive would ensure that we never run out of temporary space due to a mismatch in size between the two volumes. While this was perhaps ill-advised, due to how GMS handles storage, it did bring to light the rather show-stopping issues that occur when /tmp becomes unavailable. Would very much appreciate your assistance with the /tmp and Volume.pm issues any time you can manage. -- Liviu On 16 Jan 2016, at 3:29 am, Avi Ramu <[email protected]mailto:[email protected]> wrote: This is really strange since as you have mentioned there hasn't been an update to the gms-pub branch. We definitely expect issues with using the master branch for this project and hence would not recommend that, there is code unique to both of these branches. Two questions,
If this is still an issue we'd be glad to take a look at this and the other issue, could you email me on the email address on my profile! Thanks, — NOTICE |
Running the installer since the merge I now get this error on initial priming of the genome database:
This type of error is not immediately reminiscent of error we observed in past merge attempts. Though issues relating to changes in the db schema were common.
The text was updated successfully, but these errors were encountered: