Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BULK UPLOAD: Indore - Health Data (7 datasets) #199

Open
sophialinn opened this issue Aug 22, 2019 · 4 comments
Open

BULK UPLOAD: Indore - Health Data (7 datasets) #199

sophialinn opened this issue Aug 22, 2019 · 4 comments

Comments

@sophialinn
Copy link

Indore data is ready to be uploaded. There are 7 datasets. Data and metadata can be found here:
https://drive.google.com/drive/folders/1UeaaPf0YCO3maL1rd4NH2YrQSMkjZvyU?usp=sharing

@kalxas
Copy link
Collaborator

kalxas commented Aug 27, 2019

Finished processing 8 layers in 362.0 seconds.

6 Created layers
0 Updated layers
0 Skipped layers
2 Failed layers
45.250000 seconds per layer

@kalxas
Copy link
Collaborator

kalxas commented Aug 27, 2019

/data/Indore/Indore Data/ind_specialistclinics.shp                           
================                                                                             
Traceback (most recent call last):                                              
  File "/usr/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 830, in upload
    charset=charset)                                                                      
  File "/usr/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 599, in file_up
load                                                                                         
    defaults=defaults                                            
  File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in mana
ger_method                                                                                   
    return getattr(self.get_queryset(), name)(*args, **kwargs)                              
  File "/usr/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 413, in get
_or_create                                                                                   
    return super(MultilingualQuerySet, self).get_or_create(**kwargs)                         
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 466, in get_o
r_create                                                                                     
    return self._create_object_from_params(lookup, params)                                   
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 500, in _crea
te_object_from_params                                                                        
    obj = self.create(**params)                                                              
  File "/usr/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 405, in cre
ate                                                                                          
    return super(MultilingualQuerySet, self).create(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 394, in creat
e
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python2.7/site-packages/polymorphic/models.py", line 83, in save
    return super(PolymorphicModel, self).save(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 848, in save_b
ase
    update_fields=update_fields, raw=raw, using=using,
  File "/usr/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 193, in s
end
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python2.7/site-packages/geonode/decorators.py", line 53, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/signals.py", line 87, in geo
server_post_save
    producer.geoserver_upload_layer(payload)
  File "</usr/local/lib/python2.7/site-packages/decorator.pyc:decorator-gen-3>", line 2, in g
eoserver_upload_layer
  File "/usr/local/lib/python2.7/site-packages/geonode/messaging/producer.py", line 71, in sy
nc_if_local_memory
    worker.run(timeout=broker_socket_timeout)
  File "/usr/local/lib/python2.7/site-packages/kombu/mixins.py", line 170, in run
    for _ in self.consume(limit=None, **kwargs):
  File "/usr/local/lib/python2.7/site-packages/kombu/mixins.py", line 192, in consume
    conn.drain_events(timeout=safety_interval)
  File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 315, in drain_event
s
    return self.transport.drain_events(self.connection, **kwargs)            
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 963, in
 drain_events                                                                   
    get(self._deliver, timeout=timeout)                                                     
  File "/usr/local/lib/python2.7/site-packages/kombu/utils/scheduling.py", line 56, in get
    return self.fun(resource, callback, **kwargs)                                            
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 1001, i
n _drain_channel                                                 
    return channel.drain_events(callback=callback, timeout=timeout)                          
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 745, in
 drain_events                                                                               
    return self._poll(self.cycle, callback, timeout=timeout)                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 402, in
 _poll                                                                                       
    return cycle.get(callback)                                                               
  File "/usr/local/lib/python2.7/site-packages/kombu/utils/scheduling.py", line 56, in get   
    return self.fun(resource, callback, **kwargs)                                            
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 406, in
 _get_and_deliver                                                                            
    callback(message, queue)                                                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 983, in
 _deliver                                                                                    
    callback(message)                                        
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 633, in
 _callback                                                                                   
    return callback(message)                  
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 624, in _receive_cal
lback                                                                                        
    return on_m(message) if on_m else self.receive(decoded, message)                       
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 590, in receive
    [callback(body, message) for callback in callbacks]                                      
  File "/usr/local/lib/python2.7/site-packages/geonode/messaging/consumer.py", line 104, in o
n_geoserver_messages                                  
    geoserver_post_save_local(layer)                                                         
  File "/usr/local/lib/python2.7/site-packages/geonode/decorators.py", line 53, in wrapper
    return func(*args, **kwargs)                                                           
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/signals.py", line 143, in ge
oserver_post_save_local                                                                      
    charset=instance.charset)                                                                
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/upload.py", line 204, in geo
server_upload                                                                                
    cat.save(gs_resource)                                                                    
  File "/usr/local/lib/python2.7/site-packages/geoserver/catalog.py", line 279, in save
    data = obj.message()                                                                     
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 231, in message
    self.serialize(builder)                                                                  
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 226, in serialize
    writer(builder, self.dirty[k])              
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 149, in write     
    bbox_xml(builder, b)                      
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 281, in bbox_xml  
    builder.end("minx")
  File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1445, in end
    self._flush()                                                                            
  File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1402, in _flush
    text = "".join(self._data)                                                              
TypeError: sequence item 0: expected string, int found                                    
                                                                                             
                                                                                             
/data/Indore/Indore Data/ind_generalclinics.shp                  
================                                                                             
Traceback (most recent call last):                                                           
  File "/usr/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 830, in upload
    charset=charset)                                                                         
  File "/usr/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 599, in file_up
load                                                                                         
    defaults=defaults                                                                        
  File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in mana
ger_method                                                                                   
    return getattr(self.get_queryset(), name)(*args, **kwargs)                               
  File "/usr/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 413, in get
_or_create                                                                                   
    return super(MultilingualQuerySet, self).get_or_create(**kwargs)                         
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 466, in get_o
r_create                                                     
    return self._create_object_from_params(lookup, params)                                   
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 500, in _crea
te_object_from_params                         
    obj = self.create(**params)                                                              
  File "/usr/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 405, in cre
ate                                                                                        
    return super(MultilingualQuerySet, self).create(**kwargs)                           
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 394, in creat
e                                                                                            
    obj.save(force_insert=True, using=self.db)        
  File "/usr/local/lib/python2.7/site-packages/polymorphic/models.py", line 83, in save      
    return super(PolymorphicModel, self).save(*args, **kwargs)                            
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)                                  
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 848, in save_b
ase                                                                                          
    update_fields=update_fields, raw=raw, using=using,                                       
  File "/usr/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 193, in s
end                                                                                          
    for receiver in self._live_receivers(sender)                                       
  File "/usr/local/lib/python2.7/site-packages/geonode/decorators.py", line 53, in wrapper   
    return func(*args, **kwargs)                                                          
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/signals.py", line 87, in geo
server_post_save                                                                            
    producer.geoserver_upload_layer(payload)    
  File "</usr/local/lib/python2.7/site-packages/decorator.pyc:decorator-gen-3>", line 2, in g
eoserver_upload_layer                         
  File "/usr/local/lib/python2.7/site-packages/geonode/messaging/producer.py", line 71, in sy
nc_if_local_memory     
    worker.run(timeout=broker_socket_timeout)
nc_if_local_memory                                                                  [21/1972]
    worker.run(timeout=broker_socket_timeout)                                
  File "/usr/local/lib/python2.7/site-packages/kombu/mixins.py", line 170, in run            
    for _ in self.consume(limit=None, **kwargs):                                
  File "/usr/local/lib/python2.7/site-packages/kombu/mixins.py", line 192, in consume       
    conn.drain_events(timeout=safety_interval)                                            
  File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 315, in drain_event
s                                                                                            
    return self.transport.drain_events(self.connection, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 963, in
 drain_events                                                                                
    get(self._deliver, timeout=timeout)                                                     
  File "/usr/local/lib/python2.7/site-packages/kombu/utils/scheduling.py", line 56, in get   
    return self.fun(resource, callback, **kwargs)                                            
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 1001, i
n _drain_channel                                                                             
    return channel.drain_events(callback=callback, timeout=timeout)                          
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 745, in
 drain_events                                                                                
    return self._poll(self.cycle, callback, timeout=timeout)                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 402, in
 _poll                                                                                       
    return cycle.get(callback)                                                               
  File "/usr/local/lib/python2.7/site-packages/kombu/utils/scheduling.py", line 56, in get
    return self.fun(resource, callback, **kwargs)                                            
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 406, in
 _get_and_deliver                             
    callback(message, queue)                                                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 983, in
 _deliver                                                                                  
    callback(message)                                                                   
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 633, in
 _callback                                                                                   
    return callback(message)                          
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 624, in _receive_cal
lback                                                                                     
    return on_m(message) if on_m else self.receive(decoded, message)                       
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 590, in receive     
    [callback(body, message) for callback in callbacks]                                      
  File "/usr/local/lib/python2.7/site-packages/geonode/messaging/consumer.py", line 104, in o
n_geoserver_messages                                                                         
    geoserver_post_save_local(layer)                                                         
  File "/usr/local/lib/python2.7/site-packages/geonode/decorators.py", line 53, in wrapper   
    return func(*args, **kwargs)                                                       
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/signals.py", line 143, in ge
oserver_post_save_local                                                                   
    charset=instance.charset)                                                                
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/upload.py", line 204, in geo
server_upload                                   
    cat.save(gs_resource)                                                                    
  File "/usr/local/lib/python2.7/site-packages/geoserver/catalog.py", line 279, in save
    data = obj.message()                                                                     
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 231, in message
    self.serialize(builder)
_poll                                                                               [0/1972]
    return cycle.get(callback)                                               
  File "/usr/local/lib/python2.7/site-packages/kombu/utils/scheduling.py", line 56, in get   
    return self.fun(resource, callback, **kwargs)                               
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 406, in
 _get_and_deliver                                                                         
    callback(message, queue)                                                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 983, in
 _deliver                                                        
    callback(message)                                                                        
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/virtual/base.py", line 633, in
 _callback                                                                                  
    return callback(message)                                                                 
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 624, in _receive_cal
lback                                                                                        
    return on_m(message) if on_m else self.receive(decoded, message)                         
  File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 590, in receive     
    [callback(body, message) for callback in callbacks]                                      
  File "/usr/local/lib/python2.7/site-packages/geonode/messaging/consumer.py", line 104, in o
n_geoserver_messages                                                                         
    geoserver_post_save_local(layer)                                                         
  File "/usr/local/lib/python2.7/site-packages/geonode/decorators.py", line 53, in wrapper   
    return func(*args, **kwargs)                                                             
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/signals.py", line 143, in ge
oserver_post_save_local                                                                      
    charset=instance.charset)                                                                
  File "/usr/local/lib/python2.7/site-packages/geonode/geoserver/upload.py", line 204, in geo
server_upload                                                                                
    cat.save(gs_resource)                                                                    
  File "/usr/local/lib/python2.7/site-packages/geoserver/catalog.py", line 279, in save    
    data = obj.message()                                                                
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 231, in message   
    self.serialize(builder)                                                                  
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 226, in serialize
    writer(builder, self.dirty[k])                                                           
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 149, in write  
    bbox_xml(builder, b)                                                                   
  File "/usr/local/lib/python2.7/site-packages/geoserver/support.py", line 281, in bbox_xml  
    builder.end("minx")                                                                      
  File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1445, in end                
    self._flush()                                                                            
  File "/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1402, in _flush             
    text = "".join(self._data)                                                               
TypeError: sequence item 0: expected string, int found

@kalxas
Copy link
Collaborator

kalxas commented Aug 27, 2019

Indore: Municipal Government Administrative Boundary : Metadata updated!
Indore: Ayush Centers : Metadata updated!
Indore: Dental Clinics : Metadata updated!
indore: general clinics : Failed to update
Exception: Layer matching query does not exist.
Indore: Pharmacies : Metadata updated!
Indore: Private Hospitals : Metadata updated!
indore: specialist clinics : Failed to update
Exception: Layer matching query does not exist.
Indore: Veterinary Clinics : Metadata updated!

@kalxas kalxas modified the milestones: Stage 1, maintenance Sep 13, 2019
@kalxas
Copy link
Collaborator

kalxas commented Sep 13, 2019

Both layers have empty geometries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants