Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlalchemy.Metadata.reflect failed because ONLY_TABLES contains weird view names on labs #2

Open
chelsyx opened this issue Aug 29, 2018 · 2 comments

Comments

@chelsyx
Copy link

chelsyx commented Aug 29, 2018

When using mwdb.Schema to connect to production database, tables cannot be load because of those weird view names on labs:

>>> import mwdb
>>> enwiki = mwdb.Schema("mysql+pymysql://analytics-store.eqiad.wmnet/enwiki" + "?read_default_file=/path/to/my.cnf")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/my/virtualenv/lib/python3.5/site-packages/mwdb/schema.py", line 114, in __init__
    self.meta.reflect(views=True, only=only_tables or self.ONLY_TABLES)
  File "/path/to/my/virtualenv/lib/python3.5/site-packages/sqlalchemy/sql/schema.py", line 3957, in reflect
    (bind.engine, s, ', '.join(missing)))
sqlalchemy.exc.InvalidRequestError: Could not reflect: requested table(s) not available in Engine(mysql+pymysql://analytics-store.eqiad.wmnet/enwiki?read_default_file=/path/to/my.cnf): (archive_userindex, filearchive_userindex, ipblocks_ipindex, localisation, localisation_file_hash, logging_logindex, logging_userindex, mark_as_helpful, msg_resource_links, oldimage_userindex, recentchanges_userindex, revision_userindex, updates, user_properties_anon, watchlist_count, wikilove_image_log)

It works after I remove those names from ONLY_TABLES (https://github.com/mediawiki-utilities/python-mwdb/blob/master/mwdb/schema.py#L37). But I'm sure it's not the right way to fix it because it won't work on both production database and public replicas.

@halfak
Copy link
Member

halfak commented Sep 4, 2018

Yes. This is a serious issue. I've reported it to the SQLAlchemy folks and it might be fixed in the current version. I can't find the bug in a quick web search so maybe it was addressed? It would be worth updating mwdb's requirements to pull in a more recent version of SQLAlchemy to see if it is still an issue. I'm pretty backlogged so please feel free to try to beat me to it :D

@chelsyx
Copy link
Author

chelsyx commented Sep 6, 2018

Hmmm... Updating mwdb's requirements doesn't help: I changed the requirement to sqlalchemy>1.2 (the most recent version of SQLAlchemy is 1.2.11 released on 2018-08-20) but the error is still there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants