Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get custom domains from own models instead of AMC db #1239

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

melvinsoft
Copy link

@melvinsoft melvinsoft commented Sep 5, 2022

Change description

Soon we will deprecate AMC and the postgres database won't exist anymore, I realized setting up a custom domains, all the info this commands gets from the AMC database is the same as the alternative domains

Type of change

  • Bug fix (fixes an issue)
  • New feature (adds functionality)

Related issues

This PRs works together with:
appsembler/configuration#410

Checklists

Development

  • Lint rules pass locally
  • Application changes have been tested thoroughly
  • Automated tests covering modified code pass

Security

  • Security impact of change has been considered
  • Code follows company security practices and guidelines

Code review

  • Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
  • "Ready for review" label attached and reviewers assigned
  • Changes have been reviewed by at least one other contributor
  • Pull request linked to task tracker where applicable

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@coveralls
Copy link

coveralls commented Sep 5, 2022

Pull Request Test Coverage Report for Build 3061069918

  • 14 of 14 (100.0%) changed or added relevant lines in 2 files are covered.
  • 244 unchanged lines in 20 files lost coverage.
  • Overall coverage increased (+0.03%) to 49.033%

Files with Coverage Reduction New Missed Lines %
common/djangoapps/track/shim.py 1 64.86%
openedx/core/djangoapps/appsembler/eventtracking/apps.py 1 94.74%
openedx/core/djangoapps/content/block_structure/block_structure.py 1 94.86%
openedx/core/djangoapps/appsembler/api/tests/test_enrollment_api.py 2 99.09%
openedx/core/djangoapps/appsembler/api/tests/test_registration_api.py 4 94.74%
openedx/core/djangoapps/appsembler/api/tests/test_registration_api_v2.py 4 96.19%
openedx/core/djangoapps/appsembler/sites/tests/test_utils.py 4 93.1%
openedx/core/djangoapps/appsembler/api/sites.py 6 89.09%
openedx/core/djangoapps/appsembler/api/tests/test_throttling.py 7 59.38%
openedx/core/djangoapps/appsembler/sites/tests/test_site_config_client.py 8 86.24%
Totals Coverage Status
Change from base Build 2970894981: 0.03%
Covered Lines: 110941
Relevant Lines: 226260

💛 - Coveralls

Copy link

@OmarIthawi OmarIthawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@melvinsoft thanks a lot for removing AMC db call from the command.

I think we need a new field AlternativeDomain.cert_pending to distinguish between pending certs and ready certs. Such logic shouldn't be needed whenever we use an external service like Treafik, but until then I think this fix is needed.

I've checked the code and I'm not sure if it'll work well or not given that we don't store the cert_pending state anywhere and we may be moving from one unreliable to another unreliable workflow.

Please also add tests for this code.

@melvinsoft melvinsoft marked this pull request as ready for review September 15, 2022 12:10
@melvinsoft
Copy link
Author

@OmarIthawi Thank you so much for the review.

Here is the issue, as you know we have been talking about fixing custom domains and move to something like Traefik, so adding a new field here, also means we need to change logic in Tahoe cert agent and our Ansible playbooks, I don't even know how to start implementing that logic. I understand the tests, but this command is only being used by Tahoe cert agent, the piece we want to deprecate, so I don't think it worth it.

In the other hand, I already consumed all my coding buffers for this sprint and this tasks, so I cannot invest more myself in this, what I achieved is something it can be executed with a runbook or a script, and allow an engineer to set up a new custom domains in about 1 hr, with our current conflicting priorities and capacity, we shouldn't fix this further at this stage.

Copy link

@OmarIthawi OmarIthawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again Maxi Sorry for not making my original review clearer.

Required changes: Please add tests now that AMC database isn't accessed.

Suggested change: Add cert_pending field we agreed that we'll not perform this change.

Suggested change: Only return active sites.

cursor.execute("SELECT custom_domain FROM organizations_microsite WHERE custom_domain != '';")
rows = cursor.fetchall()
domains = [{'domains': row} for row in rows]
altertive_domains = AlternativeDomain.objects.values_list('site__domain', flat=True)
Copy link

@OmarIthawi OmarIthawi Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended: Can we filter by get_active_sites?

Suggested change
altertive_domains = AlternativeDomain.objects.values_list('site__domain', flat=True)
altertive_domains = AlternativeDomain.objects.filter(
site__in=get_active_sites(),
).values_list('site__domain', flat=True)

As long as the Ansible playbook uses www-data this change should be okay.

- name: List custom domains
  become: www-data
  shell: ./manage lms custom_domains_list 

@github-actions
Copy link

Checking git merge conflicts against https://github.com/edx/edx-platform.git

Comparing with open-release/koa.master
Benchmark conflicts with main 111
Current conflicts 111
Summary Good work! No added conflicts.
Comparing with open-release/lilac.master
Benchmark conflicts with main 254
Current conflicts 254
Summary Good work! No added conflicts.
Comparing with open-release/maple.master
Benchmark conflicts with main 285
Current conflicts 285
Summary Good work! No added conflicts.
Comparing with open-release/nutmeg.master
Benchmark conflicts with main 293
Current conflicts 293
Summary Good work! No added conflicts.
Comparing with master
Benchmark conflicts with main 290
Current conflicts 290
Summary Good work! No added conflicts.

@thraxil
Copy link

thraxil commented Sep 15, 2022

I added a simple test to show how easy it is. It should be expanded with some more test cases to, eg, verify that Omar's recommendation of only including active sites works correctly.

Copy link

@OmarIthawi OmarIthawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @melvinsoft and @thraxil. This pull request looks great already after adding the tests.

Let's get it merged and I may add the get_active_sites later.

If you're happy with it, please merge it.

@thraxil thraxil removed their request for review January 24, 2023 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants