Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: service group resource subscribe ration #6479

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

refactor: service group resource subscribe ration #6479

wants to merge 13 commits into from

Conversation

iutx
Copy link
Member

@iutx iutx commented Jan 9, 2025 •
edited
Loading

What this PR does / why we need it:

  1. fix mysql resource config error

  2. optimize resources getter
    Global SubscribeRatio

  3. If the SubscribeRatio isn't configured in the non-production workspace, we'll go with the global settings.
    Otherwise, we'll stick to the workspace-specific configuration.

  4. Also affects in the production workspace.

  5. Init container not set resource limit

  6. Sidecar container set similar subscribe ration rule

Which issue(s) this PR fixes:

Specified Reviewers:

/assign @sfwn @luobily

ChangeLog

Language Changelog
🇺🇸 English fix mysql resource config error
🇨🇳 中文 修复 mysql 资源配置错误

Need cherry-pick to release versions?

Add comment like /cherry-pick release/1.0 when this PR is merged.

For details on the cherry pick process, see the cherry pick requests section under CONTRIBUTING.md.

@erda-bot
Copy link
Member

erda-bot commented Jan 9, 2025

Add assignees failed, err: Post "https://api.github.com/repos/erda-project/erda/pulls/6479/requested_reviewers": proxyconnect tcp: dial tcp 8.218.184.45:34894: i/o timeout

@iutx iutx marked this pull request as draft January 9, 2025 12:05
@iutx iutx requested review from sfwn and luobily January 10, 2025 05:33
@codecov Codecov
Copy link

codecov bot commented Jan 10, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 43.16354% with 212 lines in your changes missing coverage. Please review.

Project coverage is 15.12%. Comparing base (d217119) to head (7a61d41).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
...rator/scheduler/executor/plugins/k8s/deployment.go 25.45% 37 Missing and 4 partials ⚠️
...r/executor/plugins/k8s/addon/redis/legacy/redis.go 0.00% 27 Missing ⚠️
.../scheduler/executor/plugins/k8s/subscriberatios.go 79.31% 21 Missing and 3 partials ⚠️
...er/executor/plugins/k8s/addon/rocketmq/rocketmq.go 5.26% 18 Missing ⚠️
...ator/scheduler/executor/plugins/k8s/statefulset.go 28.00% 18 Missing ⚠️
...r/plugins/k8s/addon/elasticsearch/elasticsearch.go 0.00% 17 Missing ⚠️
...cheduler/executor/plugins/k8s/addon/redis/redis.go 12.50% 14 Missing ⚠️
.../executor/plugins/k8s/addon/daemonset/daemonset.go 9.09% 10 Missing ⚠️
.../executor/plugins/k8s/addon/sourcecov/sourcecov.go 0.00% 9 Missing ⚠️
...trator/scheduler/executor/plugins/k8s/daemonset.go 0.00% 9 Missing ⚠️
... and 6 more
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6479   +/-   ##
=======================================
  Coverage   15.12%   15.12%           
=======================================
  Files        3675     3676    +1     
  Lines      377005   376911   -94     
=======================================
- Hits        57018    57016    -2     
+ Misses     314540   314446   -94     
- Partials     5447     5449    +2
Flag Coverage Δ
by-github-actions 15.12% <43.16%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...orchestrator/scheduler/executor/plugins/k8s/k8s.go 23.59% <100.00%> (-2.00%) ⬇️
...cheduler/executor/plugins/k8s/addon/redis/unify.go 0.00% <0.00%> (ø)
...cheduler/executor/plugins/k8s/addon/mysql/mysql.go 55.12% <80.00%> (-6.07%) ⬇️
...orchestrator/scheduler/executor/plugins/k8s/job.go 47.90% <57.14%> (+2.71%) ⬆️
...tools/orchestrator/scheduler/executor/util/util.go 37.59% <50.00%> (+0.37%) ⬆️
...cheduler/executor/plugins/k8s/addon/canal/canal.go 31.14% <73.91%> (-1.17%) ⬇️
...or/scheduler/executor/plugins/k8s/addon/process.go 0.00% <0.00%> (ø)
.../executor/plugins/k8s/addon/sourcecov/sourcecov.go 1.46% <0.00%> (+0.05%) ⬆️
...trator/scheduler/executor/plugins/k8s/daemonset.go 0.00% <0.00%> (ø)
.../executor/plugins/k8s/addon/daemonset/daemonset.go 5.17% <9.09%> (+0.20%) ⬆️
... and 7 more

... and 8 files with indirect coverage changes

@iutx iutx marked this pull request as ready for review January 15, 2025 10:06
@erda-bot
Copy link
Member

Add assignees failed, err: Post "https://api.github.com/repos/erda-project/erda/pulls/6479/requested_reviewers": proxyconnect tcp: dial tcp 8.218.184.45:34895: i/o timeout

@iutx iutx changed the title fix: mysql resource config error refactor: service group resource subscribe ration Jan 20, 2025
@iutx iutx added orchestrator workload management refactor labels Jan 20, 2025
@erda-bot
Copy link
Member

Add assignees failed, err: Post "https://api.github.com/repos/erda-project/erda/pulls/6479/requested_reviewers": proxyconnect tcp: dial tcp 8.218.184.45:34895: i/o timeout

@iutx iutx self-assigned this Jan 20, 2025
@iutx iutx requested a review from CeerDecy January 22, 2025 09:21
sfwn
sfwn reviewed Jan 22, 2025
View reviewed changes
internal/tools/orchestrator/scheduler/executor/plugins/k8s/deployment.go

return nil
}

func (k *Kubernetes) UpdateContainerResourceEnv(originResource apistructs.Resources, container *corev1.Container) {
for index, env := range container.Env {
var needToUpdate = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is too large, it should be split.

sfwn
sfwn requested changes Jan 22, 2025
View reviewed changes
internal/tools/orchestrator/scheduler/executor/plugins/k8s/k8s.go
@@ -458,9 +447,9 @@ func New(name executortypes.Name, clusterName string, options map[string]string)
k.elasticsearchoperator = elasticsearchoperator
redisoperator := redis.New(k, deploy, sts, svc, ns, k, k8ssecret, client)
k.redisoperator = redisoperator
mysqloperator := mysql.New(k, ns, k8ssecret, pvc, client)
mysqloperator := mysql.New(k, ns, k, k8ssecret, pvc, client)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two k is strange.

sfwn
sfwn requested changes Jan 22, 2025
View reviewed changes
internal/tools/orchestrator/scheduler/executor/plugins/k8s/subscriberatios.go
@@ -0,0 +1,201 @@
// Copyright (c) 2021 Terminus, Inc.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the file name is subscriberations?

internal/tools/orchestrator/scheduler/executor/plugins/k8s/subscriberatios.go
DefaultRatio = 1.0
)

type SubscribeRatios struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think OverSubscribeRatios is better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sfwn sfwn sfwn requested changes

@luobily luobily Awaiting requested review from luobily

@CeerDecy CeerDecy Awaiting requested review from CeerDecy

Requested changes must be addressed to merge this pull request.

Assignees

@iutx iutx

Labels
orchestrator workload management refactor
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@iutx @erda-bot @sfwn