Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JULIA_EXCLUSIVE with auto threads errors on 1.10, mac/arm64 #50702

Open
daviehh opened this issue Jul 28, 2023 · 2 comments · May be fixed by #57136
Open

JULIA_EXCLUSIVE with auto threads errors on 1.10, mac/arm64 #50702

daviehh opened this issue Jul 28, 2023 · 2 comments · May be fixed by #57136
Labels
multithreading Base.Threads and related functionality regression 1.10 Regression in the 1.10 release

Comments

@daviehh
Copy link
Contributor

daviehh commented Jul 28, 2023

Running

JULIA_EXCLUSIVE=1 julia --threads=auto

with v1.10 now errors with

ERROR: Too many threads requested for JULIA_EXCLUSIVE option

on macos, versioninfo is

Julia Version 1.10.0-beta1
Commit 6616549950e (2023-07-25 17:43 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 10 × Apple M1 Pro
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
  Threads: 1 on 8 virtual cores
@daviehh
Copy link
Contributor Author

daviehh commented Jul 28, 2023

Maybe related to the new gcthreads?

JULIA_EXCLUSIVE=1 julia --threads=auto --gcthreads=1

runs ok, and --gcthreads=2 results in

ERROR: Too many threads requested for JULIA_EXCLUSIVE option

@brenhinkeller brenhinkeller added the multithreading Base.Threads and related functionality label Aug 3, 2023
@simonbyrne simonbyrne added the regression 1.10 Regression in the 1.10 release label Jan 13, 2024
@simonbyrne simonbyrne mentioned this issue Jan 13, 2024
Open
@DatName
Copy link

DatName commented Feb 29, 2024

For me, on Linux pc 6.5.0-21-generic #21~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 13:32:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

and julia 1.10.0 (and 1.10.1)

pc:~ $ julia
julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 20 × 12th Gen Intel(R) Core(TM) i9-12900H
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, alderlake)
  Threads: 1 on 20 virtual cores
Environment:
  JULIA_EXCLUSIVE = 1
  JULIA_PKG_USE_CLI_GIT = true

I get ERROR: Too many threads requested for JULIA_EXCLUSIVE option. for threads above 15:

pc:~/julia/julia-1.10.0/bin $ ./julia --banner=no --threads=auto
ERROR: Too many threads requested for JULIA_EXCLUSIVE option.
pc:~/julia/julia-1.10.0/bin $ ./julia --banner=no --threads=20
ERROR: Too many threads requested for JULIA_EXCLUSIVE option.
pc:~/julia/julia-1.10.0/bin $ ./julia --banner=no --threads=15
ERROR: Too many threads requested for JULIA_EXCLUSIVE option.
pc:~/julia/julia-1.10.0/bin $ ./julia --banner=no --threads=14
julia>

Since error comes from

if (exclusive) {
        if (nthreads > jl_cpu_threads()) {
            jl_printf(JL_STDERR, "ERROR: Too many threads requested for %s option.\n", MACHINE_EXCLUSIVE_NAME);
            exit(1);
        }
...

I checked that jl_cpu_threads() == 20:

julia> ccall(:jl_cpu_threads, Int32, ())
20

xal-0 added a commit to xal-0/julia that referenced this issue Jan 21, 2025
@xal-0
Fix JULIA_EXCLUSIVE setting affinity on non-worker threads
582db83 
With JULIA_EXCLUSIVE=1, we would try to give both worker and interactive threads
an exclusive CPU, causing --threads=auto to produce a "Too many threads
requested" error.

Fixes JuliaLang#50702.
xal-0 added a commit to xal-0/julia that referenced this issue Jan 22, 2025
@xal-0
Fix JULIA_EXCLUSIVE setting affinity on non-worker threads
57a7ee5 
With JULIA_EXCLUSIVE=1, we would try to give both worker and interactive threads
an exclusive CPU, causing --threads=auto to produce a "Too many threads
requested" error.

Fixes JuliaLang#50702.
@xal-0 xal-0 linked a pull request Jan 22, 2025 that will close this issue
Open
xal-0 added a commit to xal-0/julia that referenced this issue Jan 24, 2025
@xal-0
Fix JULIA_EXCLUSIVE setting affinity on non-worker threads
8bb6b8f 
With JULIA_EXCLUSIVE=1, we would try to give both worker and interactive threads
an exclusive CPU, causing --threads=auto to produce a "Too many threads
requested" error.

Fixes JuliaLang#50702.
xal-0 added a commit to xal-0/julia that referenced this issue Jan 24, 2025
@xal-0
Fix JULIA_EXCLUSIVE setting affinity on non-worker threads
e998efa 
With JULIA_EXCLUSIVE=1, we would try to give both worker and interactive threads
an exclusive CPU, causing --threads=auto to produce a "Too many threads
requested" error.

Fixes JuliaLang#50702.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
multithreading Base.Threads and related functionality regression 1.10 Regression in the 1.10 release
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix JULIA_EXCLUSIVE setting affinity on non-worker threads xal-0/julia
4 participants
@simonbyrne @DatName @brenhinkeller @daviehh