Add Support for `BitVector` and `BitMatrix` Indexing #708

RoyiAvital · 2024-10-27T07:31:56Z

Fixes #707 .

ericphanson · 2024-10-27T11:22:17Z

src/atoms/IndexAtom.jl

+    return [xi for (xi, ii) in zip(x, I) if ii]
+end
+
+function Base.getindex(x::AbstractExpr, I::BitVector)
+    return [xi for (xi, ii) in zip(x, I) if ii]


I think we need some boundschecking on these, since we should get an error if I is too large or has the wrong shape, but zip won't do that

I copied the code from the Boolean matrix / vector case.
Fixing it here won't change there.

So maybe get this in and add an issue for bound checking for the Boolean / Bit cases later?

ah ok. Maybe we can just change the signature of those to

Base.getindex(x::AbstractExpr, I::Union{<:AbstractVector{Bool}, <:BitVector})

? Instead of adding two new methods. Then we would only need to add the checking in one place later.

I'm OK with anything :-).

@ericphanson , Done.

By the way, didn't contribute anything real here. All credit is you and @odow which I basically copied his text.
So the credit should be yours.

Fixes jump-dev#707.

odow · 2024-10-27T20:39:36Z

src/atoms/IndexAtom.jl

+function Base.getindex(
+    x::AbstractExpr,
+    I::Union{AbstractMatrix{Bool},<:BitMatrix},
+)
    return [xi for (xi, ii) in zip(x, I) if ii]


Hmm. So these methods have been there forever:
92c93fd
but they don't really with the vibe of Convex because they return vector/matrices instead of a new atom.

codecov · 2024-10-27T21:05:59Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.24%. Comparing base (cc71b51) to head (3af9f28).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #708      +/-   ##
==========================================
+ Coverage   98.21%   98.24%   +0.02%     
==========================================
  Files          87       87              
  Lines        5214     5186      -28     
==========================================
- Hits         5121     5095      -26     
+ Misses         93       91       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

odow · 2024-10-27T21:27:07Z

So @RoyiAvital's example now works (with a small tweak).

It's a bit annoying that BitVector requires different syntax to Vector{Int}:

julia> using Convex

julia> using ECOS

julia> N = 100
100

julia> x = Variable(N)
Variable
size: (100, 1)
sign: real
vexity: affine
id: 178…163

julia> y = rand(N);

julia> threshold = sort(y; rev = true)[5]
       # BitVector
0.9244665817970124

julia> p = y .>= threshold;

julia> constraints = x[p] .== y[p]
5-element Vector{Constraint{MathOptInterface.Zeros}}:
 == constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ [-0.951834;;]
 == constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ [-0.993115;;]
 == constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ [-0.977036;;]
 == constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ [-0.983307;;]
 == constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ [-0.924467;;]

julia> model = minimize(sumsquares(x - vY), constraints)
Problem statistics
  problem is DCP         : true
  number of variables    : 1 (100 scalar elements)
  number of constraints  : 5 (5 scalar elements)
  number of coefficients : 106
  number of atoms        : 12

Solution summary
  termination status : OPTIMIZE_NOT_CALLED
  primal status      : NO_SOLUTION
  dual status        : NO_SOLUTION

Expression graph
  minimize
   └─ qol (convex; positive)
      ├─ + (affine; real)
      │  ├─ 100-element real variable (id: 178…163)
      │  └─ 100×1 Matrix{Float64}
      └─ [1;;]
  subject to
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.951834;;]
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.993115;;]
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.977036;;]
   ⋮


julia> solve!(model, ECOS.Optimizer)
       # findall
[ Info: [Convex.jl] Compilation finished: 0.0 seconds, 265.953 KiB of memory allocated

ECOS 2.0.8 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
 0  +0.000e+00  -3.537e-03  +7e+01  2e-01  8e-04  1e+00  4e+01    ---    ---    1  1  - |  -  - 
 1  -3.848e-03  -4.005e-04  +1e+00  6e-03  2e-05  2e-02  7e-01  0.9816  1e-04   1  1  1 |  0  0
 2  +7.394e-02  +1.124e-01  +5e-01  3e-03  7e-06  5e-02  3e-01  0.6382  8e-02   1  2  2 |  0  0
 3  -5.985e-01  +4.737e-01  +5e-01  5e-02  5e-05  1e+00  3e-01  0.2932  8e-01   2  2  2 |  0  0
 4  +7.193e-01  +7.258e-01  +4e-02  3e-03  3e-06  1e-02  2e-02  0.9329  3e-04   2  2  2 |  0  0
 5  +1.275e+00  +1.315e+00  +1e-02  3e-04  5e-07  4e-02  7e-03  0.7002  1e-01   2  2  2 |  0  0
 6  +1.318e+00  +1.323e+00  +9e-04  3e-05  5e-08  5e-03  7e-04  0.9101  9e-04   1  1  1 |  0  0
 7  +1.329e+00  +1.331e+00  +3e-04  1e-05  2e-08  3e-03  2e-04  0.8508  2e-01   1  1  1 |  0  0
 8  +1.332e+00  +1.332e+00  +2e-05  1e-06  2e-09  2e-04  2e-05  0.9118  1e-03   1  1  1 |  0  0
 9  +1.332e+00  +1.332e+00  +5e-06  3e-07  4e-10  5e-05  5e-06  0.8998  1e-01   1  1  1 |  0  0
10  +1.332e+00  +1.332e+00  +6e-07  3e-08  5e-11  7e-06  6e-07  0.8800  3e-03   1  1  1 |  0  0
11  +1.332e+00  +1.332e+00  +2e-07  3e-08  1e-11  2e-06  1e-07  0.8801  1e-01   1  1  1 |  0  0
12  +1.332e+00  +1.332e+00  +2e-08  2e-09  2e-12  3e-07  2e-08  0.8488  8e-03   1  1  1 |  0  0
13  +1.332e+00  +1.332e+00  +6e-09  5e-09  1e-12  8e-08  6e-09  0.8578  1e-01   1  1  1 |  0  0

OPTIMAL (within feastol=4.9e-09, reltol=4.8e-09, abstol=6.4e-09).
Runtime: 0.000621 seconds.

Problem statistics
  problem is DCP         : true
  number of variables    : 1 (100 scalar elements)
  number of constraints  : 5 (5 scalar elements)
  number of coefficients : 106
  number of atoms        : 12

Solution summary
  termination status : OPTIMAL
  primal status      : FEASIBLE_POINT
  dual status        : FEASIBLE_POINT
  objective value    : 1.3321

Expression graph
  minimize
   └─ qol (convex; positive)
      ├─ + (affine; real)
      │  ├─ 100-element real variable (id: 178…163)
      │  └─ 100×1 Matrix{Float64}
      └─ [1;;]
  subject to
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.951834;;]
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.993115;;]
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ index (affine; real)
   │     │  └─ …
   │     └─ [-0.977036;;]
   ⋮


julia> p = findall(y .>= threshold);

julia> constraints = x[p] == y[p]
== constraint (affine)
└─ + (affine; real)
   ├─ index (affine; real)
   │  └─ 100-element real variable (id: 178…163)
   └─ 5×1 Matrix{Float64}

julia> model = minimize(sumsquares(x - vY), [constraints])
Problem statistics
  problem is DCP         : true
  number of variables    : 1 (100 scalar elements)
  number of constraints  : 1 (5 scalar elements)
  number of coefficients : 106
  number of atoms        : 4

Solution summary
  termination status : OPTIMIZE_NOT_CALLED
  primal status      : NO_SOLUTION
  dual status        : NO_SOLUTION

Expression graph
  minimize
   └─ qol (convex; positive)
      ├─ + (affine; real)
      │  ├─ 100-element real variable (id: 178…163)
      │  └─ 100×1 Matrix{Float64}
      └─ [1;;]
  subject to
   └─ == constraint (affine)
      └─ + (affine; real)
         ├─ index (affine; real)
         │  └─ …
         └─ 5×1 Matrix{Float64}


julia> solve!(model, ECOS.Optimizer)
[ Info: [Convex.jl] Compilation finished: 0.0 seconds, 252.016 KiB of memory allocated

ECOS 2.0.8 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
 0  +0.000e+00  -3.537e-03  +7e+01  2e-01  8e-04  1e+00  4e+01    ---    ---    1  1  - |  -  - 
 1  -3.848e-03  -4.005e-04  +1e+00  6e-03  2e-05  2e-02  7e-01  0.9816  1e-04   1  1  1 |  0  0
 2  +7.394e-02  +1.124e-01  +5e-01  3e-03  7e-06  5e-02  3e-01  0.6382  8e-02   1  2  2 |  0  0
 3  -5.985e-01  +4.737e-01  +5e-01  5e-02  5e-05  1e+00  3e-01  0.2932  8e-01   2  2  2 |  0  0
 4  +7.193e-01  +7.258e-01  +4e-02  3e-03  3e-06  1e-02  2e-02  0.9329  3e-04   2  2  2 |  0  0
 5  +1.275e+00  +1.315e+00  +1e-02  3e-04  5e-07  4e-02  7e-03  0.7002  1e-01   2  2  2 |  0  0
 6  +1.318e+00  +1.323e+00  +9e-04  3e-05  5e-08  5e-03  7e-04  0.9101  9e-04   1  1  1 |  0  0
 7  +1.329e+00  +1.331e+00  +3e-04  1e-05  2e-08  3e-03  2e-04  0.8508  2e-01   1  1  1 |  0  0
 8  +1.332e+00  +1.332e+00  +2e-05  1e-06  2e-09  2e-04  2e-05  0.9118  1e-03   1  1  1 |  0  0
 9  +1.332e+00  +1.332e+00  +5e-06  3e-07  4e-10  5e-05  5e-06  0.8998  1e-01   1  1  1 |  0  0
10  +1.332e+00  +1.332e+00  +6e-07  3e-08  5e-11  7e-06  6e-07  0.8800  3e-03   1  1  1 |  0  0
11  +1.332e+00  +1.332e+00  +2e-07  3e-08  1e-11  2e-06  1e-07  0.8801  1e-01   1  1  1 |  0  0
12  +1.332e+00  +1.332e+00  +2e-08  2e-09  2e-12  3e-07  2e-08  0.8488  8e-03   1  1  1 |  0  0
13  +1.332e+00  +1.332e+00  +6e-09  5e-09  1e-12  8e-08  6e-09  0.8578  1e-01   1  1  1 |  0  0

OPTIMAL (within feastol=4.9e-09, reltol=4.8e-09, abstol=6.4e-09).
Runtime: 0.000871 seconds.

Problem statistics
  problem is DCP         : true
  number of variables    : 1 (100 scalar elements)
  number of constraints  : 1 (5 scalar elements)
  number of coefficients : 106
  number of atoms        : 4

Solution summary
  termination status : OPTIMAL
  primal status      : FEASIBLE_POINT
  dual status        : FEASIBLE_POINT
  objective value    : 1.3321

Expression graph
  minimize
   └─ qol (convex; positive)
      ├─ + (affine; real)
      │  ├─ 100-element real variable (id: 178…163)
      │  └─ 100×1 Matrix{Float64}
      └─ [1;;]
  subject to
   └─ == constraint (affine)
      └─ + (affine; real)
         ├─ index (affine; real)
         │  └─ …
         └─ 5×1 Matrix{Float64}

RoyiAvital · 2024-10-29T19:09:22Z

@odow , I thought @ericphanson has already reviewed it.
Or are you after a review of your changes?

odow · 2024-10-29T19:37:00Z

I'm after a new review because of the different syntax required for indexing with BitVector and Vector{Int}.

odow · 2024-11-04T23:55:26Z

Thoughts @ericphanson?

RoyiAvital added 2 commits October 27, 2024 09:21

Add Support for BitVector and BitMatrix Indexing

ab44000

Fixes jump-dev#707 .

Add Support for BitVector and BitMatrix Indexing

0d5b3aa

Fixes jump-dev#707.

ericphanson reviewed Oct 27, 2024

View reviewed changes

RoyiAvital and others added 3 commits October 27, 2024 14:32

Add Support for BitVector and BitMatrix Indexing

e84d666

Fixes jump-dev#707.

Update IndexAtom.jl

3ade074

Update test_atoms.jl

52fc1f9

odow reviewed Oct 27, 2024

View reviewed changes

Update test_atoms.jl

3af9f28

odow requested a review from ericphanson October 27, 2024 22:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Support for `BitVector` and `BitMatrix` Indexing #708

Add Support for `BitVector` and `BitMatrix` Indexing #708

RoyiAvital commented Oct 27, 2024

ericphanson Oct 27, 2024

RoyiAvital Oct 27, 2024

ericphanson Oct 27, 2024 •

edited

Loading

RoyiAvital Oct 27, 2024

RoyiAvital Oct 27, 2024

RoyiAvital Oct 27, 2024

odow Oct 27, 2024

codecov bot commented Oct 27, 2024 •

edited

Loading

odow commented Oct 27, 2024

RoyiAvital commented Oct 29, 2024

odow commented Oct 29, 2024

odow commented Nov 4, 2024

Add Support for BitVector and BitMatrix Indexing #708

Are you sure you want to change the base?

Add Support for BitVector and BitMatrix Indexing #708

Conversation

RoyiAvital commented Oct 27, 2024

ericphanson Oct 27, 2024

Choose a reason for hiding this comment

RoyiAvital Oct 27, 2024

Choose a reason for hiding this comment

ericphanson Oct 27, 2024 • edited Loading

Choose a reason for hiding this comment

RoyiAvital Oct 27, 2024

Choose a reason for hiding this comment

RoyiAvital Oct 27, 2024

Choose a reason for hiding this comment

RoyiAvital Oct 27, 2024

Choose a reason for hiding this comment

odow Oct 27, 2024

Choose a reason for hiding this comment

codecov bot commented Oct 27, 2024 • edited Loading

Codecov Report

odow commented Oct 27, 2024

RoyiAvital commented Oct 29, 2024

odow commented Oct 29, 2024

odow commented Nov 4, 2024

Add Support for `BitVector` and `BitMatrix` Indexing #708

Add Support for `BitVector` and `BitMatrix` Indexing #708

ericphanson Oct 27, 2024 •

edited

Loading

codecov bot commented Oct 27, 2024 •

edited

Loading