Element wise addition after multiplication

[pathfinder49]

Attempting element wise addition to a variable after multiplication raises an error. If I understand the supported operations page correctly this should be possible?

Examples

1 * Variable(1) .+ ones(1)

does not compile (ERROR: MethodError: no method matching iterate(::Convex.MultiplyAtom))

1 .* Variable(1) .+ ones(1)

does not compile (ERROR: MethodError: no method matching iterate(::Convex.DotMultiplyAtom))

1 * Variable(1) + ones(1)

works fine.

[ericphanson]

Thanks for raising the issue; you're right that .+ is mentioned as supported. This is also sort of discussed in #218.

I'm not quite sure what we should do here. Maybe the first step is to just update the docs to not say that broadcasting is supported; we could also try to give better errors in this situation.

The problem is that elementwise manipulations on Convex.jl Variables are inefficient, since we treat a Variable of any size as a single object (not as an array of scalar objects like JuMP does), so given the option of + or .+ that should do the same thing mathematically, we should encourage the efficient one (+).

One can also do something like X = [Variable() for i = 1:3, j = 1:2] to make an array of scalar variables. Then broadcasting works, but you end up with typeof(X) == Array{Variable,2} which doesn't have methods for other operations (like norms or other atoms). So we could try to support that somehow, but that's probably a big project.

Do you have a particular use case for broadcasted operations? Or does just switching to eg + instead of .+ work for you?

[pathfinder49]

I came across this when trying to minimise multiple competing cost terms.

Here is an example:

# setup
using Convex
var = Variable(10)
response_mat = rand(100, 10)
competing_target_responses = rand(100, 3)
weights = rand(100, 3)
constraint = var <=10.
constraint += var >= -10.

# cost function to be minimised
response_var = response_mat * var  # matrix multiplication
# deviation from several targets (bug here!)
deviation = response_var .- competing_target_responses  
cost = sum(deviation .* deviation .* weights)

problem = minimize(cost, constraint)

I was able to work around the broadcasting with a for loop.

[thompsonmj]

I'm also coming across the error:

MethodError: no method matching iterate(::Convex.MultiplyAtom)

Here is the full problem:

function chisq(α,β,Y,ȳ)

	χ² = sum(square(Y - (α .+ β*ȳ)))	

	return χ²
end

α = Variable(n_replicates,1)
β = Variable(n_replicates,1)

Y = reshape(data_anchor[100:900,:,1],(102,801))
ȳ = reshape(data_anchor_mean[100:900,1],(1,801))


problem = minimize(chisq(α,β,Y,ȳ))

solve!(problem, () -> SCS.Optimizer(verbose=false))

It doesn't appreciate my attempt to do α .+ β*ȳ, where α is a size (102,1) Variable and β*ȳ is a (102,801) Convex.MultiplyAtom. I would just like α to be added to each column of β*ȳ.

[odow]

Closing in favor of #479