Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AbstractDifferentiation.jl #29

Open
mateuszbaran opened this issue Feb 10, 2021 · 3 comments
Open

AbstractDifferentiation.jl #29

mateuszbaran opened this issue Feb 10, 2021 · 3 comments

Comments

@mateuszbaran
Copy link
Member

TODO: check if https://github.com/JuliaDiff/AbstractDifferentiation.jl can be used for specifying differentiation backend (also relevant to Manopt.jl).

@kellertuer
Copy link
Member

Yes, this is very interesting. We also have to discuss, where to put AD stuff (here or into Manopt) as well as for Hessians for example how/where to convert from the euclidean hessian to the Riemannian one.

To elaborate: The Matlab package has a lot of these functions, see for example

https://github.com/NicolasBoumal/manopt/blob/fd98d5417c7084f9bfdf8b086fbdd66799fafca5/manopt/manifolds/sphere/spherefactory.m#L122-L127

Where the gradient is fine with me, you have to project the euclidean gradient onto the tangent plane to get the Riemannian one. This might not always be the case, for example for fixed rank matrices, the euclidean one is one matrix, but turning it into the Riemannian one also includes changing the representation accordingly – though this will be done by project, too, for the gradient.

For the hessian we might have to check how to cover this exactly, because it would be nice if one could just take a function from the embedding (say R^{m,n}, the set of all matrices) do Riemannian AD and obtain a Riemannian Hessian.

@mateuszbaran
Copy link
Member Author

mateuszbaran commented Feb 12, 2021

Hessians were discussed a while ago here: JuliaManifolds/Manifolds.jl#202 (comment) . I still think Weingarten maps are a good idea (at least as the default mechanism, it should also be possible to implement Riemannian Hessians without Weingarten maps).

I'm not sure where AD support should be put, maybe in a new package?

@kellertuer
Copy link
Member

Sure, an extra package would also be possible; keeps the library of manifolds cleaner and makes AD independent of the optimisation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants