Add IGO-CMA Optimiser

[MichaelClerx]

And variants?

[MichaelClerx]

@naunauyoh first of all: How are you? Hope you're doing OK in this crazy time.

About IGO, do I remember it correctly that the IGO-CMA algorithm had a fixed learning rate, which made it perform fairly poorly compared to other methods?

Do you think we should add an IGO-CMA method to PINTS?

[ArnaudYoh]

I am doing great. What about you and the team? I hope that everyone is safe. IGO-CMA was definitely performing worse than CMA-ES and indeed had a fixed learning rate (sigma=1). The method itself might not be useful for the users of PINTS but at same time it could provide insight on the inner workings of CMA-ES since IGO-CMA still converges towards the right values. What do you think?

…

On Wed, 8 Apr 2020 at 01:37, Michael Clerx ***@***.***> wrote: @naunauyoh <https://github.com/naunauyoh> first of all: How are you? Hope you're doing OK in this crazy time. About IGO, do I remember it correctly that the IGO-CMA algorithm had a fixed learning rate, which made it perform fairly poorly compared to other methods? Do you think we should add an IGO-CMA method to PINTS? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#915 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYCQOPFR5EKNS65HIKWH7DRLO2NDANCNFSM4ITGVJQA> .

[MichaelClerx]

Thanks Arnaud, I think we're all good too!

I think you're right that it could still be a good idea to implement.

Sticking some refs here for future ref

• Olivier et al. 2017 https://dl.acm.org/doi/10.5555/3122009.3122027
• Akimoto et al. 2020 https://link.springer.com/article/10.1007/s00453-011-9564-8 This one seems to have stuff on setting the (fixed) learning rates, as well as quite compact definitions we might want to compare our CMA-ES results to
• https://www.cs.ox.ac.uk/msctheses/1724/