From NMF to FunkMF or FMRegressor

[ananiask8]

Hello. I have a use case which I was solving using surprise/NMF but I started experimenting with river in order to enable incremental learning. My user-item matrix is sparse, where values range from 0 to 1: think of them as probability values. I see river doesn't support NMF but this is not essential for me. My main question is about: 1) FunkMF vs FMRegressor; 2) initialization; and, 3) the optimizer to use.

Since in my use case the entries are probabilities, I tried initializing with a constant value np.sqrt(0.5/n_factors); the performance for me was comparable to what I was getting with surprise but when I inspected the vectors I saw they were basically the same value repeated across factors for a particular user or item (e.g. user 1 could have factors [3, 3, 3], while user 2 could have factors [1, 1, 1]). I'm guessing some amount of randomization is necessary so that the gradients of the individual factors are more decoupled from each other? In any case, just wanted to hear a word of confirmation about this.

Currently I'm working with a squared loss and AdaGrad (initially I was trying SGD but I've seen better results with AdaGrad) using FunkMF. My final question would be about your thoughts about using FMRegressor vs FunkMF in this particular use case. At the moment I am not interested in including any other features, but rather in the vanilla matrix factorization task.

[MaxHalford]

Since in my use case the entries are probabilities, I tried initializing with a constant value np.sqrt(0.5/n_factors); the performance for me was comparable to what I was getting with surprise but when I inspected the vectors I saw they were basically the same value repeated across factors for a particular user or item (e.g. user 1 could have factors [3, 3, 3], while user 2 could have factors [1, 1, 1]). I'm guessing some amount of randomization is necessary so that the gradients of the individual factors are more decoupled from each other? In any case, just wanted to hear a word of confirmation about this.

@gbolmier can you take a look if you're around? @ananiask8 do you have some data to work with?

Currently I'm working with a squared loss and AdaGrad (initially I was trying SGD but I've seen better results with AdaGrad) using FunkMF. My final question would be about your thoughts about using FMRegressor vs FunkMF in this particular use case. At the moment I am not interested in including any other features, but rather in the vanilla matrix factorization task.

Good question. Again, @gbolmier will know more than I do.

[gbolmier]

Hey @ananiask8 👋

I'm guessing some amount of randomization is necessary so that the gradients of the individual factors are more decoupled from each other? In any case, just wanted to hear a word of confirmation about this.

Yes, I think you want to add randomization here to help the model learning different weights focusing on different aspects of the problem.

Currently I'm working with a squared loss and AdaGrad (initially I was trying SGD but I've seen better results with AdaGrad) using FunkMF. My final question would be about your thoughts about using FMRegressor vs FunkMF in this particular use case. At the moment I am not interested in including any other features, but rather in the vanilla matrix factorization task.

I would suggest you to go through our couple of guides on matrix factorization if you haven't yet, might be useful to you. From my own experience, I had more success with SGD for optimization and an FMRegressor for modelling, but every problem is different. I also can't recall a case where reco.BiasedMF was not performing better than FunkMF. However, I had a case once where BiasedMF was doing better than FMRegressor but there wasn't a lot of signal to learn from the data, I guess mostly because of the bias–variance tradeoff.

Hope this helps :)