Mixed models: How to generate predictions "fixed" (marginalised?) over the random factors

[DominiqueMakowski]

Let's start with a simple linear mixed model that takes participant as random factors.

using Turing, StatsFuns, DataFrames, Random

# Generate correlated x and y data
data = DataFrame(x=1:100, y=1:100, participant=repeat(1:10, inner=10))
data.y = data.x .+ randn(100) .* 10

# Linear
@model function model_Linear(y, x, participant, n)

    # Priors - Fixed Effects
    μ_intercept ~ Normal(0, 0.5)
    μ_x ~ Normal(0, 0.5)

    σ ~ truncated(Normal(0.0, 1), lower=0)

    # Priors - Random Effects
    μ_intercept_random_sd ~ truncated(Normal(0.0, 0.3), lower=0)
    μ_x_random_sd ~ truncated(Normal(0, 0.3), lower=0)

    # Participant-level priors
    μ_intercept_random ~ filldist(Normal(0, μ_intercept_random_sd), n)
    μ_x_random ~ filldist(Normal(0, μ_x_random_sd), n)

    for i in 1:length(y)
        # Compute μ
        μ = μ_intercept + μ_intercept_random[participant[i]]
        μ += (μ_x + μ_x_random[participant[i]]) * x[i, 1]

        # Likelihood
        y[i] ~ Normal(μ, σ)
    end
end

posteriors = sample(model_Linear(data.y, data.x, data.participant, 10), NUTS(), 200)

I can easily generate predictions on the original data, useful for instance for posterior predictive checks.

# Predictions on whole dataset
pred = predict(model_Linear(fill(missing, 100), data.x, data.participant, 10), posteriors)

However, often for visualization purposes / postprocessing (marginal effects computation) we would need to generate predictions on new data, for instance on a data "grid" with specific values. While these values are often specifically chosen for the fixed effects, we are often interested in "marginalising" over the random effects, i.e., having the average prediction for the average random factor.

I thought that one way to go about it would be to also set participant as missing, but that doesn't seem to be the correct way.

# Predictions on new data "grid"
grid = DataFrame(x=[0, 50, 100], participant=[missing, missing, missing])

pred2 = predict(model_Linear(fill(missing, 3), grid.x, grid.participant, 3), posteriors)

Any advice on how to achieve something like that?

[torfjelde]

we are often interested in "marginalising" over the random effects, i.e., having the average prediction for the average random factor.

Maybe I'm misunderstanding, but do you just want

μ_intercept + μ_x * x[i, 1]

?

If you want to marginalize over the participants, then you need a notion of what the distribution of the participants are (which it doesn't seem like you have here, IIUC)

[DominiqueMakowski]

Ideally, I want μ assuming that μ_x_random[participant[i]]=0 and μ_intercept_random[participant[i]] = 0 (basically, predictions without the deviations from the fixed effects due to specific participants)

[DominiqueMakowski]

In other words, I'd like to nullify, for the predictions, any of the influence created by participant if that makes sense