You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
Why exclude down_proj when executing a let? like model.mlp.down_proj.temp_weight = model.mlp.down_proj.weight.
I think the down_proj can be smoothed with up_proj, like smooth_fc_fc_temporary(model.mlp.up_proj,model.mlp.down_proj, model.down_smooth_scale, model.down_smooth_shift)
Am I right?
The text was updated successfully, but these errors were encountered:
Hi,
Why exclude down_proj when executing a let? like
model.mlp.down_proj.temp_weight = model.mlp.down_proj.weight
.I think the down_proj can be smoothed with up_proj, like
smooth_fc_fc_temporary(model.mlp.up_proj,model.mlp.down_proj, model.down_smooth_scale, model.down_smooth_shift)
Am I right?
The text was updated successfully, but these errors were encountered: