‹ All episodes

Argmax

3: VICReg

March 21, 2022 Vahe Hagopian, Taka Hasegawa, Farrukh Rahman Season 1 Episode 3
3: VICReg
Argmax
More Info
Argmax
3: VICReg
Mar 21, 2022 Season 1 Episode 3
Vahe Hagopian, Taka Hasegawa, Farrukh Rahman

Todays paper: VICReg (https://arxiv.org/abs/2105.04906)

Summary of the paper
VICReg prevents representation collapse using a mixture of variance, invariance and covariance when calculating the loss. It does not require negative samples and achieves great performance on downstream tasks.

Highlights of discussion

  • The VICReg architecture (Figure 1)
  • Sensitivity to hyperparameters (Table 7)
  • Top 5 metric usefulness
Apple Podcasts Spotify Amazon Music Overcast Castro Castbox +
Show Notes

Todays paper: VICReg (https://arxiv.org/abs/2105.04906)

Summary of the paper
VICReg prevents representation collapse using a mixture of variance, invariance and covariance when calculating the loss. It does not require negative samples and achieves great performance on downstream tasks.

Highlights of discussion

  • The VICReg architecture (Figure 1)
  • Sensitivity to hyperparameters (Table 7)
  • Top 5 metric usefulness