Skip to content

Autoregressive-models

约 56 个字 6 张图片 预计阅读时间不到 1 分钟

一种特殊的 Bayesian-networks

Fully Visible Sigmoid Belief Network (FVSBN)

Neural Autoregressive Density Estimation (NADE)

RNADE

Masked Autoencoder for Distribution Estimation (MADE)

Modern Architectures

Like RNN, CNN, attention, transformer, GPT….

Summary

Pros

  • Easy to evaluate likelihoods
  • Easy to train

Cons

  • Requires and ordering
  • Generation is sequential
  • Cannot learn features used in an unsupervised way