BERT has a mouth, and it must speak: BERT as a Markov random field language model
We show that BERT (Devlin et al., 2018) is a Markov random field language model. This
formulation gives way to a natural procedure to sample sentences from BERT. We generate
from BERT and find that it can produce high-quality, fluent generations. Compared to the
generations of a traditional left-to-right language model, BERT generates sentences that are
more diverse but of slightly worse quality.
formulation gives way to a natural procedure to sample sentences from BERT. We generate
from BERT and find that it can produce high-quality, fluent generations. Compared to the
generations of a traditional left-to-right language model, BERT generates sentences that are
more diverse but of slightly worse quality.
[CITATION][C] BERT has a mouth, and it must speak: BERT as a markov random field language model. arXiv
A Wang, K Cho - arXiv preprint arXiv:1902.04094, 2019
Showing the best results for this search. See all results