this post was submitted on 09 Aug 2024
1 points (100.0% liked)

Machine Learning

2005 readers
2 users here now

founded 5 years ago
MODERATORS
 

When training a transformer on positionally encoded embeddings, should the tgt output embeddings also be positionally encoded? If so, wouldn't the predicted/decoded embeddings also be positionally encoded?

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here