The article discusses the implementation of a contrastive learning approach in a multi-prototypical setup. It highlights that adding contrastive loss improves performance on various datasets, particularly when using discourse-aware mechanisms that consider relative positions of sentences. Results show that while the discourse-aware variant offers the most efficacy, the approach using memory banks has mixed results, occasionally hindering performance due to positional factors. Notably, the single prototypical variant performs similarly to the best contrastive method, emphasizing the importance of prototypes in consolidating knowledge from nearby instances. Overall, a combination of these methods yields optimal results.
Incorporating contrastive loss improves performance across datasets, with the discourse-aware variant being most effective as it positions similar labels closer in embedding space.
The single prototypical variant competes with the best contrastive variant, suggesting that guiding points through prototypes can aggregate knowledge from neighboring instances.
While macro-F1 performance benefits from memory banks, discourse-awareness leads to diminishing returns due to positional factors affecting the loss calculations.
Combining discourse-aware contrastive learning with multiple prototypes highlights the importance of capturing nuanced relationships within data.
#machine-learning #contrastive-learning #natural-language-processing #prototypical-networks #model-performance
Collection
[
|
...
]