Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Error when cls_pooling="mean" or cls_pooling="max" #3552

Open
chelseagzr opened this issue Oct 7, 2024 · 0 comments · May be fixed by #3558
Open

[Bug]: Error when cls_pooling="mean" or cls_pooling="max" #3552

chelseagzr opened this issue Oct 7, 2024 · 0 comments · May be fixed by #3558
Labels
bug Something isn't working

Comments

@chelseagzr
Copy link

Describe the bug

When cls_pooling is "mean" or "max", document embeddings fail to be created.
After digging into the codes, I think the issue is: return result is missing in the functions document_mean_pooling and document_max_pooling.

To Reproduce

from flair.embeddings import TransformerDocumentEmbeddings
from flair.data import Sentence

embeddings = TransformerDocumentEmbeddings(
    model="xlm-roberta-base",
    layers="-1",
    cls_pooling="mean",
    allow_long_sentences=True
)
sentence = Sentence("Today is a good day")
embeddings.embed(sentence)

Expected behavior

Document embeddings using mean pooling are created for the sentence.

Logs and Stack traces

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[4], line 11
      4 embeddings = TransformerDocumentEmbeddings(
      5     model="xlm-roberta-base",
      6     layers="-1",
      7     cls_pooling="max",
      8     allow_long_sentences=True
      9 )
     10 sentence = Sentence("Today is a good day")
---> 11 embeddings.embed(sentence)

File /pyzr/active_venv/lib/python3.12/site-packages/flair/embeddings/base.py:50, in Embeddings.embed(self, data_points)
     47     data_points = [data_points]
     49 if not self._everything_embedded(data_points):
---> 50     self._add_embeddings_internal(data_points)
     52 return data_points

File /pyzr/active_venv/lib/python3.12/site-packages/flair/embeddings/transformer.py:709, in TransformerBaseEmbeddings._add_embeddings_internal(self, sentences)
    707 if self.document_embedding:
    708     document_embedding = embeddings["document_embeddings"]
--> 709     self.__extract_document_embeddings(document_embedding, sentences)
    711 if self.token_embedding:
    712     token_embedding = embeddings["token_embeddings"]

File /pyzr/active_venv/lib/python3.12/site-packages/flair/embeddings/transformer.py:693, in TransformerBaseEmbeddings.__extract_document_embeddings(self, sentence_hidden_states, sentences)
    692 def __extract_document_embeddings(self, sentence_hidden_states, sentences):
--> 693     for document_emb, sentence in zip(sentence_hidden_states, sentences):
    694         sentence.set_embedding(self.name, document_emb)

TypeError: 'NoneType' object is not iterable

Screenshots

No response

Additional Context

No response

Environment

Versions:

Flair

0.14.0

Pytorch

2.4.1+cu121

Transformers

4.45.1

GPU

False
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
1 participant
@chelseagzr