Get all layers hidden states of deberta(Huggingface).

Usually, we use deberta like:

all_plm_output = self.plm(document_batch[doc_id][:self.plm_batch_size, 0],  # [1, 512]
                                      token_type_ids=document_batch[doc_id][:self.plm_batch_size, 1],
                                      attention_mask=document_batch[doc_id][:self.plm_batch_size, 2])
# self.plm is a pretrained deberta-v3-base

And we can get the output as follows:

To get all the hidden layers’ output, I check the docs of deberta in huggingface.Click here to docs: DeBERTa (huggingface.co)

forward() has a argument named “output_hidden_states”, which set to “True” can output all hidden states.

code:

all_plm_output = self.plm(document_batch[doc_id][:self.plm_batch_size, 0],  # [1, 512]
                                      token_type_ids=document_batch[doc_id][:self.plm_batch_size, 1],
                                      attention_mask=document_batch[doc_id][:self.plm_batch_size, 2],
                                      output_hidden_states=True)

result:

We can get all hidden states in “hidden_states” now.