Removing FC layer at the top of the transformer model

[loukasilias]

Hi all,
I would like to ask you how I can remove the fully connected layer at the top of the transformer layer.
I want to get the output of the transformer networks and not the final prediction.
Moreover, I would like to ask if these models come with the pretrained weights.

Thank you.

[mghayour]

I have same question on this, do we have any updates !?

[mcihadarslanoglu]

maybe this post can help you.