I am trying to access the hidden layers when using TransformerEncoder and TransformerEncoderLayer. I could not find anything like that in the source code for these classes.
I am not using hugging face but I know one can get hidden_states and last_hidden_state. I am looking for something similar.
Do you know how I can access them?
Topic encoder transformer pytorch python
Category Data Science
Not exactly sure which hidden layer you are looking for, but the TransformerEncoderLayer class simply has the different layers as attributes which can easily access (e.g. self.linear1 or self.self_attn). The TransformerEncoder is simply a stack of TransformerEncoderLayer layers, which are stored in the layer attribute as a list. For each layer in the list you can then access the hidden layers as mentioned.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.