Could we Add Linear Projection Layer in pre-trained model?

### Model description As you know, if our model is too high positional_embedding size, then input vectors become too sparse data, so it leads to a performance decrease. However, I have an idea to overcome it. Before positional embedding, we add a new trainable linear projection layer to reduce its dimension. For example, If our model’s original positional embedding size is 4096, then we enlarge our input size as 8192. Then trainable linear projection layer projects 8192 to 4096 and then we perform positional embedding. let LP is a trainable Linear Projection Layer, PE(X) is positional Embedding. Then our calculation formula is as follows. "Original Structure" X(R^4096)-> PE(X, where X is in R^4096)-> model(PE(X)) "Proposed Structure" X(R^8192)-> LP(X, where X^8192) -> X'(R^4096) -> PE(X') -> model(PE(X')) Can we run this with the current library? If not, can you add? ### Open source status - [ ] The model implementation is available - [ ] The model weights are available ### Provide useful links for the implementation _No response_