Temporal encoding based on day-of-year and hour-of-day has already been added providing encodings reflecting the cyclical nature of temporal embeddings in the context of (SST) prediction. Nevertheless, a straight postional encoding based on month has been preserved. As discussed if #54 and #64 this may not be optimal, and given the the choice to fold months into the batch dimension (#64) it becomes even less feasible.
In order to preserve an explicit month derived feature (set) within the temporal encoding, it appears advisable to drop the current month PE and replace it by an expansion of the temporal encoding based on month of year.
Temporal encoding based on day-of-year and hour-of-day has already been added providing encodings reflecting the cyclical nature of temporal embeddings in the context of (SST) prediction. Nevertheless, a straight postional encoding based on month has been preserved. As discussed if #54 and #64 this may not be optimal, and given the the choice to fold months into the batch dimension (#64) it becomes even less feasible.
In order to preserve an explicit month derived feature (set) within the temporal encoding, it appears advisable to drop the current month PE and replace it by an expansion of the temporal encoding based on month of year.