Prerequisites
UtilsRL verison when proposing this request
0.4.8
What I am expecting
For now although we have RNN and attention blocks implemented, we cannot use them directly because it's somewhat incompatible with the RL actors and critics.
- RNN will yield intermediate cell states and cell hiddens each time we forward-pass the tensors. The block will return the intermediate products as result, but RL actors and critics are unable to collect and pass them to buffers.
- Attention blocks may need a 'reduce' method the absorb the
timestep dimension. For example, in Decision Transformer, we pass in a sequence of tuples but only need the last action prediction as the output of our actor.
In short, we need a bridging module between the backends and frontends to efficiently 1) collect the intermediate results, and 2) reduce the dimension and arrange the results into their expected shape.
Possible solutions
No response
Any additional messages which might help
No response
Urgency
Very urgent, this feature is relevant and should be implemented immediately.
Prerequisites
UtilsRL verison when proposing this request
0.4.8
What I am expecting
For now although we have RNN and attention blocks implemented, we cannot use them directly because it's somewhat incompatible with the RL actors and critics.
timestepdimension. For example, in Decision Transformer, we pass in a sequence of tuples but only need the last action prediction as the output of our actor.In short, we need a bridging module between the backends and frontends to efficiently 1) collect the intermediate results, and 2) reduce the dimension and arrange the results into their expected shape.
Possible solutions
No response
Any additional messages which might help
No response
Urgency
Very urgent, this feature is relevant and should be implemented immediately.