How to start forward immediately without waiting for results of pull?

[DanlinJia]

It seems that MXNet now only allows starting the forward function after the pull function of the last iteration is finished.
I wonder is there a trick that can help remove this kind of solid dependency and therefore, enable an immediate forward function after the last backward function?

[szha]

what's the expected behavior when calling forward without an up-to-date copy of the parameter?

[DanlinJia]

Yes, you are right. It makes no sense to do forward on the old parameters.
I think my real question is how to coordinate two workers to perform specific execution order at run time.
For example, I want worker 2 to start the forward path after worker 1 finishing its backwward path.
I'm curious about what kind of mechanism we can use in a symbolic programming model to enable inter-process communications?