In all previous experiments, we applied our method in batch mode: we performed multiple training passes over the data for each task.
However, efficiently learning from streaming data might require observing each training sample only once to make sure computation is not becoming a bottleneck.
Unsurprisingly, training for multiple epochs results in better and more robust accuracy on past tasks; it is however worth noting that our method still improves over time in the online learning scenario.
It is possible that, given enough tasks, all three curves would converge.
Collection
[
|
...
]