It has become increasingly common to pre-train models to develop general-purpose abilities and knowledge that can then be “transferred” to downstream tasks.

In applications of transfer learning to computer vision, pre-training is typically done via supervised learning on a large labelled dataset like ImageNet. In contrast, modern techniques for transfer learning in NLP often pre-train using unsupervised learning on unlabeled data.

In spite of being widely popular there are still few pressing questions bothering transfer learning in ML. Read More