Multi-task learning (MTL) can boost the performance of individual tasks by mutual learning among multiple related tasks. However, when these tasks assume diverse complexities, their corresponding losses involved in the MTL objective inevitably compete with each other and ultimately make the learning biased towards simple tasks rather than complex ones. To address this imbalanced learning problem, we propose a novel MTL method that can equip multiple existing deep MTL model architectures with a sequential cooperative distillation (SCD) module. Specifically, we first introduce an efficient mechanism to measure the similarity between tasks, and group similar tasks into the same block to allow their cooperative learning from each other. Based on this, the grouped task blocks are sorted in a queue to determine the learning sequence of the tasks according to their complexities estimated with the defined performance indicator. Finally, a distillation between the individual task-specific models and the MTL model is performed block by block from complex to simple manner, achieving a balance between competition and cooperation among learning multiple tasks. Extensive experiments demonstrate that our method is significantly more competitive compared with state-of-the-art methods, ranking No.1 with average performances across multiple datasets by improving 12.95% and 3.72% compared with OMTL and MTLKD, respectively.

The domain adversarial neural network (DANN) methods have been successfully proposed and attracted much attention recently. In DANNs, a discriminator is trained to discriminate the domain labels of features generated by a generator, whereas the generator attempts to confuse it such that the distributions between domains are aligned. As a result, it actually encourages the whole alignment or transfer between domains, while the inter-class discriminative information across domains is not considered. In this paper, we present a Discrimination-Aware Domain Adversarial Neural Network (DA2NN) method to introduce the discriminative information or the discrepancy of inter-class instances across domains into deep domain adaptation. DA2NN considers both the alignment within the same class and the separation among different classes across domains in knowledge transfer via multiple discriminators. Empirical results show that DA2NN can achieve better classification performance compared with the DANN methods.