How to measure the performance of a domain adaptation /Transfer learning technique?

Question

Given that the performance you achieve depends on how far the target from the source domain is, how can you judge the performance of an algorithm?

Answer 1

You can measure the divergence between the source and target domain using KL-Divergence (there are some ways to estimate k-l divergence e.g. depdended on k-nn algorithm). Then you can check if there is a correlation between the divergence and the accuracy of the models considering a few cases of source-target pairs of datasets. You can compare several algorithms of Transfer Learning/Domain Adaptation using the same source-target datasets.