To achieve accurate lesion classification, features from the lesion
attention maps can be extracted by an encoder, such that high-level
lesion features can be captured for the classifier module. Thus, in each
branch, an encoder is incorporated after the segmentor to extract each
domain’s specific features. Besides, we propose to fuse the lesion
features and the prostate features to boost the classification accuracy.
Skip connection and concatenation operations are introduced to reuse
prostate features from the segmentors.
We design a domain transfer module (in Figure 4 ) without
requiring target labels in the training process. The semantics features
from both the prostate region and attention map are fused, such that
deep coral features from fully connected (FC) layers can be captured for
feature affinity. Deep Coral loss [25] is employed
to minimize cross-domain feature distribution discrepancy, owing to its
generality, transferability, and ease of implementation. It is defined
as the difference of second-order covariances between domains. Our
domain transfer loss \({\mathcal{L}_{Coral}}\) is defined as: