Using Experts’ Perceptual Skill for Dermatological Image Segmentation

                            http://mathewsopenaccess.com/dermatology-current-issue.html

There is a growing reliance on imaging equipment in medical domain, hence medical experts’ specialized visual perceptual capability becomes the key of their superior performance. In this paper, we propose a principled generative model to detect and segment out dermatological lesions by exploiting the experts’ perceptual expertise represented by their patterned eye movement behaviors during examining and diagnosing dermatological images. The image superpixels’ diagnostic significance levels are inferred based on the correlations between their appearances and the spatial structures of the experts’ signature eye movement patterns. In this process, the global relationships between the superpixels are also manifested by the spans of the signature eye movement patterns. Our model takes into account these dependencies between experts’ perceptual skill and image properties to generate a holistic understanding of cluttered dermatological images. A Gibbs sampler is derived to use the generative model’s structure to estimate the diagnostic significance and lesion spatial distributions from superpixel-based representation of dermatological images and experts’ signature eye movement patterns. We demonstrate the effectiveness of our approach on a set of dermatological images on which dermatologists’ eye movements are recorded. It suggests that the integration of experts’ perceptual skill and dermatological images is able to greatly improve medical image understanding and retrieval.

Image understanding in knowledge-rich domains is particularly challenging, because experts’ domain knowledge and perceptual expertise are demanded to transform image pixels into meaningful contents. This motivates using active learning methods to incorporate experts’ specialized capability into this process in order to improve the segmentation performance. However, traditional knowledge acquisition used by active learning methods, such as manual markings, annotations and verbal reports, poses series of significant problems. Because tacit (implicit) knowledge as an integral part of expertise is not consciously accessible to experts, it is difficult for them to identify exactly the diagnostic reasoning processes involved in decision-making, On the other hand, empirical studies suggest that eye movements, as both direct input and measurable output of real time signal processing in the brain, provide us an effective and reliable measure of both human cognitive processing and perceptual skill. Recently, studies try to incorporate perceptual skill into image understanding approaches. Experts’ eye movement data are projected into image feature space to evaluate feature saliency by weighting local features close to each fixation. Salient image features are then mapped back to spatial space in order to highlight regions of interest and at-tention selection. Furthermore, a conceptual framework is developed to measure image feature relevance based on corresponding patterned eye movement deployments defined by pair-wise comparison between multiple experts’ eye movement data. 

In an image retrieval study images are segmented generically based on image features first. In the later step of image matching, the similarity measures between query image segments and candidate images are weighted by subjects’ fixation data. There are significant limits associated with these approaches. Some of them treat eye movements as a static process by solely using fixation locations without taking the dynamic nature into account. Other studies apply dynamic models to capture sequential information of eye movements, but the cardinality is heuristic and the eye movement descriptions are limited. Since these studies directly use the observed eye movement data, which are noisy and inconsistent, to evaluate image features, their methods’ reliability and robustness are undermined.