Generalizing the Evaluation of Medical Image Processing Tools by Use of Gabor Patterns

Objective: Tools for medical image processing are usually evaluated by observers with radiological experience and with complex tasks. For easing evaluation of filtering and enhancement tools, the observer's task can be generalized. Methods: By describing aspects of the MCS method (Mammographic Contrast Sensitivity) we illustrate issues of selecting a metric for assessing visual performance, the observer's task and the image material to be used, aiming at a generalization of the design of studies for the evaluation of medical image processing tools. Concerning the metric, we distinguish acuity from contrast sensitivity. With respect to the observer's task, we distinguish tasks of discrimination from those at a higher level of recognition, Finally, we show the advantage of using medical images for evaluating image processing tools by comparing the results for measurements on homogeneous background and mammographic images. Results:The perceptual level of the observer's task and the complexity of the used image material influences the outcome of observer studies, particularly also from crowding effects. The design of a study should minimize the impact of the observer's experience on the outcome. This can be achieved by using nonanatomical, standardized perceptual targets like Gabor patterns, used in the context of medical images. Conclusions: Understanding the concepts of perception helps designing observer studies that are as complex as required, but at the same time as simple and general as possible. Performing an observer study may be simplified by a study design which does not require radiological experience of the observers, if the study aims at the evaluation of tools that shall support basic perception tasks, such as e.g. contrast enhancement.