Information Sciences 167 (2004) 1–7 www.elsevier.com/locate/ins
Artificial color image logic H. John Caulfield a
a,*
, Jian Fu b, Seong-Moo Yoo
c
Alabama A&M University Research Institute, P.O. Box 313, Normal, AL 35762, USA Computer Sciences Department, Alabama A&M University, Normal, AL 35762, USA Electrical and Computer Engineering Department, University of Alabama in Huntsville, Huntsville, AL 35899, USA
b c
Received 10 March 2003; received in revised form 20 June 2003; accepted 17 September 2003
Abstract Artificial Color is simply biomimetic spectral sensing and processing to achieve spectral discrimination. Using Artificial Color means to construct image plane filters; we can then perform logic operations on those filters before applying them to scenes. Boolean and fuzzy logic can both be used. We illustrate the concept by the Boolean AND of two such masks applied to a complex scene. 2003 Elsevier Inc. All rights reserved. Keywords: Color; Artificial Color; Spectral analysis; Image algebra; Image segmentation
1. Introduction It seems worthwhile to begin this discussion with a brief review of that which inspired and guided it – Natural Color. Here are two typical definitions, • ‘‘Color vision consists in the use of wavelength discrimination in the construction of visual representations. A color quality is one that is generated from such processing’’ [1]. • ‘‘Color vision is the ability to detect and analyze changes in the wavelength composition of light’’ [2].
*
Corresponding author. E-mail address: john.caulfi
[email protected] (H. John Caulfield).
0020-0255/$ - see front matter 2003 Elsevier Inc. All rights reserved. doi:10.1016/j.ins.2003.09.027
2
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
Both definitions correctly identify color as a creation of the brain that is attributed to perceived objects and emphasize the role of Natural Color in the functioning of the animal that uses it. Natural Color was invented by nature countless millions of years ago, and indeed it has been invented many times independently [3–5]. The reason nature finds it so useful is that it provides much better discrimination among objects in a scene than does achromatic vision. In parallel, animals and plants have found ways to produce Natural Colors as signifiers of various things – sexual willingness, anger, danger, etc. In humans, most of us use three types of cone cells with spectrally overlapping sensitivity curves to sense the light coming into our eyes. Some of us use only two cone types and others (only some women) use four types. Each experiences the world differently as do the countless other animals that share our world with us. Those detected signals are normalized to eliminate brightness variation and then used to produce spectral discriminants that are attributed by our brains to objects in the world. Those objects have no color; of course, as color is the name we give to the brain-computed discriminant. Artificial Color is the use of data from two or more spectrally overlapping sensor sensitivity curves to compute spectral discriminants to assign to objects in the scene. That is a very broad definition [6]. The first demonstrated Artificial Color [7] was made in the.form of a binary multiplicative map to multiply the original image. It simply retained those parts of the original scene that were recognized spectrally as belonging to the class of interest. In that particular case, we used a color CCD camera to photograph some vegetables on a plate on a patterned tablecloth see Fig. 1. We trained a binary filter pattern for ‘‘green pepper but not snow peas and not carrots’’ using a powerful discrimination method called Margin Setting [8,9]. The detailed training is not the focus of this paper, and other methods might work just as well. We used 15 widely spaced samples of each vegetable. Note that we did not train it on plates or tablecloths. We might expect some parts of them to survive the filtering. The filtered result is shown in Fig. 2. The purpose of this paper is to generalize the work on Artificial Color filters by showing how to combine multiple such filters by Boolean or even fuzzy logic before applying them to the original scene. The image above is so dark that it is hard to understand (because most of the original scene did not survive the filtering). To make it easier to see, we replaced those blacks with whites to form Fig. 3.
2. Filter tasks We can make filters for any task – something Natural Color does not allow. We will illustrate this using a new filter designed for ‘‘vegetables but not plate and not tablecloth.’’ We expect the green pepper and the snow peas and the
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
3
Fig. 1. This is the baseline image used in prior Artificial Color filtering and the one to be used in this paper as well.
Fig. 2. The green pepper survived the filtering but the carrots and snow peas did not. Several other features that did survive are the shadows of the green pepper and snow peas as well as some features in the tablecloth. The camera saturated in the bright parts of the green pepper, so they appeared white not green and were rejected.
carrots to survive, but not the plate or tablecloth. As the table cloth had multiple colors, we took 15 samples of each into our training set. Fig. 4 show the results of filtering the original scene with this new filter. From those two examples, it is easy to imagine numerous other tasks for an Artificial Color filter. We might want to get more subtle and distinguish green
4
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
Fig. 3. Shows the same image as Fig. 2 except that black has been replaced by white for easy viewing. The same replacement will be used in all subsequent images as well.
peppers from their shadows and the shadows of snow peas. That might be possible, because such a filter would not have any tasks relating to plates, tablecloths, or snow peas. This last remark is the most critical one with respect to the value of multiple Artificial Color filters. No filter has to do everything. That is in stark contrast with Natural Color. Primate Natural Color, for instance, uses the same set of colors for innumerable different tasks including
Fig. 4. The Artificial Color filter ‘‘vegetables but not plate and not tablecloth’’ operating on the original image gives this. We have replaced the black with white to make the image easier to view in this and the next image.
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
• • • • • •
5
Telling ripe fruit from ‘‘green’’ fruit. Sensing emotions (embarrassment, anger, and so forth). Recognizing camouflaged predator and prey. Sensing health threats (fever, sunburn, etc.). Separating poisonous plants from edible one. Telling coral snakes from king snakes.
Of course, the price paid for such versatility is inferior performance on a specific task to an Artificial Color designed for that task.
3. Boolean algebra of masks The masks described so far are purely binary. This makes them suitable for Boolean operations – AND, OR, NOT, XOR, etc. The particular Boolean operations depend on the tasks that need to be performed. Operations on the tasks are the same as operations on the corresponding filters. For example, we might want to find portions of the scene that are both ‘‘green pepper and not snow peas and not carrots’’ AND ‘‘vegetables and not plate and not tablecloth.’’ To do this, we can AND the corresponding maps and then use that map to operate on the original scene. In this example, we get the image of Fig. 5.
4. Fuzzifying the process We achieved a binary mask by thresholding a decision function. Suppose, instead, we chose to set the highest value to 1 and let the others fall where they may. We would have a fuzzy filter. Boolean operations would have to be replaced by fuzzy ones. The fuzzy AND, for instance, would give the minimum value of the two filters at each pixel.
5. Remarks The fact that the shadow of the green pepper does not get filtered out actually proves something of great importance. In the shadow region, the predominant illumination is that reflected from the green pepper. Thus, even though the intensity of the green pepper and its shadow are greatly different, the Artificial Colors treat them the same! The advantages of multiple masks and using Boolean operations on them has been discussed and illustrated. These are operations that have no counterparts in Natural Color. That is why even a crude camera like a ‘‘color’’ CCD
6
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
Fig. 5. ANDing the masks used to produce Figs. 2 and 3 yielded a new mask that produced this image when used on the original scene. Ideally, only the green pepper should survive. Clearly this filter is effective but imperfect. It does, however, illustrate the use of Boolean operation on multiple masks – the primary topic of this paper.
camera can give exquisite discrimination for particular tasks. Specialists tend to outperform generalists in the task to which they are specialized, and logical combination can put multiple specialists to work on a complex task no one specialist and certainly no one generalist can perform. It seems worth noting that this work combines well with Ritter’s work on image algebra [10,11]. Indeed, if the masks are regarded as binary images, this is a special case of image algebra.
Acknowledgement This work was performed under Director’s funds from the Army Research Laboratory under Dr. Patti Gillespie, Contract No. DAAD17-02-C-0113.
References [1] M. Matthen, The disunity of color, Philosophical Review 108 (1999) 47–84. [2] MIT Encyclopedia of Cognitive Sciences, Available from . [3] C. Neumeyer, Evolution of colour vision, in: J.R. Cronly-Dillon, R.L. Gregory (Eds.), Evolution of the Eye and Visual System, Macmillan, Basingstoke and London, 1991, pp. 284– 305. [4] E. Thompson, Colour vision, evolution, and perceptual content, Synthese 104 (1995) 1–32.
H. John Caulfield et al. / Information Sciences 167 (2004) 1–7
7
[5] T.H. Goldsmith, Optimization, constraint, and history in the evolution of the eyes, Quarterly Review of Biology 56 (1990) 281–322. [6] H.J. Caulfield, Artificial color, Neurocomputing 51 (2003) 463–465. [7] J. Fu, H.J. Caulfield, S.R. Pulusani, Artificial color vision, Journal of Electronic Imaging, submitted for publication. [8] H.J. Caulfield, A. Karavolos, J.E. Ludman, Improving optical Fourier pattern recognition by accommodating the missing information, Information Sciences, in press. [9] H.J. Caulfield, Generalization in heteroassociative neural networks, Neural Computing, submitted for publication. [10] G.X. Ritter, J.N. Wilson, Handbook of Computer Vision Algorithms in Image Algebra, CRC Press, 1996. [11] G.X. Ritter, J.N. Wilson, J.L. Davidson, Image algebra: an overview, CVGIP: Image Understanding 49 (1990) 297–331.
