Introduction 26 24 2 9 25 13 15 5 6 10 14 22 23 4 16 3 20 blood withdrawal; vein inspection in dark skin; detection of veins through iodide; inspection of varicose veins and nevi pigmentosum. Methods Instrumental Setup 1 2 Figure 1. Experimental setup. Two CMOS-cameras, with apochromatic lenses and dual-band LED-arrays, simultaneously stream Left (L) and Right (R) image data to a dual processor PC. Both cameras captured color images within the visible range (VIS, 400–780 nm) and grey-scale images within the near infrared range (NIR, 910–920 nm) by sequentially switching between LED-array emission bands.  2 Figure 2. Schematic diagram of image aquisition. The sequentially acquired alternating VIS and NIR raw image frames form 3D-matrices for the Left and Right channel. The NIR image size is smaller than the VIS image size to increase framerate while maintaining an overview of the imaged area. Enlarged details illustrate the NIR transparency of the Bayer pattern RGB-filters which are applied to obtain a VIS color image. The time domain axis is expressed in image cycles. Along this image cycle axis, the control signals for NIR and VIS LEDs (synchronized with respectively NIR and VIS camera exposures) are visualized.  Data Aquisition Multispectral stereoscopic movies were recorded in several typical clinical settings for which the technique was considered as possibly useful. LED currents and diaphragm settings were chosen so that for each movie saturated pixels were avoided. A preview mode allowed aiming, adjustment of converging angle α and focusing of the cameras. After software triggering the stereo-camera streamed a sequence of 8-bit digitally encoded image cycles to PC-memory (using auto-incremental numbering). All images were automatically saved on a fast SATA harddisk-array. Camera and light source settings were automatically stored in a text file and located in the same directory. All patients and volunteers gave their informed consent for filming as well as for publishing the resulting image material. No diagnosis or therapy was based upon any of our results. Data Processing General Aspects ++ 3 Figure 3. Schematic diagram of image processing. The images captured within the visible range (VIS) and the images captured within the near infrared range (NIR) are combined, which reveals blood vessel patterns below the skin. The left and middle column focus on edge-enhancement and suppression of superficial artifacts, the right column serves to fill-in the blood vessel lumen. For raw VIS & NIR images as well as processed results see figures 6, 7, 8 and 9.  The processing method allowed discrimination between image information obtained from the tissue surface versus image information obtained from within the tissue. In order to achieve this, a distinction was made between shadows, reflections and absorption contrasts for both VIS and NIR. Suppressing Shadows on the Surface Since NIR and VIS beams were matched closely, shadows produced by irregular shapes at the tissue surface (e.g. skin structure, skin folds, nevi, hair, etc.) or by objects between the light sources and the tissue (fingers, needles, surgical tools, etc.) also matched well in both wavelength ranges. Our algorithm excluded such matching shadows from enhancement and left all useful aspects of shadows unaffected (e.g. depth clues). Useful Effects of Shadows and Lighting Geometry Due to the fact that the VIS and IR shadows matched very closely, the algorithm was able to selectively enhance contrast from below the surface while leaving shadows on the surface unaffected (thus not distorting these important depth clues). Due to the shallow angle of the lightbeams, the skin texture was pronounced. Due to the lighting from two sides, objects that were brought towards the tissue surface (e.g. needles, scalpels, probes, etc.) when lighted from both sides, could produce two separate (not too heavy) shadows. These shadows met and typically formed a “V” pattern when an object touched the surface in the middle of the field of view, thus providing extra information for depth perception. Enhancing Absorption Contrast of Blood Vessel Walls Edge detection by a Prewitt image filter was performed on each VIS and NIR image. Pixel positions containing edge information above a certain adjustable threshold in both spectral regions were classified as surface artefacts and excluded from enhancement. The boundary regions of absorption contrasts caused by structures below the tissue surface, produced edges that were mainly present in the NIR image. Enhancement was selectively performed only for pixel positions where the NIR image contained more edge information than the corresponding VIS image. These “valid” pixels identified the vessel boundaries. The positions of these valid pixels were stored in a 1st NIR mask. Discarding Reflections on the Surface nd Enhancing Absorption Contrast of Blood Vessel Lumen The “content” of blood vessels was separately enhanced by raising pixel values of the normalized NIR image to the power of N (with N user adjustable between 0.5 and 2.5) while discriminating NIR pixels below a freely adjustable noise threshold and excluding information from identified shadows. Multiplication with the 2nd NIR mask then produced a final enhancement mask for subsequential backprojection into the VIS image by pixel-to-pixel multiplication. Suppressing Contrasts Originating from Melanin Pigment 4 separate VIS composed 11 R VIS  G VIS  B VIS  NIR VIS  R VIS  NIR VIS  R VIS  NIR VIS  4 9 Figure 4. I R I G I B I NIR I VIS I R I VIS I NIR I VIS  Data Presentation The images could either be displayed on an auto-stereoscopic liquid crystal display (LCD) monitor or on a conventional CRT monitor equipped with shutter glasses. 5 Figure 5. Principle of applied autostereoscopic LCD-monitor (reprinted with permission from Sharp). In 2D mode, only one camera-channel is displayed (either from the L or R camera) and the parallax barrier is not actuated. Both eyes of an observer therefore receive the same image and a conventional flat image with full resolution is seen. In 3D mode, both camera channels are displayed (L&R) and the parallax barrier is actuated. The left eye and right eye of an observer now receive different images, and a stereoscopic (in-depth image) with halve resolution is seen.  The CRT-monitor (iiyama vision master 21) was used at a resolution setting of 1280 × 1024 in combination with wireless shutter glasses (e-Dimensional) and thus provided stereoscopic information without sacrificing resolution. The enhancement algorithm occurs on a pixel-to-pixel basis and does not affect resolution. Monocopic raw VIS preview (normal full color vision) Monoscopic raw NIR preview (greyscale) Off-line stereoscopic looped VIS view with and without enhanced blood vessel back-projection (with freely adjustable enhancement settings). Off-line stereoscopic looped raw NIR view or enhanced NIR view (with freely adjustable enhancement settings) Stereoscopic stills in all modes Monoscopic stills in all modes (freely switchable between Left and Right) All modes offered the possibility for pause and scrolling forward or reverse frame-by-frame. By means of virtual slider controls and virtual pushbuttons the user interface allowed freely adjustable settings for shadow suppression, pigment suppression, noise threshold and vessel lumen fill-in contrast. Results Results for Blood Withdrawal 6 6 6 Figure 6. Routine blood withdrawal. Image pairs showing unprocessed images for VIS (a) and NIR (b) as well as the result after application of the new image processing method (c). Note the forked shadow (which is not effected by the enhancement algorithm), the clearly visualized subcutaneous bleeding and the improved visibility of the needle tip.  Results for Dark Skin 7 7 7 Figure 7. Influence of skin pigmentation. A dark skin color (a) has no significance for the applied NIR wavelength of 920 nm. Blood vessels provide good contrasts (b) and the resulting enhanced image (c) offers an improved visualization of the vasculature.  Results for Vein Detection Through Iodide 8 8 8 Figure 8. Vein detection through iodide. Within the visible range, the superficial vasculature is only vaguely discernable (a). After filling the Petri-dish with a 3 mm thick layer of iodide solution, this fully blocks out the tissue view within the visual range (b), whereas a clear view of the vasculature remains possible at the applied NIR wavelength of 920 nm (fig 8c).  Results for Varicose Vein and Nevi Pigmentosum Inspection 9 3 9 Figure 9. The VIS image does not contain much information about the underlying vascular pattern (a). The NIR image, however, clearly shows what’s hiding beneath the surface (b). Note that, when building the enhanced image, the nevus which is present in the VIS image can freely either be suppressed as a surface contrast (c) or be kept visible (d).  Discussion 2 stills movies 8 17 18 19 et al. 27 volumes 21 28 beneath two 1 12 Since only carefully balanced white light is projected on the skin, there is no impairment of color perception. It is up to the user to freely switch between normal full color vision, color vision plus superimposed blood vessel backprojection and stereoscopic near infrared greyscale vision (with or without enhancement features). Due to the fact that the pertaining configuration aquires VIS and NIR images sequentially, motion artifacts can lead to a backprojection shift. 2 7 Conclusions Compared to inspection with the naked eye under normal lighting conditions, the tested stereoscopic blood vessel contrast enhancer offered improved visualization in all investigated settings, providing the best stereoscopic image quality when using the CRT monitor with shutter glasses and the best monoscopic image quality when using the LCD set to monoscopic mode. Our technique supports perception of depth, 3-dimensional motion and discrimination between tissue surface and underlying structures. It also has potential as an educational tool by offering the possibility to look and record trough the eyes of an experienced specialist. Further improvements on penetration depth, frame-rate and focus depth-of-field form targets for momentary ongoing further research.