Note: Work described within is a summary of a paper that was published in Proceedings IEEE Visualization '97; a PDF version is available for downloading.

Abstract

Volume visualization of medical images must address two important issues. First, it is difficult to segment medical scans into individual materials based only on intensity values. This can result in volumes that contain large amounts of unimportant or unnecessary material. Second, although greyscale images are the normal method for displaying medical volumes, these types of images are not necessarily appropriate for highlighting regions of interest within the volume. Studies of the human visual system have shown that individual intensity values are difficult to detect in a greyscale image. In these situations colour is a more effective visual feature, since the low-level visual system can rapidly and accurately detect the presence or absence of a particular target colour in a multi-coloured image.

We addressed both problems during the visualization of CT scans of abdominal aortic aneurysms. We have developed a classification method that empirically segments regions of interest in each of the 2D slices. We use a perceptual colour selection technique to identify each region of interest in both the 2D slices and the 3D reconstructed volumes. The result is a colourized volume that the radiologists are using to rapidly and accurately identify the locations and spatial interactions of different materials from their scans. Our technique is being used in an experimental post-operative environment to help to evaluate the results of surgery designed to prevent the rupture of the aneurysm. In the future, we hope to use the technique during the planning of placement of support grafts prior to the actual operation.

Introduction

One promising application of our techniques is improving certain types of volume visualization. Volume visualization displays data representing volumetric objects as 3D models on a computer screen. This normally involves both the reconstruction and the display of the 3D model. Volume visualization allows users to view their datasets in an appropriate three-dimensional context. Various techniques, such as the use of transparency and cutting planes, can be applied to simultaneously display both the surface and the interior of the volume.

A well-known and important area of volume visualization is the display of 3D medical volumes. Medical techniques like computed tomography (CT) and magnetic resonance imaging (MRI) use equipment which scans the body in a user-chosen direction. The result is a set of 2D intensity maps representing cross-sectional slices through the body. Different intensities in a slice correspond to different materials (e.g., bone, muscle, and fat) detected during the scan. The intensity values returned by the scanner are often adjusted to try to highlight areas of interest within the volume. Stacking the slices together and resampling the data allows for the reconstruction of a volume representation of the scanned region (Figure 1). Finally, the results are displayed onscreen, allowing radiologists to visually explore and analyze the output of the medical scanner.

Although 3D reconstructed volumes are often a dramatic improvement over the original 2D slices, the techniques used to build the volumes must address a number of important problems. Different materials detected during the scan will have overlapping greyscale ranges. This makes it difficult to isolate a particular type of material based only on the intensity values in the 2D slices. The 3D reconstructed volumes are often displayed in greyscale, using pixel intensities which are directly proportional to tissue density. In many cases this is appropriate, since difference in intensity most effectively captures the high spatial-frequency differences which occur in these types of images. However, greyscale images may not be as useful when a user is searching for the location of a particular material within the volume. Partial occlusion of the interior of the volume, the inability of the low-level visual system to rapidly and accurately find specific intensity values, and the possibility of overlapping greyscale ranges can combine to result in a time consuming search through the image. In the worst case, the material in question may be accidentally overlooked, reducing the value of the time and effort spent to build and display the volume.

Our work addresses both problems in the context of a specific set of medical images: CT scans of abdominal aortic aneurisms. First, we have developed a new classification method to empirically determine the correct boundaries for regions of interest in our scans; material which does not correspond to one of these regions is eliminated from each of the 2D slices. Second, we use a colourization technique to distinguish each of our regions of interest in both the 2D slices and the 3D reconstructed volume. A perceptual colour selection algorithm is used to guarantee that the colours we choose are all equally differentiable from one another. The result is a colourized volume which the radiologists can use to rapidly and accurate identify the exact locations of individual materials of interest. Our techniques have already been used in a post-operative environment to assess the effectiveness of grafts placed within aortic aneurisms. In the future, we hope to use our visualizations to help to plan placement of the grafts prior to an operation.

Abdominal Aortic Aneurisms

An abdominal aortic aneurism (AAA) is a focal dilation of the abdominal aorta (which has a normal diameter 2 cm) usually below the level of the renal arteries (Figure 2). Patients that develop AAAs likely have a basic genetic predisposition as this disease unequivocally runs in families. Other contributing factors probably include smoking and high blood pressure. By the age of 80, over 5% of white males will develop an AAA. AAAs are among the top ten causes of death in white males over 55, due to their propensity to silently enlarge and subsequently rupture resulting in shock and ultimate death in more than 80% of cases. For unknown reasons the prevalence in elderly males is five times that of females.

Traditional repair of an AAA entails a major operation with an incision into the aneurism, evacuation of the clot contained within, placement of a synthetic graft, and wrapping of the graft with the remnants of the wall of the AAA. Recently a new treatment option, endovascular stenting, has been proposed and is currently undergoing clinical trials. This procedure does not require general anesthesia and can be done less invasively by simply placing a self-expanding stent (Figure 3) via a catheter into the AAA to stabilize it. Less fit patients are able to withstand the procedure, hospital stay is cut to 1 to 2 days, and post-operative recovery is shortened considerably.

Several key pieces of information are necessary prior to undertaking any repair (either operative or endovascular), including: accurate measurement of AAA diameter and length, determination of distance between the renal arteries and the proximal end of the AAA, and measurement of the distance between the distal end of the AAA and the aortic bifurcation (Figure 2). We hypothesize that 3D reconstructions which can be rotated and viewed from any angle will be the most accurate representation of the relationship between the neck of the aneurism and the renal arteries and thus will provide the best measurements.

Segmentation, Classification, and Colourization

After segmentation, we use colour to highlight each of the three objects of interest (the arteries, the aneurism, and the tynes) in our reconstructed 3D volumes. Normally, greyscale images are used to visualize 3D medical volumes, because the high spatial-frequency components in a typical medical volume are most easily detected by the visual system through differences in luminance.

The radiologists are interested in rapidly identifying the general location of each object of interest, in particular they want to see where the tynes lie within the arteries and the aneurism. We felt representation using colour would be most effective for this type of visual region detection. Studies of the human visual system have shown that only the two end values of a greyscale range can be rapidly and accurately detected in a greyscale image; values from the interior of the range are much more difficult to identify. Previous work in our laboratory found that up to seven different colours can be displayed simultaneously, while still allowing a viewer to rapidly and accurately determine whether any one of the colours is present or absent in an image. Finally, our segmentation procedure allows us to identify uniquely the three objects of interest. Material which does not correspond to one of these objects is removed from the CT slices before they are visualized.

Figure 4 shows a greyscale reconstruction of the an abdominal aortic aneurism. Our masking technique has been used to remove material in the volume which does not correspond to the aneurism, the artery walls, or the tynes. Whereas images with no masking are essentially unusable, the representation in Figure 4 provides a variety of information about the scanned region. However, since the voxel intensities are directly proportional to the original pixel intensities in the 2D CT slices, certain areas in the volume suffer from a lack of definition. The tyne positions are only visible outside of the aneurism. When the aneurism and the arteries overlap, the locations of the tynes are hidden. Decreasing the opacity of the aneurism will eventually show the tynes, but at that point the aneurism itself becomes so transparent that its boundaries and depth can no longer be identified.

We use our colour selection algorithm to choose three colours to represent the three objects in our reconstructed volumes. The colours are chosen to maximize perceived difference to one another using a multi-criteria selection technique. This is in contrast to the traditional methods of colour selection for medical volumes, which are largely based on aesthetics. In our case, The radiologists ask us to avoid using greens during visualization, since green and green-yellow are conceptually identified as bile in these types of medical images. We used our selection technique to choose colours from the non-green regions of our monitor's colour gamut. The result was three colours: a purple-blue (monitor RGB=142, 141, 163) which is used to represent the aneurism, a yellow (monitor RGB=194, 149, 8) which is used to represent the blood vessels, and a red (monitor RGB=255, 0, 6) which is used to represent the tynes.

The following movie shows 25 slices taken from a CT scan; each slice has been partially segmented and classified, to highlight the regions of interest. The yellow, circular region at the center of the slice represents the aorta and iliac arteries. The red around the edge of each artery represents the tynes holding a stent graft in place. The large purple region surrounding the artery represents the aneurism. The slices are taken moving down the body through the abdominal region (Figure 2). The final slices show the aorta dividing into two smaller, iliac arteries.

The final four figures show colourized version of the reconstructed AAA volume. In Figure 6a, a colourmap chosen by radiologists was applied to the resulting volume. The colourmap was obtained in a trial and error fashion to try to best represent data from individual datasets; perceptual issues were not considered when the colourmap was designed. Colouring the volume provides a dramatic improvement over the greyscale image; it does a good job of highlighting the artery walls and the tyne locations (represented by yellow and red, respectively). However, the aneurism itself (shown as a red cloud around the artery) is not as well defined. In particular, when the aneurism overlaps with the artery, it disappears from view.

We used our colour selection technique to choose three new colours. The radiologists asked us to avoid choosing green or green-yellow colours, since these are associated with bile in the context of coloured medical volumes. In spite of this, we were able to pick three colours which had a very strong perceived difference from one another. These colours were used to produce the volume in Figure 6b. The location of the tynes within the artery walls is as clear as in Figure 6a. Moreover, the entire 3D structure of the aneurism can be clearly identified.

Figure 6c uses Volren's interactive cutting planes to slice diagonally through the volume. The slice allows us to measure the cross-sectional area of the aneurism and the two iliac arteries. Figure 6d uses a viewing direction which is orthogonal (or "straight-on") to one of the iliac arteries. This viewing direction shows the zigzag pattern the tynes make when they are embedded within the artery. Our masking technique is capable of extracting this level of detail in each 2D slice, even in these smaller blood vessels.

(a)		(b)
(c)		(d)

The radiologists have now started to use our results to view their CT scans. The images in Figure 6 represent a post-operative scan of a patient who underwent the endovascular stenting procedure. Our visualizations allow the radiologists to identify important details in the volume. For example, when the radiologists looked at the volume in Figure 6b, they immediately noticed an area in the artery which was not supported by the grafts (the two large yellow areas inside the aneurism in Figure 6b which have no red, and hence no tynes supporting them). This occurred because one of the grafts in the iliac artery was not pushed up far enough to mesh with its upstream partner. Although this poses little risk to the patient, it is very useful for the radiologists to be able to examine the results of their surgical procedures in this manner. In the future, we hope to use our volume rendering technique to allow the radiologists to help in planning the surgical procedure.