Development of heart motion reconstruction framework based on the 4D echocardiographic data

Abnormalities in heart motion can eventually lead to life threatening cardiac injuries there-fore measurements of dynamic heart functions are of great clinical importance. The images of moving spatial heart structures can be eﬃciently acquired using 4D echocardiography. Un-fortunately, because of the low quality such images do not allow for precise measurements. To overcome this problem images need to be further processed and moving structures have to be extracted. In this work we present a method for estimating heart motion from the 3D echocardiographic image sequence. On the basis of this method we have developed an appli-cation that enables qualitative and quantitative (i.e. volume changes, stroke volume, ejection fraction and cardiac output parameters) description of the heart wall motion. We provide a set of tools for denoising images using the anisotropic diﬀusion algorithm extended to the fourth dimension and the time averaging method based on non-linear registration eﬃciently parameterized using the B-spline based Free Form Deformation. We have also developed a non-linear deformable segmentation algorithm for extraction of the inner ventricular surface. The motion of the left ventricle is reconstructed in our approach by recovering deformations of the matter during the cardiac cycle. All the obtained results using our framework can be eﬃciently presented in 3D using a set of newly developed heart motion visualization tools.


Introduction
Ultrasonographic examination of the heart (echocardiography), together with techniques based on electrocardiography, is one of the most frequently used methods of the heart examination. Using this modality, vital information about morphology and hemodynamics of the heart could be collected by simple, bedside assessment. Echocardiography, even real time 3D, is a relatively inexpensive (when compared to CT or MRI) data acquisition technique. In clinical practice the analysis of the data mainly relies on the visual inspection of the acquired views and on the physician's experience. Such methods lead to qualitative and subjective assessment without taking into account individual quantitative information included in images. Another problem of the echocardiographic analysis are artifacts from the thorax (e.g. emphysema), which in 5-10% of patients makes echocardiography not useful. To reveal all these the vital information and decrease information noise, automated computer-based analysis is highly desirable.
Recently, methods were proposed for the reconstruction of heart motion from the 4D ultrasound images. For the left ventricle segmentation surface based methods (using shape and motion constraints) have been proposed to deal with speckle noise in the echocardiograms [1]. Biomedical models have been investigated for the modeling of cardiac cycle [2]. In this work we present a set of the heart wall visualization methods. These methods make an extensive use of the left ventricle motion description method [3] briefly presented in the next section.

Methodology
The heart reconstruction method implemented in our framework consists of five main stages (see Fig. 1). In the first step images are filtered using 4D anisotropic diffusion [4]. Next non-rigid registration of the 3D time sequence is performed to obtain the description of deformation field. Non-rigidly registered images are used to compute an average 3D dataset. The next phase consists of the shape and texture based segmentation followed by a triangulation step resulting in the 3D surface model. In the final step deformation operator is applied to the surface model in order to recover the time motion of the left ventricle.
In the first stage (see Fig. 1a) our goal is to reduce noise from the sequence of 3D ultrasound data. We have decided to use the anisotropic diffusion scheme   and take into account temporal consistency of the acquired data. The diffusion algorithm has been extended to the fourth dimensional block of data with the time taken as the fourth dimension. As it was presented in [3] such filtering drastically reduces the speckle noise and enhances the structure boundaries.
Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 06/04/2022 11:33:06 The speckle noise may lead to partial disappearing of the image boundaries. The time diffusion may help to recover some of the missing boundary parts.
In the second stage (see Fig. 1b) we want to describe the motion of the whole matter within the 3D image sequence. Such an approach gives possibility to model the motion taking into consideration individual patient specific anatomical features. In order to achieve realistic motion we have to extract heart dynamics by studying 3D movement of a corresponding anatomy between the reference frame (at time T 0 ) and the following frames (T 1 − T 8 ). We recover the transformation that aligns the reference frame with all other frames using intensity based 3D volume registration. In our work we have used FFD [5] (free-form-deformation) based method which is a well known powerful tool for modeling 3D deformable objects. We have selected MSD (mean square difference) as the similarity function: where I RI represents the reference image intensities, I F I represents the corresponding transformed intensities of the floating image and N is the total number of overlapping voxels. In order to deal with large displacements in the registration process, we use a classical incremental multi-resolution procedure.
After obtaining the 3D frames of the deformation field we are able to describe motion of the whole matter in the volume object. The third phase of our method involves noise reduction by temporal averaging (see Fig. 1c). The deformation field frames are used to generate new datasets elastically aligned with the reference frame T . After this step an average dataset from the reference frame and all the deformed datasets is created. Such an approach enables noise smoothing together with preservation of image structures boundaries.
The fourth stage applies the segmentation procedure to recover the shape of the left ventricle (see Fig. 1d). In our work we have used an iterative deformable boundary approach for the segmentation of the ventricular inner surface. The selected method uses energy function consisting of texture based and shape based terms. It is a 3D extension of an algorithm proposed in [6]. Texture based energy term is calculated on the basis of texture intensity energy map which represents the probabilities of the intensity values i being consistent with the current segmentation model (updated in every iteration). This term has been formulated as the Shannon's entropy [7]. The shape energy term takes into account gradient information (revealed using the Canny-Deriche's 3D boundary detection filter [8]) available in the source image. The main idea of this term is to deform (shrink or expand) segmentation model towards image Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 06/04/2022 11:33:06 U M C S boundaries. In this segmentation algorithm, starting with an initial estimate, a deformable model evolves under the influence of the defined energy to converge to the desired boundary of an image structure object. The model deformations are efficiently parameterized using the B-spline based free form deformation. The flow diagram of the segmentation algorithm is presented in detail in Fig. 2.

Fig. 2. Deformable segmentation algorithm diagram
The last stage of the algorithm involves 3D surface mesh generation and application of several newly developed visualization methods. Using the recovered ventricular surface we are able to reconstruct the cardiac motion by applying the deformation field operator. Using such approach we deal with a single object deforming in time so we are able to interpolate between the frames to obtain smooth motion. The motion of the heart can be also characterized in terms of its local variations. It is possible to calculate displacement vectors: total displacement (relative to the reference frame T 0 ) and displacement between consequent time frames which can be seen as an instantaneous velocity. We can also visualize the motion occurring on the surface (twisting) by decomposing the instantaneous velocity vectors into tangential and normal components. All Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 06/04/2022 11:33:06 U M C S of these local variations can be visualized in different ways. In our work we used two kinds of techniques -color and vector based. We colorized moving surface according to the length values of displacement vectors. It is very useful to deal with small surface deformations. When the motion is significant it is better to visualize vector values using arrows representing length and spatial orientation of moving matter. We call the last method of motion visualization used in our system line-paths method. In this method we select a small set of surface points and visualize the path of their motion during the cardiac cycle using colorized polygons. Colors of the line segments represent consequent time frames. Such method allows to estimate the viability of the heart using a single image. In addition to the line-paths method we may generate so called activity surface. With this method we can visualize total path length values (in single cardiac cycle) for every surface point. The method applied to the single static image allows us to estimate spatial extents of pathological regions.

Framework implementation
Our framework has been implemented in the C++ language using Trolltech QT [9] and Kitware VTK [10] libraries for visualization purposes. It has been compiled and tested under Windows and Linux operating systems. The presented motion reconstruction method is semiautomatic. With opening the 4D source dataset the user has an ability to select parameters for anisotropic diffusion filtering (number of iterations, diffusion function, threshold parameters) and non-linear registration (number of multi-resolution levels, desired quality of matching (from the minimum FFD grid size 8 × 8 × 8 up to 32 × 32 × 32).
At this point the procedure of filtering, matching and average dataset generation steps is performed automatically. The most time consuming part is 3D registration. It takes for the pair of 3D images from 3 to 11 minutes (depending on the desired quality) on typical 2.0GHz PC, so the whole matching process may take around an hour. Afterwards, the user has to select parameters (initial segmentation shape, weights for energy functional) using interactive tools and apply the segmentation procedure. This step takes from 1 to 5 minutes of computational time. The last step involves only selection of visualization method and color lookup table. The user can interactively manipulate with the object and save the resulting 3D animation on disk. Beside the visualization techniques it is also possible to calculate some statistical parameters like stroke volume, ejection fraction and cardiac output [11] using volume differences in the end-diastolic and end-systolic phases (minimum and maximum volumes in the cardiac cycle) and the heart rate. An exemplary illustration of the graphical user interface of the developed framework is visible in Fig. 3.
Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 06/04/2022 11:33:06 U M C S Fig. 3. Example of the heart motion reconstruction framework: deformable segmentation module (interactive initial ellipsoid selection on the 2D perpendicular projections, selection of energy functional weights), volume non-linear registration module (selection of matching quality parameters), 3D interactive module (spatial presentation of 2D perpendicular projections superimposed with semi transparent surface of the left ventricle and motion vectors)

Conclusions and future work
In this work we have presented a method for the dynamic heart motion reconstruction from the 4D echocardiographic images. We have also presented implementation details and developed software framework. In the near future we want to parallelize registration procedures to obtain more acceptable running times of the whole procedure. We are also greatly interested in development of fully automatic segmentation procedures of the inner wall of the left ventricle. We plan to extend the segmentation algorithm to be able to detect the outer wall of the heart muscle. Then it will be possible to estimate global parameters like wall thickening or ventricular mass [11]. In the physician's opinion, the proposed methods and developed tools enable to detect pathological regions of the beating heart with high precision and in the near future may be useful in their daily clinical practice.