Algorithms for Medical Image Analysis at a Massive Scale

Computational Tools for Medical Image Analysis

Repositories of medical images (e.g. ultrasound, X-rays, magnetic resonance) are growing rapidly, creating unprecedented opportunities to study anatomical variations and diseases (e.g. Alzheimer’s or chronic obstructive pulmonary disease). Applications such as computer-assisted diagnosis may one day be critical in addressing the ever-increasing demands on our health system. To do this, however, we need computational tools capable of analyzing image data at a massive scale.

For Matthew Toews, Professor at the École de technologie supérieure (ÉTS) since the spring of 2015, his main area of research is the development of such tools. His algorithms seek to detect patterns of interest throughout the images, to learn from many examples which patterns are related to categories of patient, diseases, etc. This information can then be used to aid in the diagnosis and prognosis of new patients.

Professor Toews’s career path

Professor Toews became interested in image processing during a one-year internship at Toshiba Inc., in Japan, as part of a bachelor’s degree in electrical and computer engineering at the University of British Columbia. During this time he met his future wife, a woman from Québec, who encouraged him to pursue graduate studies at McGill University in Montréal, where he completed his Masters and PhD degrees in computer vision and medical imaging analysis. He continued on at the Harvard Medical School in Boston, conducting postdoctoral research in the field of 3D medical imaging analysis. He then chose to pursue his career as a professor at the École de technologie supérieure to continue his research in this field.

From Classifying Faces to Analyzing the Brain

The research topic of his PhD was to build a system to detect and classify human faces according to traits such as gender or age, from images taken from arbitrary points of view (full-face, profile, etc.). His solution was to develop a machine learning algorithm, based on detecting and classifying points of interest in face images via a process closely analogous to the human visual system. In fact, the human brain unconsciously records features of the human face and associates them with traits such as gender or age to help us better recognize the people around us. In order for the algorithm to “learn” these features, it must first be fed with thousands of examples of face images that that serve as references. The algorithm then associates features detected in a new face image with those found in reference images, in order to predict the traits the person in the photo.

At the Harvard Medical School, Matthew Toews continued investigating machine learning algorithms for analyzing 3D medical images, such as brain MRIs or CT scan images of the body. His approach was able to automatically identify brain patterns linked to neurological diseases, e.g. atrophy of the hippocampus associated with Alzheimer’s disease, and to predict the disease status of new brain scan with high accuracy. This type of system could be useful in a number of applications, including the diagnosis of a disease before the onset of the related symptoms, evaluation of disease progression, and prediction of disease severity.

Matthew Toews also developed machine learning algorithms that can identify or segment the main visceral organs from scans of the human body. These algorithms use a set of reference images in which organs have been manually delineated by expert radiologists. These delineations are then transferred to a new patient image by matching the new image to the reference image set, thereby transferring expert knowledge of human anatomy to new patient data.

Future Areas of Research

Professor Toews plans to focus his research on developing and improving similar machine learning techniques for neurological diseases exhibiting distinctive patterns, such as Parkinson’s disease or multiple sclerosis. His research applies equally non-brain data, and he is also investigating chronic obstructive pulmonary disease in lung scans and prostate cancer in MRI. His research activities are conducted in collaboration with researchers at the CR-CHUM, in order to develop systems that can be applied in a practical hospital setting.

The algorithms that he is developing are based on a relatively simple yet powerful image matching methodology. Images are encoded in terms of sets of distinctive patterns or features, and then used to rapidly identify similar images in large databases (big data). Notable aspects of these “feature-based” algorithms is that they can be used with arbitrarily large data sets, and they become increasingly accurate as the number of reference images increases.

The engineering theories and techniques used in professor Toews’ research apply in contexts other than medical imaging, and he is also involved in developing algorithms related to robotic vision. One of his students recently developed a robot vision system capable of focusing on objects in the field of view. This robot has two cameras that relay video streams to a computer via cables, much as the optic nerve connects the human eye to the brain. Algorithms running on the computer identify objects of interest in the videos, and send commands motors that move the cameras in order to focus on and to track the objects, in a manner similar to eye muscles in humans.