Following the growth of diabetes, the number of patients suffering from diabetic retinopathy is also increasing. The diagnosis and monitoring of retinopathy is based on fundus photographs and their analysis, which uses a lot of public health resources.
“The number of fundus photographs has increased by 45% over the past five years. Already now, their screening is a huge project, and the work will only increase”, explains Professor of Ophthalmology at the University of Oulu Nina Hautala. “It takes a few minutes for one person to analyse one image, but, for instance, in the area of Oulu University Hospital (OYS), over 200 photographs are taken for screening on a weekly basis.”
Automatising the analysis of retinopathy photographs would make the screening more effective, save time and labour, and speed up access to treatment. A collaborative project of the University of Oulu and Ilmenau University of Technology in Germany has taken on the challenge, and is currently developing an application based on machine learning, which will detect changes in the fundus.
Wide range of changes in the fundus challenges machine learning
In practice, the application looks for the same things in the fundus photographs as an ophthalmologist: visible signs of disease. The application has to be taught how to recognise such signs by using photographic material; one of the reasons for the German researchers to contact the University of Oulu was the screening system of Finnish diabetes patients, which has collected fundus photographs systematically throughout the patients’ medical history.
“In Germany, no such system exists. The project will use data consisting of several tens of thousands of images”, Hautala explains. “We started off by explaining what is unusual in the photographs, and which changes are signs of disease and which ones are harmless. On the basis of this information, technical researchers created algorithms, which the application can use to detect diabetic changes.”
The challenge is the broad spectrum of fundus changes in the eye. The application must learn to recognize, for example, microaneurysms, intraretinal bleeding, lipid deposits, retinal microinfarcts and neovascularisation, including all grades of severity and other possible variations. “Thousands if not tens of thousands of repetitions will be required”, Hautala says.
The underlying technologies are not new as such. “Image analysis uses conventional imaging methods, in other words, signal processing and shape recognition, as well as convolutional [deep learning] neural network, which has been created using well-known technologies,” says Researcher Katri Kukkola from the Optoelectronics and Measurement Techniques unit at the University of Oulu.
The abundance of variations in the fundus will also complicate the collection and pre-treatment of data used as teaching material. “It will take a lot of time and resources. The same thing may look different in different devices or if studied by different people”.
It is also problematic to simulate human activity in the first place. “The application detects same signals as doctors do, but as a result of their experience, doctors are likely to take into account more things unconsciously than what they verbalise”, Kukkola states.
Automation can help screening within a few years
The project has not led to a finished application yet, but Nina Hautala believes that automation could be utilised within a few years in the clinical screening process.
“Now we are faced with a big job: the software will run through tens of thousands of images, and we will make sure that it is indisputable and secure. Sensitivity [probability of detecting disease-related changes] and specificity [accuracy of detecting changes] are approaching a sufficient level, over 90%.” According to Hautala, the corresponding percentages of the human eye have not been studied, “but it is known that early changes are better detected in an image than when a doctor examines the fundus with a microscope.”
However, the application would still only be an aid, the current precision of which is sufficient to identify a healthy fundus and mild background retinopathy, which does not require treatment. Automatic screening of these groups would save resources for the treatment and monitoring of actual retinopathy. In addition, the application could assist in monitoring, as it is capable of identifying disease progression.
A doctor will still be needed for the diagnosis and the assessment of treatment, Hautala believes. “There are so many individual factors affecting the patient’s condition – for example, other diseases or another ocular disease – which are difficult to teach to a machine.”
The developed methods are to be licensed for companies
Another ongoing project, called Crystal, aims to extend the automatisation to other eye diseases as well. “The aim is to create models for an application that can be used to screen retinopathy but also open-angle glaucoma, and age-related macular degeneration”, Katri Kukkola, who is involved in the project, says.
Since the latter two are diagnosed by doctors through various methods, not just photographic analysis, the application would complete the analysis as a “data fusion”, drawing attention to the patient's background, symptoms, visual acuity, and the visual field in addition to the fundus photograph.
The methods include analysis of larger image areas and direct measurements. “The Crystal project has developed new ways of producing measurement data about the optic nerve. We do not know any commercial applications which would provide similar information.”
Machine learning is a hot topic in eye diseases. The annual publication rate has risen in the 2010s from a few to thousands.
Hautala says that, in terms of diabetic retinopathy, there is only one finished application providing competition. The joint project of Ilmenau and Oulu has commercial objectives (as well as the Crystal-project): the plan is to license the developed methods for companies, Katri Kukkola says.
“A talented company could apply at least some of the solutions within a year, but the testing and evaluation required for medical will take more time.”
Photo: Professor of Ophthalmology Nina Hautala examines the fundus of the eye with a biomicroscope. "Early changes are usually better detected in an image than when a doctor examines the fundus with a microscope” she says. The reason for this are the filters used in the camera.
Text: Jarno Mällinen
Photos: University of Oulu / Mikko Törmänen
Main photo: Screening for diabetic retinopathy includes the detection of microaneurysms, intraretinal bleeding, lipid deposits, retinal microinfarcts and neovascularisation.
Last updated: 11.1.2019