Research challenge and context
Whenever we browse the Web, we set off a chain of complex processes that allow images to be retrieved and displayed on our screens. Over the last decade, one aspect of image processing research in Electronics and Computer Science at Southampton has focused on enabling computers to keep up with the ever-growing demand for – and volume of – high-resolution images on the internet. Another stream of research has looked at ways to use machine learning and mathematical tools to analyse vast libraries of images.
Southampton’s image capture research has aimed to provide specialist imaging systems for museums and other cultural heritage institutions. While there have been huge advances in digital photography over recent years, traditional techniques still result in ‘flat’ 2D images. Southampton researchers aimed to develop a system to quickly capture the fine 3D detail of artefacts, creating a practical way for museums to take pictures of, and study, large collections.
Our solution
In collaboration with end-users and industry, the researchers developed two open-source software tools: libVIPS for image processing and OpenIMAJ which extracts information from multimedia data.
Most image processing systems load the entire image into the computer's memory and then transform it in a series of steps; each of these steps needs another complete copy of the image being worked on. libVIPS takes a different approach, chopping images into small tiles which are processed by separate cores, before being reassembled into their final form at the end. This enables it to run quickly with much smaller memory requirements. This software has evolved over decades of research projects and is now maintained by an open source community.
OpenIMAJ uses advanced methods to carry out specific tasks such as feature extraction, which uses machine learning to recognise particular shapes or patterns in digital images. The software also incorporates new approaches for the analysis of multimedia data such as videos.
The researchers worked on developing an image capture method known as reflectance transform imaging (RTI). This involves placing an object under a dome, in which it is photographed around 100 times with different light angles. The resulting data allows the viewer to digitally ‘re-light’ the object in order to investigate small surface details.
What was the impact?
These tools are contributing to more cost-effective and efficient business processes, helping organisations to innovate, and enabling researchers to gain new insights into ancient artefacts.
libVIPS is being used by around 1,000 companies to help them develop websites and systems that handle millions of images, using less cloud storage, speeding up processing times and saving more than 50 per cent of computing capacity. libVIPS has enabled Wikipedia to use high-resolution images for the first time, and Amazon Web Services uses it for efficient image handling in its content distribution networks. Other organisations that have adopted libVIPS include The National Gallery, US Food and Drug Administration, Booking.com, The New York Times and Capital One. It is also being used in medical settings to quickly process large diagnostic images.
Similarly, OpenIMAJ is being used by developers across the world, including in large organisations such as Yahoo and Comcast, to quickly create applications that extract information from multimedia data.
The researchers have worked with Ordnance Survey (OS) to develop new ways to exploit aerial imagery. Previous methods for extracting data from OS images were largely manual, making them costly and limited in scope. Southampton’s work on the use of machine learning to extract specific features from aerial images has enabled OS to prototype new data products, for example tools to classify building types and agricultural land use.
Imaging systems designed and built by the Southampton team are being used for museum, archive and university research in institutions worldwide to capture highly detailed images of historical artefacts. For example, researchers at the University of Oxford are using Southampton systems to study and decode fragile tablets inscribed with the world’s oldest undeciphered script. The British Museum uses a similar system for studies where conventional photography is insufficient to capture surface details, and at The Louvre the technology has been used to capture an entire collection of ancient seals for the first time.
Photo credit: Banner image courtesy of Biblioteque Nationale de France.
Related publications
Detection of Social Events in Streams of Social Multimedia
Practical scalable image analysis and indexing using Hadoop
Joint deep learning for land cover and land use classification
Archaeological applications of polynomial texture mapping: analysis, conservation and representation
A structured light approach to imaging ancient Near Eastern cylinder seals
Find out more
Talk to our research team and find out more about this work. Professor Kirk Martinez and Dr Jonathan Hare led the research on this project.
Modelling for safety-critical systems
Our researchers have developed a set of open-source formal modelling and verification tools to support robust software design in industries such as aerospace, rail and defence.
Read more
Find out more
Find out more Talk to our research team and find out more about this work. Professor Michael Butler led the research on this project.
Staff profile
Our solution
Our researchers have developed a set of open-source formal modelling and verification tools to support robust software design in industries such as aerospace, rail and defence.
Research challenge and context
When developing or modifying software, engineers can use formal methods – a range of mathematical techniques for modelling the software and testing how it will work – to detect and prevent errors at every stage of the process. These methods of validation (checking that the software specification meets the user requirements) and verification (ensuring that the software works correctly, satisfying the specification) are crucial in contexts such as transport or aerospace, where failure could have catastrophic results.
Existing formal methods provide rich mathematical languages for modelling and reasoning about correct system behaviour. However, it is difficult to scale them for complex systems in which a software-based controller manages many features or controls a large proportion of a system’s functions – for example, a control system that supervises a large volume of traffic on a rail network.
Another issue with formal methods approaches is slow uptake in industry, where the potential improvements in reliability, and savings in testing and reworking costs, are not always recognised and the techniques are perceived as being inaccessible to non-academic engineers.
Our solution
Our researchers have developed a set of open-source formal modelling and verification tools to support robust software design in industries such as aerospace, rail and defence.
Working in collaboration with several European universities, Southampton researchers developed Rodin, a design tool capable of the verification and validation (V&V) of complex software systems.
Rodin’s core features are open source, allowing academic research groups to develop and experiment with new V&V features. However, to support the adoption of the technology in industry, the researchers developed ‘industrial strength’ plug-ins. One of these, UML-B, is an interface that provides a graphical representation of the models, making the modelling language easier to understand and more user friendly.
Another plug-in, ProB, enables engineers to add a powerful form of mathematical reasoning, called model checking, to the V&V process. Formal methods use two types of computer-based reasoning, model checking and automated deduction. Automated deduction is a logic-based approach that can scale to models with large dimensions but is not fully automatic and requires some human intervention to guide the reasoning. Model checking provides a completely automatic approach to finding design errors by restricting the dimensions of a model. Typically, formal methods tools only support one type of reasoning. ProB extends the automated deduction provided by Rodin with model checking capability, enabling engineers to use both approaches.
What was the impact?
Engineers in companies worldwide, including AWE in the UK and Thales in Austria, are successfully using Rodin in the early-stage design and validation of computer-based control systems. The toolset is helping them uncover potential design problems earlier in the development of new systems, contributing to safer system design and avoiding expensive reworkings.
Through University consultancy ECS Partners, the research team has delivered Rodin training and customisation services to a range of companies, including AWE, Sandia Labs, Thales, Deutsche Bahn and Hitachi. In addition, the Rodin toolset is being used for academic research and undergraduate and postgraduate teaching in universities worldwide.
The research team has nurtured an international community of Rodin users and developers. As an open-source toolset, free to use for research and commercial purposes, the 2018 version of Rodin has been downloaded over 10,000 times, and Southampton hosts a well-used online resource providing user and developer support. Since 2009, the University has organised Rodin workshops that attract users and developers from around the globe.
Find out more
Talk to our research team and find out more about this work. Professor Michael Butler led the research on this project.