Our research focuses on the development of Computer Vision and Machine Learning Systems. Amongst others we are working on the reconstruction of natural habitats, novel imaging techniques and algorithms to quantify the behaviour of animals. Some research projects are listed below.
Visual tracking of tiny objects in cluttered environments
Image-based tracking of animals in their natural habitats can provide rich behavioural data, but is very challenging due to complex and dynamic background and target appearances. We present an effective method to recover the positions of terrestrial animals in cluttered environments from video sequences filmed using a freely moving monocular camera. Our algorithm uses feature-based camera motion removal in combination with a generic motion model to formulate an energy function which is globally optimised using factor graphs. The algorithm is robust to occlusions, different appearances of the object and background and provides automatic initialisation.
Imaging and tracking of crawling insects in the lab
Imaging and analyzing the locomotion of semi-translucent animals such as Drosophila larvae or C. elegans has become an integral subject of biological research. However, acquiring contrast rich and high resolution measurements is still challenging. We have developed a new FTIR-based Imaging Method (FIM), which is used to detect the contact surface between the arena and the animals. FIM facilitates high-contrast and high-throughput behavioural analysis by eliminating many inaccuracies on a physical level. This system in combination with the associated tracking software FIMTrack is already used by many groups all over the world.
The introduction of 3D scanning has strongly influenced environmental sciences. If the resulting point clouds can be transformed into polygon meshes, a vast range of visualisation and analysis tools can be applied. But extracting accurate meshes from large point clouds gathered in natural environments is not trivial, requiring a suite of customisable processing steps. We have implemented Habitat3D, an open source software tool to generate photorealistic meshes from registered point clouds of natural outdoor scenes. Habitat3D offers a variety of different filtering, clustering, segmentation and meshing routines. All routines can be arbitrarily assembled to pre-defined pipelines operating on either subsets (i.e. clusters) or complete clouds.
Results are published here:
Risse, B., Mangan, M., Stürzl, W., & Webb, B. (2018). Software to convert terrestrial LiDAR scans of natural environments into photorealistic meshes. Environmental Modelling and Software, 99, 88–100.
FIM imaging to visualise and quantify internal organs
The importance of studying model organisms such as Drosophila melanogaster has significantly increased in recent biological research. Amongst others, Drosophila can be used to study heart development and heartbeat related diseases. We have established Frustrated Total Internal Reflection (FTIR) to improve the analysis of small animals like insects. This FTIR-based Imaging Method (FIM) results in an excellent foreground/background contrast and even internal organs and other structures are visible without any complicated imaging or labelling techniques. For example, the trachea and muscle organizations are detectable in FIM images. We demonstrate that FIM enables the precise quantification of locomotion features namely rolling behavior or muscle contractions by performing cluster analysis using histogram-based statistics. We also demonstrate that these imaging techniques enables automatic in vivo heartbeat detection of Drosophila melanogaster pupae based on morphological structures which are recorded without any dissection. Our approach is easy-to-use, has low computational costs, and enables high-throughput experiments.
We have developed techniques to segment and track of multiple cells in in vivo wounding. In particular, challenges like overlaps and clustered cell entities are addressed by introducing a statistical measure called hemocyte migration score.
Results are published in:
Lammel, U., Bechtold, M., Risse, B., Berh, D., Fleige, A., Bunse, I., et al. (2014). The Drosophila FHOD1-like formin Knittrig acts through Rok to promote stress fiber formation and directed macrophage migration during the cellular immune response. Development, 141(6), 1366–1380.
Sander, M., Squarr, A. J., Risse, B., Jiang, X., & Bogdan, S. (2013). Drosophila pupal macrophages - A versatile tool for combined ex vivo and in vivo imaging of actin dynamics at high resolution. European Journal of Cell Biology.
Visual 3D tracking of multiple objects
If two cameras are employed to estimate the trajectories of identical appearing objects, calculating stereo and temporal correspondences leads to an NP-hard assignment problem. Currently, there are two different types of approaches discussed in the literature: probabilistic approaches and global correspondence selection approaches. Both have advantages and limitations in terms of accuracy and complexity.
Results are published in:
Tao, J., Risse, B., & Jiang, X. (2014). Stereo and Motion Based 3D High Density Object Tracking. Image and Video Technology, 8333(Chapter 12)
Risse, B., Berh, D., Tao, J., Jiang, X., Klette, R., & Klämbt, C. (2013). Comparison of two 3D tracking paradigms for freely flying insects. EURASIP Journal on Image and Video Processing, 2013(1), 57.
Tao, J., Risse, B., Jiang, X., & Klette, R. (2012). 3D Trajectory Estimation of Simulated Fruit Flies. In Proc 27th IVCNZ, Dunedin