Image from the 3D-POP dataset.
3D Tracking from the lab to the wild.
Scroll ↓
3D Tracking with Markers
Since 2017, I have been working with problems on 2D-3D tracking of animals, primarily birds. We designed SMART-BARN, a large-scale facility for tracking animals in indoor environments. The system uses motion capture, video cameras, and audio microphones for 3D tracking insects, birds, and mammals. We demonstrate 3 case studies and 3 example studies to showcase the wide range of experiments the facility supports. The research is actively supported by the Max Planck Institute of Animal Behavior and the Center of the Advanced Studies of Collective Behavior (CASCB) at the University of Konstanz. You can read our paper in Science Advances outreach articles from New Scientist and Uni Konstanz. There are multiple groups working with these facilities and taking advantage of our insights.
Markerless Tracking in Wild Using Indoor Dataset
My methods are now used by multiple researchers at Uni Konstanz and CASCB to extend the work on 3D postures. We have recently demonstrated that data created with birds in indoor environments can be suitable for tracking birds in the wild with minimal annotations. I am now working with my colleagues to extend the marker-less tracking approach from indoor to outdoor environments. To this end, I am also working towards exploring novel problems of 3D tracking with other taxa (more below).
Automated Datasets for Markerless Tracking
Marker-based 3D tracking facilities like SMART-BARN have been used to create large-scale datasets for markerless tracking in Humans. Taking this as inspiration, I developed a novel technique to track the 3D posture (head and body orientation) of birds using markers. Furthermore, I modified this approach to create annotations necessary for markerless tracking. The details can be followed in our publication. We have created 3D-POP, one of the first and largest datasets consisting of the 3D posture of multiple interacting birds. We also show that markerless tracking works with pigeons without markers. Read more here.
Previous projects
I have publications on topics of 3D Computer vision and AR/VR for animals, healthcare, and industrial augmented reality. Please check the publication section for more details.
What’s new?
I want to explore more applications of 2D-3D animal tracking for behavior, conservation, or biodiversity monitoring. If you are interested in a discussion about 2D-3D tracking problems, I am happy to chat and consult. Do drop me an email at hnaik@ab.mpg.de.
I have started working with Alex Chan, Dr. Fumihiro Kano, and Dr. Micheal Griesser on the Siberian Jay project. We are developing a toolkit for tracking birds in 3D with multiple cameras based on the 3D-POP design. It’s the first attempt to bring 3D tracking in the wild. The project is funded by CASCB (The Centre of the Advanced Studies of Collective Behavior).
I am advising on designing a 3D tracking method for primates in a fresh collaboration with Nora Slania, Dr. Isabelle Laumer, and Richard Young from the group of Dr. Caroline Schuppli at MPI-AB. (Pictures coming soon).