Basic system overview

A view of a large (2m diameter) flight arena, called Mamarama. This is one of the arenas for which the Flydra system was developed. Click here to see photos of this arena.

Flydra consists of a network of high-speed digital cameras and computers. Each camera is connected to a computer which extracts potential fly information in 2D image coordinates and sends this information to a central computer. The central computer uses this information, together with knowledge of the camera projection geometry, to determine the location of a flying animal, such as a fly or bird.

Mathematically, Flydra is based on two well-known algorithms. The first, an Extended Kalman Filter, is used to estimate the 3D position and velocity of each animal. Fly observations (2D camera image coordinates) are used to update these estimates using a model of the optical projection of each camera. The second algorithm, known as the Nearest Neighbor Standard Filter, performs data association, linking each tracked animal with incoming 2D data.

The following video is from an 11 camera, 60 frames per second system tracking fruit flies (Drosophila melanogaster) in a two meter diameter arena, Mamarama (see image on right). Eight of the camera views are shown here. Fly position is circled in yellow, and the lower right image shows a 3D reconstruction.

You may download a high-res (1024x768) version of this movie at: tracking_movie_1024x768.mp4.

Rejection of false positives

To operate in large spaces with potentially sub-optimal imaging conditions, we have optimized Flydra to gather numerous potential 2D image locations, only some of which actually correspond to a real flying animal. Through a combination of methods (the two most important of which are classification based on heuristics regarding blob shape and data association with an existing 3D animal), only the most likely of these image locations is used to update the 3D position of the tracked animal.

This video shows tracking from nine cameras in the Mamarama system and shows all the potential 2D observations that the Flydra software evaluates. Three flies are flying in the arena. A blue circle is shown for each 2D observation, and the yellow circles show the 3D position of each fly.

You may download this movie at: tracking_movie_w_2d.mp4.

Realtime 3D tracking

By tracking animals with minimal latency (less than 40 msec), a variety of experimental designs are possible. For example, the following videos, shot by Sawyer Fuller (Caltech) and Martin Peek (Caltech), were triggered as a fly passed through a pre-defined trigger volume. By predefining a volume of interest, any object that entered this region was automatically filmed at 6000 frames per second (using Photron Fastcam APX 120 cameras). Using such techniques it is possible to capture video sequences of flies performing specific maneuvers in a completely automated way.

You may download this movie at: highspeed_3view_6000fps_20081109_2.mp4.