Learning to blindly follow hydrodynamic trails

Many aquatic animals are able to sense and respond to wakes left by other swimming animals or obstacles. Despite numerous studies on this topic, a feedback control strategy that requires only local and instantaneous sensory cues remains elusive. In this paper, we focus on the “flowtaxis” scenario, in which a blind swimmer with only access to local flow properties follows the wake left by an pitching airfoil. We use model-free reinforcement learning to train the swimmer using only local flow information as input. We find that the trained swimmer is able to reach the source of the wake by turning to the region with larger velocity magnitude, and having a sensor at the tail of the swimmer is crucial for flowtaxis. Moreover, the sensory control strategy is generalizable to thrust wakes of different Strouhal numbers, drag wakes, and three dimensional problem. These findings suggest that the traveling-wave characteristic of the wake is essential for successful flowtaxis. We prove the relationship between sensor location and the stability of our controller in locating the source via analysis in a reduced order signal field. We compare the success rate of different sensory cues, and find out that sensing velocity is the best in flowtaxis task. This work emphasizes the importance of both sensory cue and location of the sensor and has implication on other source seeking control problems with traveling-wave characteristic.