Exploration-Exploitation Model of Moth-Inspired Olfactory Navigation

1. Introduction

The exploration–exploitation (EE) model serves as a mathematical framework for understanding decision-making processes, in general, and in biological systems, in particular [1,2]. The EE model is rooted in the principles of adaptive behaviour as the model reflects the delicate balance between exploring new options and exploiting known resources to maximize a given objective [3]. In biological contexts, organisms face continual trade-offs between gathering information about their environment (exploration) and leveraging existing knowledge to attain immediate benefits (exploitation). This dynamic interplay is observed across diverse organisms [4–8] when these are optimizing their chances of finding food, mates and suitable habitats [9,10]. For instance, foraging birds exhibit exploration by scouting new locations to find food while exploiting reliable food sources they have come across in the past [11–13]. Similarly, specific fish species migrate to explore different environments during breeding seasons, while others stay in familiar habitats to exploit available resources consistently [14,15]. Despite its general usage and ability to explain various biological phenomena, the EE model has yet to be adopted for moth olfactory navigation, as far as we know.

Broadly speaking, male moths can navigate challenging and dynamic environments with complex turbulent flows, such as forests and canopies, to locate females of the same species over remarkably vast distances [16,17].

This phenomenon holds significance for biologists and engineers, as moths exhibit remarkable abilities to perform complex tasks despite their limited cognitive and sensory capacities. Recent evidence underscores the importance of olfactory navigation, with a growing body of research focused on this aspect, as highlighted by studies such as [18–22]. Moths adeptly navigate using spatially and temporally local cues carried by turbulent air, using chemo-receptors on their antennae for chemical sensing [23–25]. Additionally, they employ visual optometry for spatial orientation, as indicated by studies like [26]. Moreover, [27,28] show that these dynamics can be attributed to the moth’s (or insect in a more general context) decision-making. Despite these insights, our current understanding of how the moth’s cognitive or sensory mechanisms operate remains incomplete, underscoring the need for a new theoretical framework to unravel these intricate processes comprehensively. Ongoing research, exemplified by studies such as [29–33], emphasizes the pressing requirement for advancing our theoretical understanding in this fascinating field.

While observing the navigational paths of moths, researchers noted a complex behaviour characterized by the interchange of two similar yet distinct patterns: a relatively narrow ‘zigzagging’ (exploitation) and a broader ‘side-slips’ or ‘cross-wind’ motion (exploration) [34–36]. This observation prompted a growing body of research aimed at capturing and explaining this behaviour, employing a variety of models [34,37–45]. Various approaches, some of which rely on specific assumptions of probabilistic behaviour, memory, olfactory signal, or previous knowledge of moths, were shown to capture the observed data and reproduce outcomes similar to those biologically observed [44,46–51]. Recently, Macedo et al. [52] developed a simulator to compare various bioinspired and engineered strategies for chemical plume tracking. Although this simulator reasonably replicates known biological processes, it is based on the diffusion process without accounting for wind or turbulence. Consequently, it is unsuitable for exploring moth navigation with the effect of turbulence as one of the core elements of moth-inspired navigation mechanism [50].

In response to this limitation, Golov et al. [53], based on the previous simulator of pheromone plumes, introduced a computational platform, abbreviated MothPy, that simulates the behaviour of moth-like navigators, considering turbulence-driven convection and diffusion of pheromone clouds from a pulsating source. Although it provided the framework to study the problem in two-dimensional horizontal plumes, it must be extended to mimic moth navigation in three dimensions. Furthermore, a mathematical formalization is required to generalize these outcomes for generic olfactory navigation, following a similar line of other studies about the moth flight mechanism [54,55].

Previous biological studies focusing on the moth navigation assume that the wind direction is carrying pheromones generated by the female moth in a laminar, diffusive manner, keeping the integrity of the pheromone clouds as well as predictability over time and space [56,57]. These studies ignored the presence of turbulence and its effect on the pheromone pockets spreading [36,58,59], despite the observed influence of turbulence and wind direction fluctuations (meandering) [60,61]. However, the models, including the effect of turbulent velocity fluctuations, were not validated using real-world biological data, which leaves the question of their corroboration.

In this study, we propose an EE model for moth-inspired olfactory navigation where moths balance between exploring options as they search roughly speaking ‘cross-wind’ for olfactory signal (physically present within clouds of pheromone) and exploiting after the pheromone signal was obtained and the wind direction is locally evaluated. The contribution of this work lies in three main accomplishments. First, we show that an EE-based model without memory and limited sensory data can reasonably explain moth olfactory navigation in terms of the standard deviation in the data captured by the coefficient of determination metric. Second, we show that the moth’s navigation mechanism and behaviour are greatly affected by turbulent-like airflow fluctuations during the flight. Finally, we enhance the simulator proposed by Golov et al. [53], improving its ability to mimic the moth’s olfactory navigation dynamics.

The rest of the paper is organized as follows. Section 2 formally outlines the biological dataset, moth navigation simulator, EE model definition and evaluation, and turbulence effect on the moth navigation influence analysis. Next, §3 presents the results obtained from the analysis. Finally, §4 discusses the biological and engineering outcomes and proposes possible future work.

2. Methods

This study is divided into four main components: (i) we use moth flights in a controlled wind tunnel setting with a known wind speed and two flow conditions: undisturbed and disturbed (explained in detail herein). (ii) In parallel, we simulate the moth’s olfactory navigation using the extended MothPy simulator [53]. We add several modifications to make it biologically relevant by showing the model can reasonably explain the experimental dataset. (iii) Using these two components, we define and evaluate an EE-based model on a moth’s olfactory navigation. (iv) Finally, we explore several bioinspired navigation strategies and their properties using the EE-based model. Figure 1 provides a schematic view of the proposed study.

2.1. Experimental dataset

2.1.1. Apparatus and materials

The pink bollworm moths (Pectinophora gossypiella) were reared under laboratory control conditions at 25^°C, 60%, and 14:10 L:D at the Department of Entomology, The Volcani Center, Israel. Virgin adults were obtained by separating males and females following [62]. Newly emerged adult males or females were placed in screen cages and fed 10% sugar solution ad libitum. Three-day-old virgin males and females were used in all experiments. The experiments occurred in an open-circuit wind tunnel (1 × 1 × 3 m³) in a dark room where temperature and relative humidity were controlled at 25^°C and 60%, respectively. The mean wind speed was set to 0.25 m s⁻¹.

The tunnel has a built-in tracking system (Scope visual platform model Luteus-1300M-L) and an array of infrared (IR) lights outside the moths’ visible spectrum. Moth tracking CMOS cameras operating at 75 Hz were in stereoscopic viewing mode, generating a three-dimensional image field of (1 × 1 × 2 m³), imaging the space between the males’ releasing point downstream and male landing point on the odour source of female cages upstream. The camera observes the relevant region in the wind tunnel as the male and female moths’ positions are 0.5 m from either side of the wind tunnel along the flow.

Trajectory data of successful flights (i.e. males reaching the odour source) were processed and calibrated. The success rate for undisturbed flow was 55% approaching the source while under disturbed conditions it was 20%. Two flow conditions were chosen to test the effect of the wind flow velocity disturbances on the male moth’s flight trajectory, called undisturbed and disturbed. Undisturbed refers to a relatively steady flow maintained in the wind tunnel due to mesh screens at the wind tunnel’s inlet. Disturbed corresponds to an additional vertically oriented cylinder of 0.05 m in diameter and 1.0 m high, positioned 0.2 m upstream of the female cages. We ensured that the odour source was not in the cylinder recirculation region. The wake formed by this cylinder is characterized by repeating patterns of swirling vortices downwind from the cylinder, called the von Kàrmàn vortex street.

2.1.2. Wind tunnel measurements

The flight paths of male moths were tested at the peak of calling time, 4–5 h after the onset of the scotophase, at 25–27^°C. The female cage was located 0.2 m downwind from the end of the tunnel at the tunnel centre line (0.5 m). Males were randomly taken individually from the male cage and released at the downstream releasing midpoint, 2 m downstream of the odour source cages. Each male was free to take off. The dataset presented here includes 57 successful flights, where 35 experienced disturbed flow while the other 22 took flights in undisturbed flow. Figure 2 gives a top-view schematic view of the experimental set-up.

2.2. Olfactory navigation simulator

In this study, we adopted the olfactory navigation simulator, MothPy, proposed in [53]. The navigators are self-propelled flyers with a single sensor that provides binary odour detection (below/above its sensor threshold) and a timer without memory or learning. The plume model is based on another simulation for the plume of pheromone distribution in a turbulent flow [63]. Importantly, to represent spatially local sensing, the simulated agent could only access the local wind velocity and odour detection above the threshold at its position.

Although MothPy was successfully used in various studies, demonstrating the central concepts in moth-inspired navigation algorithms, it could not create paths comparable with measured moth flight trajectories. First, it was designed for two-dimensional plumes, while the trajectory data is captured in three dimensions. Second, the simulator allowed the self-propelled navigation agent to accelerate and manoeuvre instantly. Finally, the agent’s ability to accurately sense its state in terms of the wind’s direction and intensity as well as the movement of the pheromone clouds is unrealistic as sensors have some measuring error. We first extended the simulator to three dimensions (see MothPy 3D). Moreover, we introduced the acceleration limit to obtain more realistic flight paths. Lastly, we present a parameter that adds Gaussian noise concerning the mean (μ) and standard deviation (σ) to the input signals of the agent. This better mimics the sensory error for realistic environments, allowing the algorithm to accurately capture the moth’s navigation patterns. The exact values of μ and σ are configurable by the user. As default, we set μ = 0 and s = 2% of the average input signal.

We conducted the following fitting procedure to evaluate how well the proposed simulator captures empirical data. First, for each measured path, we matched a synthetic simulated path. Second, to obtain the artificial path, we ran the simulator for 100 times and took the average route. We computed multiple runs and considered the mean path. This is due to the stochastic nature of the simulator and because we aim to investigate the ‘principal’ behaviour a moth presents during olfactory navigation. Since the paths are not the same length, we used the dynamic time wrapping method with a mean absolute error (MAE) metric to compare the artificial and experimental paths [64]. In addition, we computed the coefficient of determination between these two to determine the portion of the data ‘explained’ by the model. Thus, the outcome of this procedure is a value, r ∈[0, 1], which indicates how well the observed moth navigation paths are replicated by the proposed simulator based on the proportion of total variation of outcomes explained by the simulator.

2.3. Exploration–exploitation model

We propose an EE dilemma-based theoretical model to explain moth olfactory navigation. We first apply the model to the MothPy 3D simulated paths. We assume that the agent’s path can be divided into k ∈ N ordered subsets (P) such that each subset (p_i ∈P) indicates either exploration (s) or exploitation (o). Two consecutive subsets cannot have the exact identification. Formally,

This definition is appropriate as presumably moths start their flight only when they receive some pheromone trigger, which means they have a predefined (and unclear) goal from the moment they take off.

The path is represented as a vector l ∈ R^3×n of size n ∈ N of real three-dimensional vectors (∀i ∈ [1, . . . , n] : l_i ∈ l) corresponding to the agent’s centre of mass. Since the absolute location in space is not informative, we computed the velocity vector at each point in the path using the numerical five-point stencil [65] formula v_i : = ( −l_i−2 + 8l_i−1 −8l_i+1 + l_i+2)/12. To measure the levels of EER, which quantifies the amount of exploration the agent performs in a given segment in time, of a subset that starts in index t_s and ends in index t_e > t_s, we adopted the metric proposed in [66]. Namely, the exploration score of a path p, which is represented by a velocity vector (V), is defined as follows:

where v̄ := ^P_vi∈V(v_i)/(y − x) [67,68]. Intuitively, this metric measures how aligned the vectors are towards a central direction (v̄), the mean velocity vector in the segment. Following this metric, a division of the path according to the EE model with k subsets maximize

In practice, equation (2.3) imposes a nonlinear optimization task that can be solved using various methods. We adopted a heuristic approach for this task to balance the computational burden, solution simplicity and accuracy of the obtained results. First, a ‘guess’ of the number of subsets, k, to range between two and arbitrary large number, z ∈ N , n. For each value of k, one must pick k −1 indexes for the division aiming to maximize equation (2.3). We used a genetic algorithm (GA) approach [69–73]. First, a random ‘population’ of size a ≫ 1 ∈ N where each individual in the population is defined by the k −1 indexes, chosen randomly uniformly between the values x and y. Following the standard practice for GAs, we expressed three operators: mutation, cross-over, and next generation. For the mutation operator, an index is chosen randomly and either increased or decreased by a value of one with a 0.5 probability; the cross-over operator consists of two individuals—I₁ and I₂ starts by randomly choosing an index j ∈ [2, . . ., k − 2] uniformly. Based on this index, two new individuals are created as follows:

Finally, we adopted the ‘tournament with royalty’ operator for the new-generation operator. Initially, the fitness of all individuals in the population is computed using equation (2.3) and ordered from high to low. A portion of the population, β ∈(0, 1), is kept for the next generation. In addition, other individuals in the population are selected using the following tournament selection process. Suppose there are w individuals to be chosen. In each iteration, an individual is selected from the remaining population with a probability corresponding to its normalized fitness score (normalized such that the sum of all individuals’ fitness scores is 1). This process is repeated until W individuals are selected. The three operators are repeated for a predefined z ∈ N number of times. At the end, the individual with the highest fitness score is taken. At this stage, we have z −1 possible EE divisions corresponding to different values of k. Therefore, we use the elbow-point [74,75] to find the best value of k and the final EE division.

		MothPy-3D + bounded	MothPy-3D + noisy	MothPy-3D + bounded
configuration	MothPy-3D	acceleration	input	acceleration + noisy input
MAE	13.845 ± 2.701	12.893 ± 2.948	13.317 ± 3.086	12.748 ± 3.027
R2	0.593 ± 0.062	0.712 ± 0.053	0.695 ± 0.064	0.720 ± 0.049

Following the GA algorithm, the EE division can be obtained. As such, we used the simulator to generate many paths while also recalling the steps in which the simulator altered its state to be the baseline of the EE division. Using these settings, we evaluate the accuracy of the proposed EE division method by computing the precision of a binary classification task. Namely, the EE division classifies each step in time into either o or s. If the division is done perfectly, the accuracy score would be one, while errors in the division location or number of divisions would lower this score.

2.4. Moth-inspired navigation mechanism analysis

The pheromone source location and its quality play a crucial role in the moth’s decision-making process during navigation. We compared this mechanism with other published navigation algorithms [53], demonstrating that the navigator based on the idea of exploitation rate proportional to the previously acquired pheromone signal yields higher success rates, see table 1. An essential part of this mechanism relies on wind direction and turbulent fluctuations, determining the mixing and spreading rates of the pheromone clouds.

Since turbulent and disturbed flows vigorously mix and disperse the pheromone clouds, as shown schematically by the ‘envelope’ of the pheromone plume, this should result in more exploration efforts. Thus, we hypothesize that there is a monotonic decreasing connection between the distance from the target (the female moth) and the EER. To compute the EER, we first EE divide the path of each male moth, as schematically presented in figure 1 (third panel). Right after, we calculate the EER in locations with some Euclidean distance from the target, up to a 0.06 m error radius, and report the mean and standard deviation values. We divide the cases where the cylinder was and was not present to account for the level of disturbance to the mean flow.

3. Results

In this section, we present the results obtained from the experiments, divided into the simulator’s ability to capture and explain real-world data, how accurate the EE model and its fitting method, and the connection between the distance from the target and exploration requirements for different turbulence settings.

3.1. Simulator efficiency

Table 1 shows the MAE and the coefficient of determination (R²) of the simulator compared with the real-world data, using the different subsets of the three improvements added to the MothPy simulator. With all three modifications included, the simulator can explain 72% of the data, achieving a 12.7% improvement over the previous MothPy version. An additional comparison with the original (two-dimensional case) MothPy with different axes trajectories is provided in the appendix.

3.2. Exploration–exploitation model evaluation

To evaluate the EE model, the simulator generated 10 000 paths, balancing between computational burden and a large sample to obtain statistically significant results. For each track, the number and the indexes of the divisions between the exploration and exploitation phases are recalled. The proposed EE fitting procedure is then computed on these paths, identifying each step for each route as either exploration (s) or exploitation (o). Overall, we obtained an accuracy score of 84.46%. To be exact, the accuracy score captures the number of samples in each path that are binary classified correctly to be either exploration or exploitation samples. The report results are taken to be the average accuracy score over all the paths. For example, figure 3 shows the random EE split of a single moth flight, as computed by the algorithm.

Moreover, in order to evaluate the dynamics of the EER as a function of the odour pocket frequency, we used the simulator to measure the average ± s.d. of n = 100 simulations with random odour pockets, each time with different frequency, ranging from one odour pocket generated from the female source every second and up to nine odour pockets generated every second. Table 2 summarizes the results of this analysis. The EER rate declines as the odour pockets frequency increases, on average. There is a sharp decline from one odour pocket a second to three with a more moderate decline as the odour pocket frequency increases.

3.3. Supporting our exploration–exploitation model hypothesis using disturbed flow case

Figure 4 depicts the flight path where the x-axis indicates the Euclidian distance of the agent (i.e. the male moth) from the target (the female moth) in centimetres. The y-axis indicates the EER. The results are shown as each set’s mean ± s.d. as a colour-shaded envelope. In addition, we divided the cases for undisturbed and disturbed flows, indicated by the red axes and the blue circles, respectively. As hypothesized, there is a monotonic decrease in EER concerning the distance from the target. One can note that both cases behave similarly up to 0.6 m from the target. However, at smaller distances, the two cases separate such that the case with the cylinder present decreases more slowly than the other, resulting in relatively higher EER.

odour pockets frequency (s)	1	3	5	7	9
exploration–exploitation rate (EER)	0.57 ± 0.08	0.52 ± 0.06	0.44 ± 0.05	0.40 ± 0.05	0.39 ± 0.05

Figure 5 presents the histograms of the cylinder (brown) and no cylinder (blue) experiments depicting the average EER such that the x-axis indicates the average EER of each flight. The y-axis indicates the number of flights with the same EER. The lines show a kernel density approximation of the data, added for clarity. As suggested, there is a monotonic decrease in exploration to exploitation rate concerning the distance from the target. Noticeably, in the cases where the cylinder is present, the EER is higher, with almost 5% more, on average. Furthermore, the cases in the undisturbed flow have less diverse dynamics than those in the disturbed flow.

4. Discussion and conclusion

The EE dilemma is a fundamental concept in decision-making that has implications across various domains [76–78]. This paper shows that an EE-based model can explain moths’ olfactory-driven flights. We offer a memory-free algorithm that captures only the upwind direction and binary indication for the presence of pheromones—both with some level of noise.

We improved the moth flight simulation, MothPy, by introducing three modifications: three-dimensional settings, bounded acceleration vector size, and Gaussian noise in the sensory input. As shown by table 1, with these modifications, the proposed simulator obtains a coefficient of determination of R² = 0.720 when tested on n = 57 moth flight paths. For comparison, the simulator received only R² = 0.593 for the same data without the changes, indicating a 21.4% relative and 12.7% absolute improvement. Based on these results, it is suggested that the simulator can capture the central navigation dynamics of a moth while missing local and settings-dependent conditions such as the influence of temperature, humidity and the agent’s properties (such as size) [79].

Using the proposed simulator, we show that the EE model can explain the moth’s olfactory navigation. Indeed, when fitting an EE model on the simulator data, which describes 72% of the real-world data, we obtained that 84.5% of the time, the EE model predicted accurately when the agents use a known signal and when it searches for such signal as indicated in table 1 and §3.2. This outcome is unsurprising as EE is known to explain many animal behaviour types in nature [80–82]. From a computational point of view, EE is useful as a model for optimization and search tasks [83,84].

Since the EE model can reasonably explain the moth’s olfactory navigation, one can use it to describe the changes in the flight behaviour of the moth in different settings. To this end, we focused on the effect turbulence causes. Thus, the main result is in figure 4, as it demonstrates that the original hypothesis of the moth-inspired navigation algorithm [61], highlighted here in terms of EE model, explains the observed moth flight paths: (i) EER reduces as the male approaches female source due to the plume width as a function of distance, or exploration is proportional to the plume width and (ii) turbulence affects pheromone plume dispersion rate in the proximity to the female source, and it increases the exploration rate. Thus, the search efforts are reduced due to the distance from the target. Indeed, it is shown in figure 4 that for both the cases with flow disturbance (i.e. the cylinder is present) and without it, for distances higher than 0.6 m, the EER decreases linearly. After this point, the presence of a cylinder, which potentially increases the turbulence, causes the agents to spend almost twice the exploration efforts compared with the case where it flows virtually in parallel and does not cause much distortion. Figure 5 further supports this outcome as the mean EER for the cases where the cylinder is present is higher, on average, and causes more diverse navigation behaviour.

Generally, when considering the classical diffusion dynamics from a computational perspective, the ‘information’ the diffusion carries within the concentration field reduces exponentially as a function of the distance from the source. Hence, an agent would have to spend more time searching to collect enough information, which is decaying and spreading across a larger volume of space. Hence, the diffusion dynamics is a lower boundary to the search required for an agent. The results are unsurprising when treating the dynamics as a spatio-temporal optimization task based on an agent’s available information. Hence, the proposed study does not introduce any novel biological understanding of moth navigation but rather is inspired by it to establish a mathematical framework, in the form of the EE model, to allow further exploration and development of robotics solutions based on similar principles.

This research has several limitations. First, the biological data used as a baseline was obtained in sterile settings where only a single goal is present to the male moth, and environmental factors such as temperature and humidity were fixed. As such, the used data only approximates what realistically happens in nature. Future work can focus on curating and using such data to evaluate the proposed simulator and model. Second, the proposed simulator assumes moths have only a single pheromone sensor. However, moths can differ between multiple discriminate between different pheromone blends and follow those emitted by conspecific females [79]. Ignoring this intricate sensory system could limit the model’s accuracy and ability to explain the moth navigation. Exploring different sets of pheromone senses and the computational effort they require is a promising future venue. Third, while we show a stochastic EE model to explain the data, it is fair to assume that a deterministic but more complex model would be able to determine the phase changing between exploration and exploitation, which can benefit engineering developments based on the proposed model [85]. Fourth, moths have two fronto-lateral sensors that sample the plume simultaneously providing a more sophisticated input signal than proposed in our model. Future work may refine our model to take this sensory behaviour into account. Finally, there are two different diffusion coefficients for vertical and horizontal directions [61], but we treated them identically, which can result in less accurate physical simulation in near-ground settings.

Our results highlight the potential of using the well-established EE model alongside moth-inspired olfactory navigation to obtain an agent based on a single spatially sparse signal with noise due to turbulence. This outcome can be the foundation of robotic agents to solve similar navigation tasks.

Ethics. This work did not require ethical approval from a human subject or animal welfare committee. Data accessibility. The data used in this study is available by a formal request from the authors. Declaration of AI use. We have not used AI-assisted technologies in creating this article.

Authors’ contributions. T.L.: conceptualization, formal analysis, investigation, methodology, software, project administration, visualization, writing— original draft, writing—review and editing; Y.G.: data curation; R.G.: data curation, writing—review and editing; A.H.: data curation; A.L.: conceptualization, formal analysis, investigation, methodology, validation, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein. Conflict of interest declaration. We declare we have no competing interests.

Funding. This research received no specific grant from funding agencies in the public, commercial or not-for-profit sectors.

Appendix A

Intuitively, one can compare the three-dimensional real-world data with a two-dimensional simulator by projecting the three-dimensional data into the XY, XZ or YZ two-dimensional plans and conducting the same analysis. Table 3 compares the two-dimensional version of the MothPy simulator in all possible projection comparison settings with the MothPy-3D version. One can note that the XY is better compared with the XZ and YZ trajectories, while all three are slightly worse, on average, compared with the three-dimensional version of the simulator. Moreover, we used a non-parametric k-sample Anderson–Darling test [86] to determine if the cases are statistically significantly different from one another with p < 0.05. The test reveals that while the three-dimensional case outperforms the two-dimensional cases with XZ and YZ trajectories, it does not surpass the XY trajectory (p = 0.094 > 0.05). Interestingly, when computing the contribution of each axis using a principal component analysis, one obtains that the X, Y and Z axes are capturing 42%, 35% and 23% of the dynamic’s information, respectively. This outcome can explain why the XY trajectory outperforms the other two trajectories as it relatively captures more data. Notably, unlike table 1 that reported the MAE of the simulator, in this experiment, we do not notify the MAE as it is lower for two dimensions compared with three dimensions only due to introducing an additional dimension.

configuration	MothPy-3D	MothPy-2D XY	MothPy-2D XZ	MothPy-2D YZ
R2	0.593 ± 0.062	0.570 ± 0.057	0.548 ± 0.044	0.539 ± 0.059

Article notes

Publication history

Received 14 December 2023 · Accepted 25 April 2024

Keywords

• physical simulation
• biological signal processing
• bioinspired algorithm

References

1. Dichio V, De Vico Fallani F. 2024 Exploration-exploitation paradigm for networked biological systems. Phys. Rev. Lett. 132, 098402. ( doi:10.1103/PhysRevLett.132.098402)
2. Kwa HL, Leong Kit J, Bouffanais R. 2022 Balancing collective exploration and exploitation in multi-agent and multi-robot systems: a review. Front. Robot. AI 8, 0–0.
3. Chen J, Xin B, Peng Z, Dou L, Zhang J. 2009 Optimal contraction theorem for exploration–exploitation tradeoff in search and optimization. IEEE Trans. Syst., Man, Cybern. 39, 680–691. ( doi:10.1109/TSMCA.2009.2012436)
4. Mehlhorn K, Newell BR, Todd PM, Lee MD, Morgan K, Braithwaite VA, Hausmann D, Fiedler K, Gonzalez C. 2015 Unpacking the exploration–exploitation tradeoff: a synthesis of human and animal literatures. Decision 2, 191–215. ( doi:10.1037/dec0000033)
5. Cook Z, Franks DW, Robinson EJH. 2013 Exploration versus exploitation in polydomous ant colonies. J. Theor. Biol. 323, 49–56. ( doi:10.1016/j.jtbi.2013.01.022)
6. Viseras A, Losada RO, Merino L. 2016 Planning with ants: efficient path planning with rapidly exploring random trees and ant colony optimization. Int. J. Adv. Rob. Syst. 13, 1–16. ( doi:10.1177/1729881416664078)
7. Cinotti F, Fresno V, Aklil N, Coutureau E, Girard B, Marchand AR, Khamassi M. 2019 Dopamine blockade impairs the exploration-exploitation trade-off in rats. Sci. Rep. 9, 6770. ( doi:10.1038/s41598-019-43245-z)
8. Lazebnik T, Spiegel O. 2023 Individual variation affects outbreak magnitude and predictability in an extended multi-pathogen SIR model of pigeons vising dairy farms. arXiv. ( doi:10.48550/arXiv.2310.08613)
9. Monk CT, Barbier M, Romanczuk P, Watson JR, Alos J, Nakayama S, Rubenstein DI, Levin SA, Arlinghaus R. 2018 How ecology shapes exploitation: a framework to predict the behavioural response of human and animal foragers along exploration–exploitation trade-offs. Ecol. Lett. 21, 779–793. ( doi:10.1111/ele.12949)
10. O’Farrell S, Sanchirico JN, Spiegel O, Depalle M, Haynie AC, Murawski SA, Perruso L, Strelcheck A. 2019 Disturbance modifies payoffs in the explore-exploit trade-off. Nat. Commun. 10, 3363. ( doi:10.1177/1729881416664078)
11. Antoniou P, Pitsillides A, Blackwell T, Engelbrecht A, Michael L. 2013 Congestion control in wireless sensor networks based on bird flocking behavior. Comput. Netw. 57, 1167–1191. ( doi:10.1016/j.comnet.2012.12.008)
12. Emlen JT. 1952 Flocking behavior in birds. Auk 69, 160–170. ( doi:10.2307/4081266)
13. Bella-Fernandez M, Sune MS, de Liano BGG. 2022 Foraging behavior in visual search: a review of theoretical and mathematical models in humans and animals. Psychol. Res. 86, 331–349. ( doi:10.1007/s00426-021-01499-1)
14. Partridge BL. 1982 The structure and function of fish schools. Sci. Am. 246, 114–123. ( doi:10.1038/scientificamerican0682-114)
15. Partridge BL, Pitcher T, Cullen JM, Wilson J. 1980 The three-dimensional structure of fish schools. Behav. Ecol. Sociobiol. 6, 277–288. ( doi:10.1007/BF00292770)
16. Elkinton JC, Schal C, Ono T, Cardé RT. 1987 Pheromone puff trajectory and upwind flight of male gypsy moth in a forest. Physiol. Entomol. 12, 399–406. ( 1987.tb00766.x) doi:10.1111/j.1365-3032
17. Brady J, Gibson G, Packer MJ. 1989 Odor movement, wind direction, and the problem of host-finding by tsetse flies. Physiol. Entomol. 14, 369–380. ( 1989.tb01105.x) doi:10.1111/j.1365-3032
18. Bau J, Cardé RT. 2015 Modeling optimal strategies for finding a resource-linked, windborne odor plume: theories, robotics, and biomimetic lessons from flying insects. Integr. Comp. Biol. 55, 461–477. ( doi:10.1093/icb/icv036)
19. Grünbaum D, Willis M. 2015 Spatial memory-based behaviors for locating sources of odor plumes. Mov. Ecol. 3, 3–11. ( doi:10.1186/s40462-015-0037-6)
20. Baker K, Dickinson M, Findley T, Gire H, Louis M, Suver M, Verhagen J, Nagel K, Smear M. 2018 Algorithms for olfactory search across species. J. Neurosci. 38, 9383–9389. (doi:10. 1523/JNEUROSCI.1668-18.2018) doi:10.1523/JNEUROSCI.1668-18.2018
21. Schneider D. 1964 Insect antennae. Annu. Rev. Entomol. 9, 103–122. ( doi:10.1146/annurev.en.09.010164.000535)
22. Slifer E. 1970 The structure of arthropod chemoreceptors. Annu. Rev. Entomol. 15, 121–142. ( doi:10.1146/annurev.en.15.010170.001005)
23. Li C, Dong H, Zhao K. 2018 A balance between aerodynamic and olfactory performance during flight in Drosophila. Nat. Commun. 9, 3215. ( doi:10.1038/s41467-018-05708-1)
24. Baker TC, Vogt RG. 1988 Measured behavioural latency in response to sex-pheromone loss in the large silk moth Antheraea polyphemus. J. Exp. Biol. 137, 29–38. ( doi:10.1242/jeb.137.1.29)
25. Kuenen LPS, Carde RT. 1994 Strategies for recontacting a lost pheromone plume: casting and upwind flight in the male gypsy moth. Physiol. Entomol. 19, 15–29. ( 1365-3032.1994.tb01069.x) doi:10.1111/j
26. Vickers NJ. 2000 Mechanisms of animal navigation in odor plumes. Biol. Bull. 198, 203–12. ( doi:10.2307/1542524)
27. Demir M, Nadakia N, Anderson HD, Clark DA, Emonet T. 2020 Walking Drosophila navigate complex plumes using stochastic decisions biased by the timing of odor encounters. eLife 9, e57524. ( doi:10.7554/eLife.57524)
28. Adden A, Stewart TC, Webb B, Heinze S. 2022 A neural model for insect steering applied to olfaction and path integration. Neural Comput. 34, 2205–2231. ( doi:10.1162/neco_a_01540)
29. Vickers NJ. 2006 Winging it: moth flight behavior and responses of olfactory neurons are shaped by pheromone plume dynamics. Chem. Sens. 31, 155–66. ( doi:10.1093/chemse/bjj011)
30. Harari AR, Zahavi T, Thiéry D. 2011 Fitness cost of pheromone production in signaling female moths. Evolution 65, 1572–1578. ( doi:10.1111/j.1558-5646.2011.01252.x)
31. Szyszka P, Emonet T, Edwards TL. 2023 Extracting spatial information from temporal odor patterns: insights from insects. Curr. Opin. Insect Sci. 59, 101082. ( 2023.101082) doi:10.1016/j.cois
32. Shigaki S, Ando N, Sakurai T, Kurabayashi D. 2023 Analysis of odor-tracking performance of silk moth using a sensory–motor intervention system. Integr. Comp. Biol. 63, 343–355. ( doi:10.1093/icb/icad055)
33. Gaurav K, Ranjan P. 2023 Moth-inspired odor source localization using robotic platforms: a comprehensive review. Adapt. Behav. 0, 0–0.
34. Kennedy JS. 1983 Zigzagging and casting as a programmed response to wind-borne odour: a review. Physiol. Entomol. 8, 109–120. ( doi:10.1111/j.1365-3032.1983.tb00340.x)
35. David CT, Kennedy JS, Ludlow AR. 1983 Finding of a sex pheromone source by gypsy moths released in the field. Nature 303, 804–806. ( doi:10.1038/303804a0)
36. Mafra-Neto A, Carde RT. 1994 Fine-scale structure of pheromone plumes modulates upwind orientation of flying moths. Nature 369, 142–144. ( doi:10.1038/369142a0)
37. Tobin TR. 1981 Pheromone orientation: role of internal control mechanisms. Science 214, 1147–1149. ( doi:10.1126/science.214.4525.1147)
38. Baker TC, Vickers NJ. 1997 Pheromone-mediated flight in moths. In Pheromone research: new directions (eds RT Carde, AK Minks), pp. 248–264. New York, NY: Chapman and Hall.
39. Cardé RT, Mafra-Neto A. 1997 Mechanisms of flight of male moths to pheromone. In Pheromone research: new directions (eds RT Cardé, AK Minks), pp. 275–290. New York, NY: Chapman and Hall.
40. Vickers NJ. 1999 The effects of chemical and physical features of pheromone plumes upon the behavioral responses of moths. In Advances in chemical signals in vertebrates (eds RE Johnston, D Miiller-Schwarze, PW Sorenson), pp. 63–76. New York, NY: Kluwer Academic/Plenum Publishers.
41. Kennedy JS, Marsh D. 1974 Pheromone regulated anemotaxis in flying moths. Science 184, 999–1001. ( doi:10.1126/science.184.4140.999)
42. Cardé RT, Hagaman TE. 1984 Mate location strategies of gypsy moths in dense populations. J. Chem. Ecol. 10, 25–31. ( doi:10.1007/BF00987640)
43. Baker TC, Willis MA, Phelan PL. 1984 Optomotor anemotaxis polarizes self-steered zigzagging in flying moths. Physiol. Entomol. 9, 365–376. ( tb00777.x) doi:10.1111/j.1365-3032.1984
44. Willis MA, Arbas EA. 1991 Odor-modulated upwind flight of the sphinx moth, Manduca sexta L. J. Comp. Physiol. A 169, 427–40. ( doi:10.1007/BF00197655)
45. Mafra-Neto A, Cardé RT. 1996 Dissection of the pheromone-modulated flight of moths using single-pulse response as a template. Experientia 52, 373–379. (doi:10.1007/ BF01919543) doi:10.1007/BF01919543
46. Belanger JH, Arbas EA. 1998 Behavioral strategies underlying pheromone-modulated flight in moths: lessons from simulation studies. J. Comp. Physiol. A 183, 345–360. (doi:10. 1007/s003590050261) doi:10.1007/s003590050261
47. Cardé RT, Willis MA. 2008 Navigational strategies used by insects to find distant, wind-borne sources of odor. J. Chem. Ecol. 34, 854–866. ( doi:10.1007/s10886-008-9484-5)
48. Balkovsky E, Shraiman BI. 2002 Olfactory search at high Reynolds number. Proc. Natl Acad. Sci. USA 99, 12589–93. ( doi:10.1073/pnas.192393499)
49. Vergassola M, Villermaux E, Shraiman B. 2007 ‘Infotaxis’ as a strategy for searching without gradients. Nature 445, 406–409. ( doi:10.1038/nature05464)
50. Li W, Farrell JA, Cardé RT. 2001 Tracking of fluid-advected odor plumes: strategies inspired by insect orientation to pheromone. Adapt. Behav. 9, 143–170. ( doi:10.1177/10597123010093003)
51. Li W. 2010 Identifying an odour source in fluid-advected environments, algorithms abstracted from moth-inspired plume tracing strategies. Appl. Bionics Biomech. 7, 3–17. ( doi:10.1155/2010/287801)
52. Macedo J, Marques L, Costa E. 2019 A comparative study of bio-inspired odour source localisation strategies from the state-action perspective. Sensors 19, 2231. ( doi:10.3390/s19102231)
53. Golov Y, Benelli N, Gurka R, Harari A, Zilman G, Liberzon A. 2021 Open-source computational simulation of moth-inspired navigation algorithm: a benchmark framework. MethodsX 8, 101529. ( doi:10.1016/j.mex.2021.101529)
54. Talley JL, White EB, Willis MA. 2023 A comparison of odor plume-tracking behavior of walking and flying insects in different turbulent environments. J. Exp. Biol. 226, jeb244254. ( doi:10.1242/jeb.244254)
55. Willis MA, Ford EA, Avondet JL. 2013 Odor tracking flight of male Manduca sexta moths along plumes of different cross-sectional area. J. Comp. Physiol. A 199, 1015–1036. ( doi:10.1007/s00359-013-0856-0)
56. Baker TC, Willis M, Haynes KF, Phelan PL. 1985 A pulsed cloud of sex pheromone elicits upwind flight in male moths. Physiol. Entomol. 10, 257–265. ( 1985.tb00045.x) doi:10.1111/j.1365-3032
57. Murlis J, Elkinton JS, Cardé RT. 1992 Odor plumes and how insects use them. Annu. Rev. Entomol. 37, 505–532. ( doi:10.1146/annurev.en.37.010192.002445)
58. Fanger PO, Melikov AK, Hanzawa H, Ring J. 1988 Air turbulence and sensation of draught. Energy Build. 12, 21–39. ( doi:10.1016/0378-7788(88)90053-9)
59. Celani A, Villermaux E, Vergassola M. 2014 Odor landscapes in turbulent environments. Phys. Rev. X 4, 041015. ( doi:10.1103/PhysRevX.4.041015)
60. Cardé RT. 2021 Navigation along windborne plumes of pheromone and resource-linked odors. Annu. Rev. Entomol. 66, 317–336. ( doi:10.1146/annurev-ento-011019-024932)
61. Liberzon A, Harrington K, Daniel N, Gurka R, Harari A, Zilman G. 2017 Moth-inspired navigation algorithm in a turbulent odor plume from a pulsating source. PLoS ONE 9, 143–170. ( doi:10.1371/journal.pone.0198422)
62. Dou X, Liu S, Soroker V, Harari A, Jurenka R. 2019 Pheromone gland transcriptome of the pink bollworm moth, Pectinophora gossypiella: comparison between a laboratory and field population. PLoS ONE 14, e0220187. ( doi:10.1371/journal.pone.0220187)
63. Farrell JA, Murlis J, Long X, Li W, Cardé RT. 2002 Filament-based atmospheric dispersion model to achieve short time-scale structure of odor plumes. Environ. Fluid Dyn. 2, 143–169. ( doi:10.1023/A:1016283702837)
64. Müller M. 2007 Dynamic time warping. In Information retrieval for music and motion, pp. 69–84. Berlin, Germany: Springer .
65. Zingg DW. 2000 Comparison of high-accuracy finite-difference methods for linear wave propagation. SIAM J. Scientific Comput. 22, 476–502. ( doi:10.1137/S1064827599350320)
66. Krongauz DL, Lazebnik T. 2023 Collective evolution learning model for vision-based collective motion with collision avoidance. PLoS ONE 18, 1–22. ( 0270318) doi:10.1371/journal.pone
67. Ariel G, Ayali A. 2015 Locust collective motion and its modeling. PLoS Comput. Biol. 11, e1004522. ( doi:10.1371/journal.pcbi.1004522)
68. Vicsek T, Czirok A, Ben-Jacob E, Cohen I, Shochet O. 1995 Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett. 75, 1226–1229. (doi:10.1103/ PhysRevLett.75.1226) doi:10.1103/PhysRevLett.75.1226
69. Searson DP, Leahy DE, Willis MJ. 2010 GPTIPS: an open source genetic programming toolbox for multigene symbolic regression. In Proc. of the Int. Multiconference of Engineers and Computer Scientists, 1, pp. 77–80.
70. Keren LS, Liberzon A, Lazebnik T. 2023 A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge. Sci. Rep. 13, 1249. ( doi:10.1038/s41598-023-28328-2)
71. Alexi A, Lazebnik T, Shami L. 2023 Microfounded tax revenue forecast model with heterogeneous population and genetic algorithm approach. Comput. Econ. 63, 1705–1734.
72. Kumar M, Husain M, Upreti N, Gupta D. 2010 Genetic algorithm: review and application. Int. J. Inf. Technol. Knowl. Manage. 2, 451–454. ( doi:10.2139/ssrn.3529843)
73. Lazebnik T, Somech A, Weinberg AI. 2022 SubStrat: a subset-based optimization strategy for faster AutoML. Proc. VLDB Endow. 16, 772–780. ( doi:10.14778/3574245.3574261)
74. Humaira H, Rasyidah R. 2020 Determining the appropiate cluster number using elbow method for K-means algorithm. In Proc. of the 2nd Workshop on Multidisciplinary and Applications (WMA), Padang, Indonesia, 24–25 January. EAI. ( doi:10.4108/eai.24-1-2018.2292388)
75. Bholowalia P, Kumar A. 2014 EBK-Means: a clustering technique based on elbow method and K-means in WSN. Int. J. Comput. Appl. 105, 17–24.
76. Berger-Tal O, Nathan J, Meron E, Saltz D. 2014 The exploration-exploitation dilemma: a multidisciplinary framework. PLoS ONE 9, 1–8. ( doi:10.1371/journal.pone.0095693)
77. Sidhu JS, Commandeur HR, Volberda HW. 2007 The multifaceted nature of exploration and exploitation: value of supply, demand, and spatial search for innovation. Organ. Sci. 18, 20–38. ( doi:10.1287/orsc.1060.0212)
78. Wilson RC, Geana A, White JM, Ludvig EA, Cohen JD. 2014 Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol.: General 143, 2074–2081. ( doi:10.1037/a0038199)
79. Golov Y, Liberzon A, Gurka R, Soroker V, Jurenka R, Harari A. 2022 Article navigation in an odorant-landscape: mate finding and mate choice in a nocturnal moth. Entomol. Gen. 42, 323–334. ( doi:10.1127/entomologia/2021/1276)
80. Dursun I, Akcay A, van Houtum GJ. 2022 Age-based maintenance under population heterogeneity: optimal exploration and exploitation. Eur. J. Oper. Res. 301, 1007–1020. ( doi:10.1016/j.ejor.2021.11.038)
81. Gameiro RR, Kaspar K, Konig SU, Nordholt S, Konig P. 2017 Exploration and exploitation in natural viewing behavior. Sci. Rep. 7, 2311. ( doi:10.1038/s41598-017-02526-1)
82. Sun Y, Lu X, Guo W. 2014 A review on simulation models for exploration and exploitation of natural gas hydrate. Arab. J. Geosci. 7, 2199–2214. ( doi:10.1007/s12517-014-1294-1)
83. Martinez-Cantin R, de Freitas N, Brochu E, Castellanos J, Doucet A. 2009 A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Auton. Rob. 27, 93–103. ( doi:10.1007/s10514-009-9130-2)
84. Kaplan R, Friston KJ. 2018 Planning and navigation as active inference. Biol. Cybern. 112, 323–343. ( doi:10.1007/s00422-018-0753-2)
85. Wang L, Pang S. 2022 Robotic odor source localization via adaptive bio-inspired navigation using fuzzy inference methods. Rob. Auton. Syst. 147, 103914. ( 2021.103914) doi:10.1016/j.robot
86. Thas O, Ottoy JP. 2004 An extension of the Anderson–Darling k-sample test to arbitrary sample space partition sizes. J. Stat. Comput. Simul. 74, 651–665. ( doi:10.1080/00949650310001623399)