The music of plants begins with their movement (new AI technology)
Looking at plants differently: from leaf micro-kinematics to sonification by artificial intelligence A motionless plant does not exist. Over a few seconds, minutes or hours, its leaves change orientation, its stems move…
Looking at plants differently: from leaf micro-kinematics to sonification by artificial intelligence
A motionless plant does not exist.
On the scale of a few seconds, minutes or hours, its leaves change orientation, its stems move, certain parts of its foliage oscillate, contract or reorganise. These movements respond to light, the circadian cycle, water status, temperature, mechanical constraints and many internal physiological processes.
Botany has been studying these phenomena for a long time. Nyctinasty, for example, refers to the rhythmic movements of certain leaves between day and night. These movements are based in particular on changes in cell volume, ionic flows and mechanisms governed by the circadian clock.
But the arrival of artificial vision is profoundly changing the way we observe these phenomena.
A leaf has measurable dynamics
A human eye looks at a plant and generally perceives a relatively fixed structure.
A camera, combined with a temporal analysis algorithm, sees something else: a field of movement.
Each region of the foliage can be described by several variables:
-
displacement amplitude;
-
speed;
-
direction;
-
acceleration;
-
oscillation frequency;
-
synchronisation between different leaves;
-
stability or irregularity of movement;
-
changes in these parameters over time.
This approach already exists in plant phenotyping. Work using dense optical flow has shown that it is possible to track leaf movements automatically and to distinguish differences linked to genotype, water supply or nitrogen.
Research published in 2026 goes even further by using the morpho-kinematic dynamics of foliage to classify different levels of water stress.
Movement thus becomes usable biological data.
It is precisely here that the PlantSound approach begins.
From movement to a “dynamic phenotype”
A photograph tells us about a plant’s morphology.
A video sequence tells us about its dynamics.
PlantSound therefore analyses not only what the plant is, but also what it does during observation.
The algorithm begins by identifying the plant and isolating its relevant structures: overall outline, leaf masses, leaves, stems and zones with exploitable dynamics.
It then calculates a displacement field between successive images.
Conceptually, for each point in the image:
where and represent the horizontal and vertical displacement observed over time.
Taken in isolation, this vector is of little interest. Calculated across thousands of points and several seconds, it produces a genuine kinematic map of the plant.
The algorithm then extracts temporal descriptors: average movement, relative kinetic energy, directional dispersion, periodicity, coherence between different regions and the appearance of sudden changes.
The plant thus acquires a digital dynamic phenotype.
This approach is directly aligned with the current development of computer-based plant phenotyping, where artificial vision, deep learning and multimodal fusion make it possible to quantify plant characteristics that cannot be tracked manually at scale.
Why recognising the species completely changes the sonification
This is a central point.
A physical variation on its own has no intrinsic musical meaning.
A change from 0.8 to 1.2 does not mean: “play a G”.
It is the sonification algorithm that establishes the correspondence.
Sonification is, moreover, a genuine scientific field: it consists of representing data through sound, in the same way that visualisation represents data graphically. Work on the subject describes hundreds of strategies linking physical quantities to pitch, rhythm, timbre, intensity or spatialisation.
PlantSound adds an essential dimension here: botanical context.
A fern, a ficus, a Monstera or a Mimosa do not have the same architecture, the same leaves, or necessarily the same scales or rhythms of movement.
Artificial intelligence therefore first provides a botanical embedding: a numerical representation of the species and its morphological characteristics.
This embedding is then used to interpret movement.
In other words:
the same visual displacement does not necessarily have the same relative value for two different plants.
It is comparable to heart rate: 120 beats per minute cannot be interpreted independently of species, size, activity and physiological context.
The point of AI is precisely to provide context before sonifying.
A musical algorithm that does not simply play “higher when there is more movement”
An elementary sonification would do this:
movement ↑ → note pitch ↑
Technically, that works. Scientifically and musically, it is poor.
PlantSound works on a multidimensional representation.
Its principle can be summarised by a state vector:
with for example:
K = kinematic activity
C = coherence of movement
R = temporal regularity
E = environmental variables
L = light and changes in light
B = botanical context identified by AI
This vector evolves continuously.
The sonification engine then transforms different dimensions into distinct musical parameters.
Overall activity acts on musical density.
The speed of changes acts on rhythm.
Coherence between different leaves acts on consonance and harmonic stability.
Slow oscillations control texture changes.
Rapid micro-variations feed finer elements: attacks, modulation, granularity or articulation.
The environment gradually alters the harmonic and spectral space.
And the botanical profile provides a musical architecture of reference specific to the plant being analysed.
We are therefore no longer converting a single signal into a sequence of notes.
We are transforming a multidimensional plant state into a multidimensional soundscape.
The importance of time: each plant becomes its own reference
This is probably one of the most interesting research directions.
Comparing two plants directly on an absolute value produces a lot of experimental noise.
Comparing a plant with itself over time produces much richer information.
PlantSound can progressively establish a baseline:
Each new observation is then compared with the plant’s recent history.
The algorithm no longer seeks only:
“How much is this leaf moving?”
It asks:
“How does this plant’s current behaviour differ from its own habitual behaviour?”
This logic opens up a fascinating direction: the behavioural digital twin of the plant.
After several observations, the AI builds an individualised representation integrating daily rhythm, usual movement amplitude, response to changes in light, stability of the foliage and evolution of the environment.
The music then ceases to be linked only to the species.
It gradually becomes linked to this individual plant, in this environment, at this moment.
The environment is part of the observed plant
A plant cannot be analysed independently of its environment.
Light changes.
Temperature changes.
Humidity changes.
The time of day alters the circadian state.
A draught of air causes mechanical movement.
Human interaction locally alters the foliage.
The algorithm must therefore distinguish as far as possible movement caused by external influence from behavioural movement.
This is a classic source-separation problem.
If all the leaves move simultaneously in the same direction, a global air movement becomes likely.
If a specific region gradually changes orientation while the rest of the foliage remains stable, the signature is different.
The AI therefore does not look only at how much the plant moves.
It analyses where, how, with what coherence and in what context it moves.
This is precisely what turns a camera into an observation instrument.
What if sound itself became a scientific tool?
This is where sonification becomes particularly interesting.
Our auditory brain is extremely efficient at perceiving changes in rhythm, repetition, breaks and temporal patterns.
A variation that is difficult to notice in several simultaneous curves sometimes becomes immediately perceptible when it alters a sound texture.
Sound then no longer serves only to make data pleasant.
It becomes another way of exploring them.
A plant whose dynamics become progressively more irregular produces a perceptible musical transformation.
A sudden change in coherence between several zones of the foliage becomes a sound break.
A recurring circadian rhythm can become a recognisable musical structure.
Music thus becomes an interface between a complex biological system and human perception.
And this idea fits within a much broader evolution in plant research: plants produce and transmit multiple categories of signals. Their electrical activity participates in their systemic responses, and studies have also shown that certain plants subjected to different stresses emit airborne ultrasounds whose characteristics can be classified by machine learning.
The next step is therefore no longer to look for a single signal that would represent the plant, but to build a multimodal representation of its state.
Vision.
Movement.
Environment.
Temporality.
Botanical identity.
Artificial intelligence.
Then sonification.
That is the convergence we are exploring with PlantSound.
The plant does not contain a hidden melody. It contains information.
The real innovation is not to claim to discover notes inside a leaf.
It consists of transforming observable plant phenomena into a representation that human beings can perceive differently.
Looking at a plant tells us about its form.
Observing it over time tells us about its dynamics.
Artificial intelligence organises this complexity.
Sonification allows us to hear it.
And that is probably where the next generation of plant music begins.
To find out more and download the Plantsound application, click here (plantsound.fr)

