Cue normalisation in multisensory integration studies

The collapse of the expected multisensory benefits is a big problem in at least two contexts. First, when trying to establish evidence for multisensory benefits – the benefit produced by greatly mismatched cues is so small that it would be virtually undetectable. Secondly, and of particular interest to us, it can make it tricky to compare different people’s ability to combine different types of cues. How could we look at variability in how people integrate if the theoretical benefit is so small to begin with? The cue mismatch problem could lead to both false positives and false negatives.

That’s where cue normalisation, or cue calibration, comes in. By measuring the variable error associated with each individual cue, it becomes possible to purposely make the best cue noisier or the worst cue easier, improving the match. Normalisation can be done at the participant level, tailoring the difficulty of each cue to each participant – or even during a session (heard in a talk by Benjamin Rowland at IMRF 2025).

Why you should think carefully about how you normalise

The goal of normalisation is simple in principle: tailor cue difficulty to each participant so that all cues have comparable levels of variable error, yielding worst/best cue ratios close to 1. In practice, it can be trickier than it looks.

A common approach is to first measure a unisensory discrimination threshold for each cue, and then scale the difficulty of the cue in the multisensory task. For example, in our study, we tried to match the sensory cues in a localisation task by defining the test space in numbers of thresholds. For example, our multisensory task involves learning to use colour to locate a hidden target. If a given participant has excellent colour discrimination abilities, then they would have a low discrimination threshold in the normalisation task, and the range of colours in the localisation task we would therefore be narrower than that of a participant which a higher discrimination threshold.

While piloting for our main study, we compared two different ways to establish the threshold. The first method consisted in a non-directional oddity task. Participants judged which of three stimuli differed from the other two, and the difference between the odd stimulus and the other two was determined by staircases to obtain a threshold for each cue. For example, participants would see three coloured circles with one having a slightly different shade. The task was running fine, but we found out the cues were still greatly mismatched in the multisensory task.

Reflecting on these results, we realised that the oddity task required participants to identify a difference – any difference – between the stimuli, whereas in the multisensory localisation task, they needed to order the stimuli (e.g., identify whether a sound is coming from further left or further right, as opposed to just not being the same). That’s why we tried a second method. It was similar to the first, but used only two stimuli in a directional task: it required participants to judge not only whether the stimuli were different, but how. For example, they would need to judge whether the second coloured circle was more yellow or more pink than the first. Both methods rely on thresholds to normalise the cues, but they differ in how closely the normalisation task matches the demands of the multisensory task.