Research tools

Table of Contents

Token Sort Ratio

Table showing example TSR scores for various responses
Table showing example TSR scores for various responses
Hans Rutger Bosker (2021). Using fuzzy string matching for automated assessment of listener transcripts in speech intelligibility studies. Behavior Research Methods, 53(5), 1945-1953, doi:10.3758/s13428-021-01542-4.

PDF Cite Dataset DOI

  • Token Sort Ratio: automatically quantifying response accuracy for speech intelligiblity experiments.
  • Author: Hans Rutger Bosker.
  • https://tokensortratio.netlify.app
  • The Token Sort Ratio [TSR] score is a fuzzy string matching metric that – at the most basic level – quantifies the orthographic match between a target string and a response string (value between 0 = no match and 100 = perfect match). The TSR score has been shown to strongly correlate with human-generated scores of percentage words correct (Bosker, 2021). It is an efficient, reliable, and accurate tool for use in speech perception research (e.g., studies that examine the perception of speech in adverse listening conditions, or degraded speech) or for generating listener intelligibility measures in clinical disciplines such as speech-language pathology or audiology.

PraatVSCode

PraatVSCode syntax highlighting
PraatVSCode syntax highlighting
  • Author: Orhun Uluşahin.
  • https://github.com/orhunulusahin/praatvscode
  • Praat is an excellent software package for speech analysis, annotation, and manipulation. However, it’s scripting interface is - let’s put it this way - ‘suboptimal’. PraatVSCode is an extension for Visual Studio Code (see screenshot) that provides syntax highlighting, autocompletion, and even an array of code snippets that writes itself. Moreover, it allows running and debugging of scripts by Praat from inside Visual Studio Code.
  • How to install:
    • Download and install Visual Studio Code.
    • Under View > Extensions, search for ‘PraatVSCode’, and click install.
    • See here for running Praat scripts from inside Visual Studio Code.
  • MIT license

POnSS

POnSS annotation workflow screenshots
POnSS annotation workflow screenshots
Joe Rodd, Caitlin Decuyper, Hans Rutger Bosker, Louis ten Bosch (2021). A tool for efficient and accurate segmentation of speech data: Announcing POnSS. Behavior Research Methods, 53, 744-756, doi:10.3758/s13428-020-01449-6.

PDF Cite Dataset DOI

  • Pipeline for Online Speech Segmentation [POnSS]
  • Lead author: Joe Rodd.
  • https://git.io/Jexj3
  • POnSS is a browser-based system that is specialized for the task of segmenting the onsets and offsets of words, that combines automatic speech recognition (WebMAUS) with limited human input.
  • MIT license

Headphone screening tests

  • When running experiments online, you may want your participants to use headphones or in-ear buds (i.e., no speakers). Moreover, you may want to verify that they are wearing them ’the right way around’: [L] in their left ear, [R] in their right ear. This is particularly important when testing multitalker listening conditions and/or virtual auditory environments. Several tools exist to verify whether participants are using headphones (as instructed) or not (exclude ’m!), based on different psychoacoustic binaural phenomena. We have implemented these on Gorilla and PsyToolkit.
  • Tone attenuation based on phase-cancellation

    Woods, K. J. P., Siegel, M. H., Traer, J., & McDermott, J. H. (2017). Headphone screening to facilitate web-based auditory experiments. Attention, Perception, & Psychophysics, 79(7), 2064–2072. doi:10.3758/s13414-017-1361-2

    • General idea: binaural tones are played and participants are asked to indicate which tone out of three is quietest. Some binaural tones are played 180° out-of-phase, attenuating perceived loudness if using speakers, but not when using headphones/in-ear buds.
    • 3min test, shared by authors on Github
    • We have implemented this headphone screening test on Gorilla and PsyToolkit. Send us an email and we’d be happy to share!
  • Huggins Pitch illusion

    Milne, A. E., Bianco, R., Poole, K. C., Zhao, S., Oxenham, A. J., Billig, A. J., & Chait, M. (2021). An online headphone screening test based on dichotic pitch. Behavior Research Methods, 53(4), 1551–1562. doi:10.3758/s13428-020-01514-0

    • General idea: participants are played white noise in one ear and the same white noise but with a phase shift of 180° over a narrow frequency band to the other ear. This results in the perception of a faint tone embedded in the noise but only when using headphones/in-ear buds. Otherwise, listeners only perceive white noise (i.e., without the faint embedded tone).
    • 3min test, shared by authors on Gorilla
    • We have also implemented this headphone screening test on PsyToolkit. Send us an email and we’d be happy to share!
  • ITD and ILD manipulations
    • General idea: participants are played six trials of three binaural white noise sounds. Interaural time differences (ITDs) and interaural level differences (ILDs) are applied to the L/R channels of the stereo stimuli such that two noise sounds are perceived as coming from the left, and one as coming from the right. Participants indicate which out of the three white noise sounds comes from the right, which is easily perceived when using headphones/in-ear buds and only when wearing them ’the right way around’: [L] in left ear, [R] in right ear.
    • 3min test, implemented on Gorilla and PsyToolkit. Send us an email and we’d be happy to share!

Last updated on Jul 7, 2022