Skip to content

Testing the performances of automated identification of bat echolocation calls: A request for prudence

Research output: Contribution to journalArticle

Standard

Testing the performances of automated identification of bat echolocation calls : A request for prudence. / Rydell, Jens; Nyman, Stefan; Eklöf, Johan; Jones, Gareth; Russo, Danilo.

In: Ecological Indicators, Vol. 78, 07.2017, p. 416-420.

Research output: Contribution to journalArticle

Harvard

APA

Vancouver

Author

Rydell, Jens ; Nyman, Stefan ; Eklöf, Johan ; Jones, Gareth ; Russo, Danilo. / Testing the performances of automated identification of bat echolocation calls : A request for prudence. In: Ecological Indicators. 2017 ; Vol. 78. pp. 416-420.

Bibtex

@article{9e0ccca25b8b4c44a1293df3b117133b,
title = "Testing the performances of automated identification of bat echolocation calls: A request for prudence",
abstract = "Echolocating bats are surveyed and studied acoustically with bat detectors routinely and worldwide, yet identification of species from calls often remains ambiguous or impossible due to intraspecific call variation and/or interspecific overlap in call design. To overcome such difficulties and to reduce workload, automated classifiers of echolocation calls have become popular, but their performance has not been tested sufficiently in the field. We examined the absolute performance of two commercially available programs (SonoChiro and Kaleidoscope) and one freeware package (BatClassify). We recorded noise from rain and calls of seven common bat species with Pettersson real-time full spectrum detectors in Sweden. The programs could always (100{\%}) distinguish rain from bat calls, usually (68–100{\%}) identify bats to group (Nyctalus/Vespertilio/Eptesicus, Pipistrellus, Myotis, Plecotus, Barbastella) and usually (83–99{\%}) recognize typical calls of some species whose echolocation pulses are structurally distinct (Pipistrellus pygmaeus, Barbastella barbastellus). Species with less characteristic echolocation calls were not identified reliably, including Vespertilio murinus (16–26{\%}), Myotis spp. (4–93{\%}) and Plecotus auritus (0–89{\%}). All programs showed major although different shortcomings and the often poor performance raising serious concerns about the use of automated classifiers for identification to species level in research and surveys. We highlight the importance of validating output from automated classifiers, and restricting their use to specific situations where identification can be made with high confidence. For comparison we also present the result of a manual identification test on a random subset of the files used to test the programs. It showed a higher classification success but performances were still low for more problematic taxa.",
keywords = "Biosonar, Methodology, Software, Species identification, Ultrasound",
author = "Jens Rydell and Stefan Nyman and Johan Ekl{\"o}f and Gareth Jones and Danilo Russo",
year = "2017",
month = "7",
doi = "10.1016/j.ecolind.2017.03.023",
language = "English",
volume = "78",
pages = "416--420",
journal = "Ecological Indicators",
issn = "1470-160X",
publisher = "Elsevier Science",

}

RIS - suitable for import to EndNote

TY - JOUR

T1 - Testing the performances of automated identification of bat echolocation calls

T2 - A request for prudence

AU - Rydell, Jens

AU - Nyman, Stefan

AU - Eklöf, Johan

AU - Jones, Gareth

AU - Russo, Danilo

PY - 2017/7

Y1 - 2017/7

N2 - Echolocating bats are surveyed and studied acoustically with bat detectors routinely and worldwide, yet identification of species from calls often remains ambiguous or impossible due to intraspecific call variation and/or interspecific overlap in call design. To overcome such difficulties and to reduce workload, automated classifiers of echolocation calls have become popular, but their performance has not been tested sufficiently in the field. We examined the absolute performance of two commercially available programs (SonoChiro and Kaleidoscope) and one freeware package (BatClassify). We recorded noise from rain and calls of seven common bat species with Pettersson real-time full spectrum detectors in Sweden. The programs could always (100%) distinguish rain from bat calls, usually (68–100%) identify bats to group (Nyctalus/Vespertilio/Eptesicus, Pipistrellus, Myotis, Plecotus, Barbastella) and usually (83–99%) recognize typical calls of some species whose echolocation pulses are structurally distinct (Pipistrellus pygmaeus, Barbastella barbastellus). Species with less characteristic echolocation calls were not identified reliably, including Vespertilio murinus (16–26%), Myotis spp. (4–93%) and Plecotus auritus (0–89%). All programs showed major although different shortcomings and the often poor performance raising serious concerns about the use of automated classifiers for identification to species level in research and surveys. We highlight the importance of validating output from automated classifiers, and restricting their use to specific situations where identification can be made with high confidence. For comparison we also present the result of a manual identification test on a random subset of the files used to test the programs. It showed a higher classification success but performances were still low for more problematic taxa.

AB - Echolocating bats are surveyed and studied acoustically with bat detectors routinely and worldwide, yet identification of species from calls often remains ambiguous or impossible due to intraspecific call variation and/or interspecific overlap in call design. To overcome such difficulties and to reduce workload, automated classifiers of echolocation calls have become popular, but their performance has not been tested sufficiently in the field. We examined the absolute performance of two commercially available programs (SonoChiro and Kaleidoscope) and one freeware package (BatClassify). We recorded noise from rain and calls of seven common bat species with Pettersson real-time full spectrum detectors in Sweden. The programs could always (100%) distinguish rain from bat calls, usually (68–100%) identify bats to group (Nyctalus/Vespertilio/Eptesicus, Pipistrellus, Myotis, Plecotus, Barbastella) and usually (83–99%) recognize typical calls of some species whose echolocation pulses are structurally distinct (Pipistrellus pygmaeus, Barbastella barbastellus). Species with less characteristic echolocation calls were not identified reliably, including Vespertilio murinus (16–26%), Myotis spp. (4–93%) and Plecotus auritus (0–89%). All programs showed major although different shortcomings and the often poor performance raising serious concerns about the use of automated classifiers for identification to species level in research and surveys. We highlight the importance of validating output from automated classifiers, and restricting their use to specific situations where identification can be made with high confidence. For comparison we also present the result of a manual identification test on a random subset of the files used to test the programs. It showed a higher classification success but performances were still low for more problematic taxa.

KW - Biosonar

KW - Methodology

KW - Software

KW - Species identification

KW - Ultrasound

U2 - 10.1016/j.ecolind.2017.03.023

DO - 10.1016/j.ecolind.2017.03.023

M3 - Article

VL - 78

SP - 416

EP - 420

JO - Ecological Indicators

JF - Ecological Indicators

SN - 1470-160X

ER -