Skip to content

Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark

Research output: Contribution to journalArticle

Standard

Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark. / Sayers, Adrian; Crowther, Michael J.; Judge, Andrew; Whitehouse, Michael R.; Blom, Ashley W.

In: BMJ Open, Vol. 7, No. 8, e015397, 01.08.2017.

Research output: Contribution to journalArticle

Harvard

APA

Vancouver

Author

Bibtex

@article{f5648b58233a4202b14b2f0bcac59d7d,
title = "Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark",
abstract = "OBJECTIVES: The use of benchmarks to assess the performance of implants such as those used in arthroplasty surgery is a widespread practice. It provides surgeons, patients and regulatory authorities with the reassurance that implants used are safe and effective. However, it is not currently clear how or how many implants should be statistically compared with a benchmark to assess whether or not that implant is superior, equivalent, non-inferior or inferior to the performance benchmark of interest.We aim to describe the methods and sample size required to conduct a one-sample non-inferiority study of a medical device for the purposes of benchmarking.DESIGN: Simulation study.SETTING: Simulation study of a national register of medical devices.METHODS: We simulated data, with and without a non-informative competing risk, to represent an arthroplasty population and describe three methods of analysis (z-test, 1-Kaplan-Meier and competing risks) commonly used in surgical research.PRIMARY OUTCOME: We evaluate the performance of each method using power, bias, root-mean-square error, coverage and CI width.RESULTS: 1-Kaplan-Meier provides an unbiased estimate of implant net failure, which can be used to assess if a surgical device is non-inferior to an external benchmark. Small non-inferiority margins require significantly more individuals to be at risk compared with current benchmarking standards.CONCLUSION: A non-inferiority testing paradigm provides a useful framework for determining if an implant meets the required performance defined by an external benchmark. Current contemporary benchmarking standards have limited power to detect non-inferiority, and substantially larger samples sizes, in excess of 3200 procedures, are required to achieve a power greater than 60{\%}. It is clear when benchmarking implant performance, net failure estimated using 1-KM is preferential to crude failure estimated by competing risk models.",
keywords = "benchmark, competing risks, medical implant, non-inferiority, sample size",
author = "Adrian Sayers and Crowther, {Michael J.} and Andrew Judge and Whitehouse, {Michael R.} and Blom, {Ashley W.}",
year = "2017",
month = "8",
day = "1",
doi = "10.1136/bmjopen-2016-015397",
language = "English",
volume = "7",
journal = "BMJ Open",
issn = "2044-6055",
publisher = "BMJ Publishing Group",
number = "8",

}

RIS - suitable for import to EndNote

TY - JOUR

T1 - Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark

AU - Sayers, Adrian

AU - Crowther, Michael J.

AU - Judge, Andrew

AU - Whitehouse, Michael R.

AU - Blom, Ashley W.

PY - 2017/8/1

Y1 - 2017/8/1

N2 - OBJECTIVES: The use of benchmarks to assess the performance of implants such as those used in arthroplasty surgery is a widespread practice. It provides surgeons, patients and regulatory authorities with the reassurance that implants used are safe and effective. However, it is not currently clear how or how many implants should be statistically compared with a benchmark to assess whether or not that implant is superior, equivalent, non-inferior or inferior to the performance benchmark of interest.We aim to describe the methods and sample size required to conduct a one-sample non-inferiority study of a medical device for the purposes of benchmarking.DESIGN: Simulation study.SETTING: Simulation study of a national register of medical devices.METHODS: We simulated data, with and without a non-informative competing risk, to represent an arthroplasty population and describe three methods of analysis (z-test, 1-Kaplan-Meier and competing risks) commonly used in surgical research.PRIMARY OUTCOME: We evaluate the performance of each method using power, bias, root-mean-square error, coverage and CI width.RESULTS: 1-Kaplan-Meier provides an unbiased estimate of implant net failure, which can be used to assess if a surgical device is non-inferior to an external benchmark. Small non-inferiority margins require significantly more individuals to be at risk compared with current benchmarking standards.CONCLUSION: A non-inferiority testing paradigm provides a useful framework for determining if an implant meets the required performance defined by an external benchmark. Current contemporary benchmarking standards have limited power to detect non-inferiority, and substantially larger samples sizes, in excess of 3200 procedures, are required to achieve a power greater than 60%. It is clear when benchmarking implant performance, net failure estimated using 1-KM is preferential to crude failure estimated by competing risk models.

AB - OBJECTIVES: The use of benchmarks to assess the performance of implants such as those used in arthroplasty surgery is a widespread practice. It provides surgeons, patients and regulatory authorities with the reassurance that implants used are safe and effective. However, it is not currently clear how or how many implants should be statistically compared with a benchmark to assess whether or not that implant is superior, equivalent, non-inferior or inferior to the performance benchmark of interest.We aim to describe the methods and sample size required to conduct a one-sample non-inferiority study of a medical device for the purposes of benchmarking.DESIGN: Simulation study.SETTING: Simulation study of a national register of medical devices.METHODS: We simulated data, with and without a non-informative competing risk, to represent an arthroplasty population and describe three methods of analysis (z-test, 1-Kaplan-Meier and competing risks) commonly used in surgical research.PRIMARY OUTCOME: We evaluate the performance of each method using power, bias, root-mean-square error, coverage and CI width.RESULTS: 1-Kaplan-Meier provides an unbiased estimate of implant net failure, which can be used to assess if a surgical device is non-inferior to an external benchmark. Small non-inferiority margins require significantly more individuals to be at risk compared with current benchmarking standards.CONCLUSION: A non-inferiority testing paradigm provides a useful framework for determining if an implant meets the required performance defined by an external benchmark. Current contemporary benchmarking standards have limited power to detect non-inferiority, and substantially larger samples sizes, in excess of 3200 procedures, are required to achieve a power greater than 60%. It is clear when benchmarking implant performance, net failure estimated using 1-KM is preferential to crude failure estimated by competing risk models.

KW - benchmark

KW - competing risks

KW - medical implant

KW - non-inferiority

KW - sample size

UR - http://www.scopus.com/inward/record.url?scp=85031672876&partnerID=8YFLogxK

U2 - 10.1136/bmjopen-2016-015397

DO - 10.1136/bmjopen-2016-015397

M3 - Article

VL - 7

JO - BMJ Open

JF - BMJ Open

SN - 2044-6055

IS - 8

M1 - e015397

ER -