NMPA published the Review Points of CT Image-Assisted Triage and Evaluation Software for Coronavirus (COVID-19)-Infected Pneumonia on March 5, 2020. The document provides guidance to AI software manufacturers on the application scope, basic requirements, risk management, software research, clinical research, instructions and software renewal.
Keyword One: Class III
“According to the Catalog of Medical Device Classification, it is class III management category with the classification code 21-04-02”, NMPA states in the review points guideline.
Two Class III AI approvals have been granted in 2020, Keyar Medical’s fractional flow reserve derived from CT angiography (FFRCT) and Lepu Medical’s ECG analysis software, and some AI imaging companies say they have completed clinical trials and the approvals could be expected by end of 2020. Given that no lung or liver related CT products with AI functions have obtained approval, the pre-marketing process for the AI-assisted diagnostic software will be accelerated.
Keyword Two: Performance requirements
The document states “The software functions include at least abnormality identification, quantitative analysis (such as lesion volume ratio, CT value distribution, etc.), data comparison (manual or automatic), and report output. Among them, abnormality identification, which infuses AI, is used for the triage of suspected patients, and quantitative analysis and data comparison are used to assess the condition of patients.”
Generally, abnormality identification infuses AI while quantitative analysis and data comparison do not. It means that the intended AI software not only needs to provide high-accuracy judgmental results, but also needs to have functions of data analysis and processing.
The quantitative analysis used to be difficult to perform with traditional analysis software. With considerable number of patients, the coronavirus outbreak provides an opportunity for AI companies to conduct clinical trials.
Keyword three: Data requirements
The data requirements can be simplified as following:
- The training data should not be less than 2000 CT images of coronavirus patients; at least 3 medical institutions, including at least 1 medical institution in a severe epidemic area; CT images of both early stage and advanced stage patients shall be provided.
- Distribution information of CT image data should be provided, based on the characteristics of the population (such as gender, age), imaging stage (early stage, advanced stage, severe stage), data source institutions, CT equipment (such as manufacturer, layer thickness), etc.
- An analysis report of the factors affecting the performance of the algorithm should be provided, based on CT equipment, imaging phases and similar symptoms.
Keyword four: Clinical design
The document points out that manufactures can adopt a single set of target value in clinical design, taking the inherent diagnostic accuracy indicators such as sensitivity and specificity suggested by triage as the main evaluation indicators.
Manufacturers should elaborate on the selection, construction method and reasons of gold standards. The gold standard criteria should be based on clinically determined results, including epidemiological history and clinical manifestations (such as imaging features), or through expert reading team.
In addition, the document proposes a number of indicators for sample size estimation that need to be clarified.
For our post on AI-aided software guideline, please click HERE.