Unity Imaging Collaborative

Dataset for model development

The dataset for model development is split into three sets:

• Training set - set of data used for directly training the neural network (~70% of the dataset)
• Tuning set - set of data used to observe the performance of the network (progress monitoring) and fine-tune the hyper-parameters (~15% of the dataset)
• Internal validation test set - set of data used to test the final performance of the network (~15% of the dataset)

The Unity Imaging dataset for model development consists of single frames from echocardiograms labelled by one of the experts from the unity collaborative. The image may have had the labels refined, reviewed, and corrected by multiple experts, but only the final set of labels are used. The labels consist of points (such as the endocardial apex), or curves (such as the endocardium).

Dataset for model validation

The dataset for model validation forms a single set:

• External validation test set - set of data used for robust external validation

A specific dataset for model validation is constructed for each task / publication (such as PLAX measurements or EF).

The Unity Imaging datasets for model validation consist of out-of-sample data. The data is derived from different studies than the development dataset, collected from different years than the develpment dataset. In future, once ethical approvals in place this data will come from different institutions.

In contrast to the dataset for model development, the dataset for model validation will have labels derived from the consensus of multiple (currently up to 11) experts. Moreover, it will have carefully constructed so that derived parameters such as EF can be calculated (by matching the systolic and diastolic frames). Furthermore, the dataset for model validation may have additional labels, such as the values produced from proprietary software, or visual "eyeball" assessment.

Unlike the dataset for model development, the labels for the datasets for model validation are derived from the consensus of multiple (currently up to 11) experts independently labelling the data. Furthermore, as in clinical practice we are interested in derived parameters (such as ejection fraction, or global longitudinal strain), the datasets for model validation are constructed for specific projects with that in mind. E.g. for ejection fraction, the matching systolic and diastolic images may be used, labelled, and the derived parameter computed from the consensus of expert labels and AI prediction and compared.