Performance Reports & Analysis IFU042(02)

What is Performance Reports Management?

The Performance Reports Management page is your central hub for data analysis in the RARE portal. It transforms raw assay data into actionable regulatory and scientific insights by evaluating how your Qualitative Classifiers and Quantitative Batch Configurations perform against a known Ground Truth.

By centralizing these reports, RARE ensures full traceability between your raw datasets, the specific configurations applied to them, and the resulting performance outcomes.

Why use Performance Reports?

A Performance Report acts as a "snapshot in time," capturing exactly how a specific version of your interpreter performed on a specific set of samples. This is essential for:

• Method Validation: Providing the statistical evidence needed for regulatory filings.

• Configuration Tuning: Comparing different threshold settings or calibration curves to find the optimal balance of sensitivity and precision.

• Quality Control: Monitoring batch-to-batch consistency and identifying potential manufacturing drift.

Key concepts

• Performance Report – A generated document or view that summarizes diagnostic and analytical metrics for a specific dataset and configuration.

• Dataset – The collection of original results and images used as the input for the report.

• Qualitative Classifier – The interpretation configuration (rules and thresholds) applied to determine qualitative results.

• Quantitative Batch ID – The specific batch configuration (calibration parameters) used for quantitative analysis.

• Ground Truth – The reference labels (e.g., known positive/negative) used to calculate the accuracy of the interpreter's results.

Managing your Reports

Accessing Performance Reports

You can find your reports by navigating to the sidebar in the RARE web portal:
Sidebar → Data Analysis → Performance Reports

Understanding the report list

The main area of the page displays a tabular list of all generated reports. This list is designed to provide an immediate audit trail of your analysis history. For each report, you will see:

• "Latest Report" column group showing the most recent generation status/timestamp

• Report Name/ID: The unique identifier for the analysis.

• Description: A brief note (if provided) explaining the context of the report.

• Status (DRAFT = saved but never generated; LOCKED = generated at least once; ARCHIVED = soft-deleted)

• Version

• Created: The exact date and timestamp when the report was generated.

• Org ID

• User ID

• Datasets: The identifier(s) of the data used for the analysis.

• Classifier and Batch columns: The specific versions of the Qualitative Classifier and Quantitative Batch ID applied.

Filtering and sorting reports

To help you find specific analyses in a growing list, you can use the built-in filters.

Filter

You can narrow down the list using the following criteria, among others:

• Report Date: Select a "From" and "To" date range to see reports created within a specific period.

• Qualitative Classifier: Filter by the specific interpretation configuration ID.

• Quantitative Batch: Filter by the specific batch or calibration configuration ID.

Sort

The list can be sorted by clicking the column headers (e.g., sorting by Created date to see the most recent analyses at the top).

Actions available

Each row in the list includes an Actions control (typically a three-dot menu or button) that allows you to:

• Click the Create New Performance Report button at the top of the page to create a new configuration.

• View Report Configuration: Opens the detailed report configuration view to examine metrics, charts, and result breakdowns.

• Export list: Downloads the report the report list.

Creating a New Performance Report

Step 1 - Describe Report

Start by giving your report a clear, descriptive name and a short explanation of its purpose. This helps with traceability and future audits.

• Enter a clear Name (e.g., "QC Panel Diagnostic Mar 2026").

• Add a Description to explain the purpose or context.

Step 2 - Select Dataset

Choose the source data for your report. The system displays all datasets that have been through interpreter reprocessing and are eligible for performance analysis.

• Browse the list of available datasets.

• Use the Filter options to narrow down by date, assay type, or operator.

• Click to select your target dataset. Once selected, it will be highlighted, and other options will be hidden.

• If you need to change your mind, click the Change button to return to the full list.

Step 3 - Dataset Overview

Before proceeding, review the summary of your selected dataset to ensure it contains the expected data.

• Check the Total Results count and the breakdown of assay types.

• Verify the Configurations (Batch and Classifier) that were used to generate these results.

• Confirm that the Ground Truth status is "Applied" for the results you wish to analyze.

Step 4 - Ground Truth Mapping

To calculate performance metrics like sensitivity and specificity, RARE needs to know which result labels correspond to your reference "Ground Truth." This mapping is flexible, allowing you to perform qualitative analysis on each specific threshold within your classifier.

Configuration Steps

• Single strip, single test line: Map the interpreter's qualitative bins (e.g., "Positive", "Negative") to the Ground Truth categories.

• Multiple strips / multiple test lines: For complex assays, ensure each test line is correctly mapped to its respective reference standard.

• The system will validate that the mapping is complete before allowing you to move to the next step.

Mapping Multi-Bin Classifiers

If your classifier uses multiple bins (e.g., Low, Medium, and High), you can generate reports for different clinical scenarios by grouping these bins against your Ground Truth. This is achieved by mapping the thresholds between the bins:

• Scenario A (Low vs. Others): You can map the first threshold so that Low is compared against a combined group of Medium + High.

• Scenario B (High vs. Others): You can map the second threshold so that Low + Medium are grouped together and compared against High.

Step 5 - Report Sections

Customize your report by selecting the specific analysis modules you need. You can toggle these on or off based on your validation requirements:

• Overview & Provenance: Includes metadata about the dataset, configurations, and the environment in which the report was generated.

• Qualitative Diagnostic Metrics: Generates contingency tables (True Positives, False Negatives, etc.) and calculates Sensitivity, Specificity, and Accuracy.

• Threshold Analysis: Includes ROC curves and threshold optimization tables to help you find the best cut-off points for your assay.

• Analytical Performance: Focuses on quantitative precision, bias, and recovery metrics.

• Batch-to-Batch Variability: Compares performance across different reagent lots or manufacturing batches if present in the dataset.

Step 6 - Review & Save

Review your configuration summary one last time.

• Check that the report name, selected dataset, and chosen sections are correct.

• Click Save Configuration to store these settings. This allows you to return to this specific setup later or share the configuration with colleagues.

Step 7 - Generate the report

After saving, the Generate Report button will appear at the bottom of the page.

• Press Generate Report to start the calculation engine.

• The system will process the data and create the final report view. Depending on the dataset size, this may take a few moments.

Interpreting Report Content

Diagnostic performance is considered for each test line, across multiple assay strips if present. The report is organized into modular chapters, providing both visual charts and detailed statistical tables to support your findings.

Every report is structured to ensure Full Traceability. The "Provenance" section identifies exactly which dataset and configuration versions were used, ensuring your conclusions are always backed by a clear audit trail.

The Four Pillars of RARE Analysis

1. Qualitative Diagnostic Metrics: The "Pass/Fail" performance of your assay (Sensitivity, Specificity, etc.).

2. Threshold Analysis: Advanced tools like ROC curves and Decision Plots to optimize your cut-off points.

3. Analytical Performance: Quantitative metrics including linearity, regression, and recovery.

4. Batch-to-Batch Variability: Comparative analysis to ensure robustness across different manufacturing lots.

Export PDF

Once the report is generated, you can take it offline for your records.

• Press Export PDF at either the top or foot of the page.

• The exported file includes all selected charts, tables, and provenance data, formatted for regulatory filing or internal sharing.

Overview & Provenance

The Overview & Provenance section serves as the definitive "technical passport" for your report, ensuring that every analysis is fully reproducible and audit-ready. By capturing a snapshot of the environment at the moment of generation, it eliminates ambiguity regarding which settings produced which results.

Based on the system configuration, this section details:

• Report Metadata: Displays the unique Report ID, the name of the operator who generated it, and the exact completion timestamp.

• Software Lineage: Lists the specific versions of the RARE Portal and Interpreter engine used, which is critical for maintaining compliance across software updates.

• Configuration Traceability: Explicitly identifies the Qualitative Classifier and Quantitative Batch ID (including their specific version numbers) applied to the data.

• Dataset Context: Summarizes the source dataset, including the total number of results analyzed and the specific assay types involved.

This level of detail ensures that if you ever need to defend a validation study or investigate a QC outlier, you have a clear, unalterable record of the logic and data used.

Qualitative Diagnostic Metrics

The Qualitative Diagnostic Metrics section provides the statistical foundation for your assay’s clinical performance. By comparing the interpreter’s results against your established Ground Truth, RARE generates a clear, objective view of how accurately the system identifies positive and negative samples.

This section is centered around a Contingency Table (or Confusion Matrix), which provides a visual breakdown of:

• True Positives (TP): Correctly identified positive samples.

• True Negatives (TN): Correctly identified negative samples.

• False Positives (FP): Negative samples incorrectly labeled as positive.

• False Negatives (FN): Positive samples incorrectly labeled as negative.

To ensure your performance reports meet the highest standards of transparency and regulatory rigor, RARE uses the following standard statistical definitions. These metrics are calculated by comparing the interpreter's results against the assigned Ground Truth.

Sensitivity (Positive Percent Agreement - PPA)

The proportion of actual positive samples that are correctly identified by the interpreter. It measures the assay's ability to avoid false negatives.

Specificity (Negative Percent Agreement - NPA)

The proportion of actual negative samples that are correctly identified by the interpreter. It measures the assay's ability to avoid false positives.

Positive Predictive Value (PPV)

The probability that a sample identified as positive by the interpreter is truly positive. This indicates the reliability of a positive result.

Negative Predictive Value (NPV)

The probability that a sample identified as negative by the interpreter is truly negative. This indicates the reliability of a negative result.

Overall Accuracy

The proportion of all samples (both positive and negative) that were correctly identified by the interpreter.

Threshold Analysis

The Threshold Analysis section provides the advanced diagnostic tools needed to optimize your assay’s performance and justify your chosen cut-off points.

This section moves beyond static metrics to show you how your interpretation logic behaves across a spectrum of possible values, which is essential for both initial calibration and retrospective performance tuning.

Based on the analysis modules, this section includes:

• ROC (Receiver Operating Characteristic) Curves: A visual representation of the trade-off between Sensitivity and Specificity. The curve plots the True Positive Rate against the False Positive Rate at various threshold settings. The Area Under the Curve (AUC) is also calculated, providing a single global metric for the interpreter's ability to discriminate between positive and negative samples.

• Threshold Optimization Table: A detailed breakdown showing how specific diagnostic metrics (Sensitivity, Specificity, PPV, NPV) would shift if you moved your threshold up or down. This allows you to select a cut-off that prioritizes the specific clinical needs of your assay (e.g., maximizing sensitivity for a screening test).

• Youden’s Index : To assist in identifying the mathematically "optimal" threshold, RARE calculates Youden’s Index for every point on the ROC curve. This index represents the point where the sum of Sensitivity and Specificity is maximized.
  
  

In addition to the optimization table, the Decision Plot provides a powerful visual summary of your assay's performance across the entire threshold range. This dual-axis chart plots both Sensitivity and Specificity simultaneously against the threshold value.

The Decision Plot allows you to:

• Identify the Crossover Point: Easily locate the "Equal Error Rate" where Sensitivity and Specificity intersect, representing a balanced performance profile.

• Visualize Performance Trade-offs: See at a glance how rapidly Sensitivity drops or Specificity rises as you adjust your cut-off, helping you identify "stable" regions where minor variations in signal won't drastically impact clinical results.

• Support Clinical Justification: By overlaying these two critical metrics, the plot provides a clear visual rationale for selecting a specific threshold—for example, choosing a point slightly to the left of the intersection to prioritize high Sensitivity for a screening application.

This graphical representation transforms the raw data of the optimization table into an intuitive tool for making informed, data-driven configuration decisions.

Toggle Explore threshold to drag and explore the impact of using other threshold values.

By using these tools, you can provide objective, data-driven justification for your classifier configurations, ensuring they are optimized for the best possible clinical outcome.

Analytical Performance

The Analytical Performance section focuses on the quantitative precision and reliability of your assay.

While diagnostic metrics evaluate clinical "Pass/Fail" accuracy, this section examines the underlying numerical data to ensure your interpreter is producing consistent, linear, and unbiased results across the entire reportable range.

Based on the analysis modules, this section includes:

• Linearity and Regression Analysis: This module compares the interpreter's quantitative output against the expected Ground Truth values. RARE generates a scatter plot with a Linear Regression line to visualize the relationship. Key statistics provided include the Slope, Intercept, and the Coefficient of Determination , which indicates how well the data fits the linear model, where and
  

• Bias and Recovery: This analysis determines if the system consistently over- or under-estimates concentrations. RARE calculates the Percent Recovery for each sample, providing a clear view of systematic error across different concentration levels.
  
  
  

• Precision (Repeatability): By analyzing replicates within your dataset, RARE evaluates the "closeness of agreement" between independent results. This is expressed as the Coefficient of Variation (%CV), a standardized measure of dispersion that allows you to assess the stability of your assay and interpreter across multiple runs.
  
  where is the standard deviation and is the mean of the replicates.

Batch-to-Batch Variability

he Batch-to-Batch Variability section provides the critical evidence needed to demonstrate the robustness of your interpreter across different manufacturing lots. In lateral flow production, slight variations in materials or reagent deposition can impact signal intensity; this section helps you verify that your Batch Configurations are successfully normalizing these differences to provide consistent results.

Based on the analysis modules, this section includes:

• Calibration Curve Overlay: This plot displays the mathematical fit for each batch configuration included in the report. By overlaying these curves, you can visually assess if different lots follow the same response trajectory or if there is a "shift" in signal intensity across the concentration range.

• Parameter Comparison Table: RARE extracts the key coefficients from the underlying 4PL or 5PL models (such as Slope, EC50, and Asymptotes) and presents them side-by-side. This allows you to quantify exactly how the biochemical response varies between lots.

• Residuals (Observed - Predicted) Box Plot: This diagnostic tool visualizes the distribution of error for each batch. It helps you identify if a specific lot has a higher degree of "noise" or a consistent bias compared to the others.

Understanding the Residuals Box Plot

The box plots in this section allow you to see at a glance how consistently each batch is performing. For each batch:

• The Center Line (Median): Represents the median residual. If this line is at zero, the batch is perfectly centered on the Ground Truth. If it is consistently above or below zero, it indicates a systematic bias for that specific lot.

• The Box (Interquartile Range - IQR): Represents the middle 50% of your data. A "taller" box indicates higher variability (lower precision) within that batch.

• The Whiskers: Extend to show the spread of the rest of the data.

• Outliers (Individual Dots): Any results that fall significantly outside the expected range are plotted as individual points, helping you identify specific samples that may have had physical defects or interpretation anomalies.

Batch Comparison Metrics

RARE calculates how each "Test" batch deviates from the designated Reference Batch using the following formulae:

Mean Residual (Bias):
The average difference between the measured values and the Ground Truth for the batch.

Batch-Specific Coefficient of Variation (%CV):
This metric quantifies the precision of the residuals for a specific batch, allowing you to compare the "tightness" of results against your reference lot.

By analyzing these plots and tables, you can objectively determine if a new manufacturing lot is performing within your established QC tolerances or if it requires a unique calibration update.

View / Edit an existing Performance Report Configuration

From the Report Configuration you can

• View Reports: Opens the detailed report view to examine metrics, charts, and result breakdowns.

• In order to make changes you need to save as a new version

• Generate Report: Re-runs the analysis using the same parameters to ensure consistency or update the view.

After you’ve done this

Once a report is generated:

• Review Metrics: Navigate to the "View Report" screen to see qualitative diagnostic metrics and ROC curves.

• Verify Lineage: Check the report metadata to ensure the correct versions of configurations were used.

• Data Safety: RARE never modifies your original dataset results during report generation; it creates a snapshot of the performance at that moment in time.