RARE Documentation

Performance Reports & Analysis IFU042(04)

Novarum DX Ltd
Instructions for Use / Guides

Document ID: IFU042
Revision: 04
Released: 2026-11-05

Use Performance Reports to create, view, export and interpret assay performance evidence in the RARE web portal. Reports now include the original pooled diagnostic and analytical sections plus population accounting, phone-to-phone variability, phone-effect margin comparisons, environmental capture robustness and temporal usability outputs.

Performance Reports Management

The Performance Reports Management page is the Data Analysis entry point for generated Performance Reports in the RARE portal. It helps LF manufacturing, quality and laboratory science users review how an assay configuration performs against known reference data, and provides the audit trail needed to understand exactly which dataset, classifier, batch configuration, software version and report configuration produced a given result.

The generated report layout uses a fixed 11-section structure. Sections that are switched off keep their section number in the contents, while included sections are printed in full.

Screenshot 2026-11-04 at 11.14.28.png
Performance Reports Management list showing the Create New Performance Report button, filters, report status and row actions.

Why use Performance Reports?

A Performance Report acts as a snapshot in time. It records how a specific dataset performed with a specific version of the qualitative classifier, quantitative batch configuration and report settings.

This is useful for:

  • Method validation and design history evidence: capturing the diagnostic and analytical metrics used to support performance claims.

  • Configuration tuning: comparing thresholds, classifiers, calibration curves and report limits before fixing an operating configuration.

  • Manufacturing and QC review: checking lot-to-lot consistency, concentration-level precision and population balance across phone models or batches.

  • Device-matrix review: exploring whether supported phone models behave similarly or whether a phone-specific effect should be investigated further.

  • Data science follow-up: using exported tabular data to investigate outliers, sparse cells, environmental telemetry or concentration-level trends outside the report view.

Key concepts

  • Performance Report – A generated report view or PDF that summarises diagnostic, analytical, device, environmental and usability metrics for a selected dataset and configuration.

  • Dataset – The set of result records used as the report input. Report selectors exclude datacollector-mode datasets that are not compatible with Performance Reports.

  • Qualitative Classifier – The interpretation configuration used to convert measured signal into qualitative calls such as positive or negative.

  • Quantitative Batch Configuration – The calibration configuration used to convert measured signal into quantitative values or to fit standard curves.

  • Ground Truth – The reference label or concentration used to evaluate the report output. Qualitative sections compare classifier calls with ground-truth classes; quantitative sections compare signal or fitted values with known concentration levels.

  • Phone model – The device model recorded against the result. The Portal uses this to stratify diagnostic metrics, calibration behaviour, environmental associations and usability/capture dynamics.

  • Telemetry – Capture information such as lux, pitch, yaw and PMF story frame timing. The environmental and temporal sections use usable telemetry where available.

Managing reports

Accessing Performance Reports

Navigate to:

Sidebar → Data Analysis → Performance Reports

What you can see and do depends on your user role, organisation access and subscription level.

Understanding the report list

The report list provides an audit trail of generated and saved report configurations. Each row gives enough metadata to identify the report and reconstruct the analytical context. Depending on the current view, expect fields such as:

  • Report name or ID: the identifier used to find the analysis later.

  • Description: optional study context, for example the assay, run, validation purpose or device matrix.

  • Latest report status and timestamp: the most recent generation state and time.

  • Status: for example DRAFT for saved but not generated, LOCKED for generated at least once, or ARCHIVED for soft-deleted records.

  • Version: the report configuration version.

  • Created: the date and time the report configuration or generated report was created.

  • Organisation and user identifiers: useful for audit traceability.

  • Dataset identifiers: the dataset or combined dataset used for the analysis.

  • Classifier and batch configuration identifiers: the qualitative and quantitative configuration versions applied during generation.

Screenshot 2026-11-04 at 11.09.34.png
Report list with generated report rows, showing the latest generation status, dataset, classifier and batch columns.


Filtering and sorting reports

Use filters to find the report you need in a long audit history. Common filters include:

  • Report date: apply a from/to date range to limit the list to reports generated within a specific period.

  • Qualitative classifier: filter by interpretation configuration ID.

  • Quantitative batch: filter by batch or calibration configuration ID.

You can also sort by column headers, for example sorting by created date to bring the most recent analyses to the top.

Actions available

Each row includes an Actions control. You can:

  • Create New Performance Report: start a new report configuration from the page header.

  • View report configuration: inspect the saved settings, selected dataset, classifier, batch group, report sections, limits and version.

Bulk Actions also allow you to

  • Export to Excel or CSV: download the report list metadata for administrative review.

Creating a new Performance Report

Prerequisites

Before you begin, make sure you have:

  • An eligible interpreter reprocessing dataset: the dataset must contain finalised results generated using the relevant classifier and/or batch configuration.

  • Appropriate permissions: you must be allowed to create and generate reports for the organisation.

  • Ground truth: qualitative ground-truth classes and/or quantitative concentration levels must be present for the analyses you intend to run.

  • Compatible configurations: the dataset, classifier and batch configuration must be compatible. The portal uses a sequential report configuration flow so that downstream options update when an upstream choice changes.

  • Phone and telemetry coverage: phone variability sections require enough known phone-model data; environmental and temporal sections require usable capture telemetry.

Step 1 – Describe report

Give the report a clear name and description. Good names include the assay, dataset or study purpose, and any important device or batch scope.

  • Use a clear Name, for example Hourglass standard curve; iPhone 14 and iPhone 17 Pro.

  • Add a Description explaining why the report is being generated, for example validation, troubleshooting, QC lot comparison or phone-matrix review.

Screenshot 2026-11-04 at 11.21.15.png
Describe Report step showing the report name and description fields.


Use consistent naming conventions so that later readers can quickly distinguish validation, QC, investigation and regression reports.

Step 2 – Select dataset

Choose the source dataset. We filter the report dataset selector to eligible non-datacollector-mode datasets and then load the classifier choices available for that dataset.

  • Review dataset identifiers, run descriptions, creation dates and result counts.

  • Select the dataset that matches the assay and study population you want to analyse.

  • If the dataset combines multiple runs or phones, check that this is intentional and documented in the report description.

Screenshot 2026-11-04 at 11.22.28.png
Dataset selector showing eligible datasets and a selected combined dataset.

Step 3 – Dataset overview

After you select the dataset, the dataset overview will show coverage and resport sections available. Confirm result counts, assay/test-line coverage, ground-truth availability and configuration compatibility.

  • Total results: check that the count matches the intended study population.

  • Phone model coverage: confirm whether results include one or multiple known phone models.

  • Verify the Configurations (Batch and Classifier) that were used to generate these results.

  • Ground truth: verify that the required qualitative classes and quantitative concentration levels are present.

Screenshot 2026-11-04 at 11.24.27.png
Dataset overview step showing available and disabled classifier options.

Step 4 – Ground Truth Mapping

To calculate performance metrics like sensitivity and specificity, RARE needs to know which result labels correspond to your reference "Ground Truth." This mapping is flexible, allowing you to perform qualitative analysis on each specific threshold within your classifier.

Screenshot 2026-11-04 at 11.29.07.png
Ground-truth mapping screen with a multi-bin classifier mapped to positive and negative ground-truth groups.

Configuration Steps

  • Single strip, single test line: Map the interpreter's qualitative bins (e.g., "Positive", "Negative") to the Ground Truth categories.

  • Multiple strips / multiple test lines: For complex assays, ensure each test line is correctly mapped to its respective reference standard.

  • The system will validate that the mapping is complete before allowing you to move to the next step.

Mapping Multi-Bin Classifiers

If your classifier uses multiple bins (e.g., Low, Medium, and High), you can generate reports for different clinical scenarios by grouping these bins against your Ground Truth. This is achieved by mapping the thresholds between the bins:

  • Scenario A (Low vs. Others): You can map the first threshold so that Low is compared against a combined group of Medium + High.

  • Scenario B (High vs. Others): You can map the second threshold so that Low + Medium are grouped together and compared against High.

Step 5 – Choose report sections and limits

Customize your report by selecting the specific analysis modules you need. You can toggle these on or off based on your validation requirements:

Section

Purpose

Main question answered

1 · Provenance & Scope

Records report metadata, dataset, configuration versions and environment.

What exactly was analysed, by whom, when and with which software/configuration versions?

1.1 · Population Accounting

Shows the data-quality funnel, missingness matrix and design-crossing matrix.

Which records were captured, selected, valid and analysed, and where are the gaps?

2 · Overview

Summarises headline findings and module status.

Which sections need attention before reading detailed tables?

3 · Qualitative Diagnostic Metrics

Reports pooled confusion matrices and diagnostic proportions.

How well does the classifier match ground truth overall?

4 · Threshold Analysis

Shows ROC/AUC, threshold optimisation and decision plots.

How would changing the operating threshold affect sensitivity and specificity?

5 · Analytical Performance

Reports standard-curve, LoD/LoQ, precision, bias and recovery outputs where applicable.

How consistent and accurate is the quantitative response across concentration?

6 · Batch-to-Batch Variability

Compares calibration curves, parameters and residuals across batches.

Set a reference batch selected for Analytical Performance.

Do manufacturing lots behave similarly enough for the intended use?

7 · Phone Variability: Qualitative Performance and Threshold Stability

Stratifies qualitative metrics, ROC/AUC and Youden-optimal thresholds by phone model.

Does qualitative classification performance differ between phone models?

8 · Phone Variability (Quantitative)

Fits phone-specific calibration models and reports concentration-level precision, residuals and variance components where estimable.

Does quantitative signal or calibration behaviour vary by phone model?

9 · Phone Effect & Equivalence

Reports phone-model effect estimates, 95% confidence intervals and comparisons with saved margins.

Are estimated phone effects within the configured numerical margins, and what evidence class is assigned?

10 · Environmental Factors and Capture Robustness

Explores lux, pitch and yaw associations with signal and precision.

Do capture conditions show patterns that merit investigation?

11 · Temporal Capture Dynamics and Usability

Shows PMF story timing, frame state, orientation and lux over capture time.

Were captures stable over time, and are there usability/capture dynamics to review?

The Portal supports a presentation density setting, for example a compact executive/triage view or a fuller regulatory/DHR view with expanded tables and matrices.

Step 6 – Review, save and generate

Review the configuration summary, then save and generate the report.

  • Check the report name, dataset, classifier, batch group, ground-truth mapping, selected sections, density and limits.

  • Click Save Configuration to store the settings and create a versioned configuration.

  • Click Generate Report to start the calculation engine.

  • For large datasets or reports with phone-specific fitting and temporal plots, allow additional processing time.

Screenshot 2026-11-04 at 11.37.22.png
Review and Save step showing the final configuration summary and Generate Report button.

Interpreting a generated report

A generated report should be read from scope to conclusion:

first confirm what data was included, then review population completeness, then interpret each analytical module. Avoid reading a single chart without checking denominators, exclusions, unavailable states and section limitations.

Important interpretation rule: A visible plot or a green-looking status is not automatically a validation conclusion.

For phone variability sections, PASS/BREACH-style conclusions require explicit numerical limits configured before report generation. Environmental outputs are exploratory, and phone-effect equivalence outputs have specific limitations described below.

Report contents and PDF export

The PDF export is an expanded, paginated view of the generated report with running headers and a table of contents.

  • Use Export PDF from the report view to create an offline record for review, sharing or filing.

  • Sections switched off in the report configuration retain their number and are listed as not included.

  • Sections switched on but not computable report unavailable or not-estimable states rather than silently omitting the problem.

  • Temporal plots in PDF may reflect the active report view selection and may use reproducible sampling for dense frame data; check the reported counts and sampling disclosures.

Screenshot 2026-11-04 at 11.41.54.png
Exported PDF contents page showing the fixed 11-section structure.

Using exported XLS or other tabular exports for further analysis

When you need deeper analysis than the report view provides, export the underlying or summary tables where available. In a spreadsheet or statistics package, useful follow-up checks include:

  • Reconcile denominators: compare section-level counts with the population funnel and missingness matrix before drawing conclusions.

  • Filter by phone, batch and concentration: inspect whether apparent phone differences are really caused by uneven concentration or batch distribution.

  • Investigate sparse cells: treat phone × batch × concentration cells with fewer than three observations as exploratory, not definitive.

  • Plot residuals and CV by concentration: a global CV can hide concentration-specific instability near LoD or at the top of the range.

  • Review telemetry coverage: environmental conclusions are weak if lux/orientation data are missing for many records or concentrated in only one phone or batch.

  • Keep identifiers: retain report ID, dataset ID, result IDs, phone model, batch ID, classifier version and configuration version in any downstream workbook so that analyses remain traceable.

Report section guide

1 · Provenance & Scope

This section is the report’s technical passport. It records the organisation, generated-by user, generated-on timestamp, RARE Portal version, dataset scope, test-line scope, phone-model scope, profile/configuration identity and calculation timestamp.

Use it to answer:

  • Was the report generated in the expected environment, for example RARE Portal v3.1.0?

  • Does the dataset ID match the validation or QC study you intended to review?

  • Which phone models, test lines, classifier split and profile version were included?

  • Is the calculation timestamp appropriate, or could the dataset have changed between selection and generation?

Screenshot 2026-11-04 at 11.51.20.png
Provenance & Scope page showing dataset, phone models, profile version and calculated-on timestamp.


1.1 · Population Accounting

Population Accounting was added to make the analysed population auditable.

It includes:

  • Data-quality funnel: Captured → In the classifier and batch selection → Ground-truth linked → Technically valid → Analysed.

  • Exclusion reasons: selection exclusions, missing ground truth, invalid captures and missing analysis signal.

  • Missing result documents: represented separately as not_found, not as an ordinary exclusion reason.

  • Missingness matrix: missing counts, percentages and denominators for fields such as signals, ground truth, image, capture status, lux and orientation.

  • Design-crossing matrix: phone × batch × concentration or class counts, including zero-count cells and sparse-cell flags.

How to interpret it: Start here before reading any performance metric. A high sensitivity value is less persuasive if many positive ground-truth records were excluded. A phone comparison is weak if one phone appears only in one batch or lacks shared concentration levels with the other phone.

Spreadsheet follow-up: Pivot the exported data by phone model, batch and concentration/class. Look for empty or sparse combinations and confirm whether every section uses the same denominator for the same filter selection.

Screenshot 2026-11-04 at 11.52.02.png
Population funnel and phone × batch × concentration matrix with sparse cells highlighted.


2 · Overview

The Overview gives a high-level summary of report findings and section states. Use it for triage, not as a replacement for detailed section review.

How to interpret it: Treat alert, review, unavailable or not-estimable states as signposts. They tell you which module needs deeper inspection, such as missing telemetry, insufficient ground truth, sparse phone comparisons or a failed fit.

Screenshot 2026-11-04 at 11.49.04.png
Overview KPI strip and module findings matrix.

3 · Qualitative Diagnostic Metrics

This section compares classifier calls with ground-truth classes for the selected test line and classifier split.

These equations evaluate the interpreter’s qualitative classifications against assigned Ground Truth reference classes (TP = True Positive, TN = True Negative, FP = False Positive, FN = False Negative).

Sensitivity (Positive Percent Agreement - PPA)

The proportion of actual positive samples that are correctly identified by the interpreter. It measures the assay's ability to avoid false negatives.

Specificity (Negative Percent Agreement - NPA)

The proportion of actual negative samples that are correctly identified by the interpreter. It measures the assay's ability to avoid false positives.

Positive Predictive Value (PPV)

The probability that a sample identified as positive by the interpreter is truly positive. This indicates the reliability of a positive result.

Negative Predictive Value (NPV)

The probability that a sample identified as negative by the interpreter is truly negative. This indicates the reliability of a negative result.

Overall Accuracy

The proportion of all samples (both positive and negative) that were correctly identified by the interpreter.

How to interpret it: For screening-style assays, false negatives may be the critical risk; for confirmatory use, false positives may dominate. Always read the denominator and confidence intervals with the point estimate. A metric based on very few positives or negatives can look impressive but be statistically fragile.

Spreadsheet follow-up: Recalculate metrics by site, run, batch, phone or concentration band to see whether the pooled result masks a subgroup problem.


Screenshot 2026-11-04 at 11.57.07.png
Confusion matrix with sensitivity, specificity, PPV, NPV and denominator counts.


4 · Threshold Analysis

Threshold Analysis shows how classifier performance changes when the decision threshold moves. It typically includes ROC curves, AUC, threshold optimisation, Youden’s Index and decision plots.

ROC (Receiver Operating Characteristic) Curves

A visual representation of the trade-off between Sensitivity and Specificity. The curve plots the True Positive Rate against the False Positive Rate at various threshold settings. The Area Under the Curve (AUC) is also calculated, providing a single global metric for the interpreter's ability to discriminate between positive and negative samples.

Threshold Optimization Table

A detailed breakdown showing how specific diagnostic metrics (Sensitivity, Specificity, PPV, NPV) would shift if you moved your threshold up or down. This allows you to select a cut-off that prioritizes the specific clinical needs of your assay (e.g., maximizing sensitivity for a screening test).

Youden’s Index

To assist in identifying the mathematically "optimal" threshold, RARE calculates Youden’s Index for every point on the ROC curve. This index represents the point where the sum of Sensitivity and Specificity is maximized.

Decision Plot

In addition to the optimization table, the Decision Plot provides a powerful visual summary of your assay's performance across the entire threshold range. This dual-axis chart plots both Sensitivity and Specificity simultaneously against the threshold value.

The Decision Plot allows you to:

  • Identify the Crossover Point: Easily locate the "Equal Error Rate" where Sensitivity and Specificity intersect, representing a balanced performance profile.

  • Visualize Performance Trade-offs: See at a glance how rapidly Sensitivity drops or Specificity rises as you adjust your cut-off, helping you identify "stable" regions where minor variations in signal won't drastically impact clinical results.

  • Support Clinical Justification: By overlaying these two critical metrics, the plot provides a clear visual rationale for selecting a specific threshold—for example, choosing a point slightly to the left of the intersection to prioritize high Sensitivity for a screening application.

This graphical representation transforms the raw data of the optimization table into an intuitive tool for making informed, data-driven configuration decisions.

How to interpret it: The mathematical optimum is not always the clinically or operationally best threshold. Choose thresholds in the context of the assay’s intended use, risk tolerance, manufacturing variability and any phone-specific instability shown in section 7.

Spreadsheet follow-up: Export threshold tables and mark candidate thresholds that preserve acceptable sensitivity/specificity over a stable range rather than at a single sharp peak.


Screenshot 2026-11-04 at 12.01.09.png
ROC curve and decision plot with chosen operating threshold and Youden-optimal threshold marked.


5 · Analytical Performance

Analytical Performance evaluates the quantitative response. Depending on the configuration and dataset, it can include standard-curve fit, LoD, LoQ, linearity/regression, bias, recovery and precision.

  • LoD: the lowest signal or concentration distinguishable from background using the configured statistical approach.

  • LoQ: the lowest concentration that can be quantified with acceptable precision and accuracy.

Linearity and Regression Analysis

This module compares the interpreter's quantitative output against the expected Ground Truth values. RARE generates a scatter plot with a Linear Regression line to visualize the relationship. Key statistics provided include the Slope, Intercept, and the Coefficient of Determination , which indicates how well the data fits the linear model, where and

Bias and Recovery

This analysis determines if the system consistently over- or under-estimates concentrations. RARE calculates the Percent Recovery for each sample, providing a clear view of systematic error across different concentration levels.

Precision (Repeatability)

By analyzing replicates within your dataset, RARE evaluates the "closeness of agreement" between independent results. This is expressed as the Coefficient of Variation (%CV), a standardized measure of dispersion that allows you to assess the stability of your assay and interpreter across multiple runs.

where is the standard deviation and is the mean of the replicates.


How to interpret it: Focus on concentration-level behaviour. A good overall regression can still hide poor precision near the decision threshold or at low concentrations. For LF assays, inspect whether the calibration curve is plausible across the useful range and whether residuals show systematic structure.

Spreadsheet follow-up: Plot residuals by concentration and batch. Compare CV% near the clinical threshold with CV% at high concentrations; the threshold-adjacent region is often more important for classification reliability.


Screenshot 2026-11-04 at 12.04.11.png
Standard curve with observed points, fitted model, residual summary and concentration-level precision table.


6 · Batch-to-Batch Variability

Batch-to-Batch Variability section provides the critical evidence needed to demonstrate the robustness of your interpreter across different manufacturing lots. In lateral flow production, slight variations in materials or reagent deposition can impact signal intensity; This section compares calibration behaviour across manufacturing lots or batch configurations. It can include curve overlays, parameter comparison tables and residual distributions.

Based on the analysis modules, this section includes:

  • Calibration Curve Overlay: This plot displays the mathematical fit for each batch configuration included in the report. By overlaying these curves, you can visually assess if different lots follow the same response trajectory or if there is a "shift" in signal intensity across the concentration range.

  • Parameter Comparison Table: RARE extracts the key coefficients from the underlying 4PL or 5PL models (such as Slope, EC50, and Asymptotes) and presents them side-by-side. This allows you to quantify exactly how the biochemical response varies between lots.

  • Residuals (Observed - Predicted) Box Plot: This diagnostic tool visualizes the distribution of error for each batch. It helps you identify if a specific lot has a higher degree of "noise" or a consistent bias compared to the others.

How to interpret it: Look for curve shifts, parameter differences and residual bias. A consistent upward or downward residual pattern for one batch may indicate lot-specific signal behaviour. Wide residual distributions suggest lower precision or inconsistent manufacturing/capture conditions.

Spreadsheet follow-up: Pivot residuals and recovery by batch and concentration. If one batch differs mainly at low concentration, the practical impact may be different from a whole-curve shift.

Screenshot 2026-11-04 at 12.06.22.png
Batch curve overlay and residual box plot.

Understanding the Residuals Box Plot

The box plots in this section allow you to see at a glance how consistently each batch is performing. For each batch:

  • The Center Line (Median): Represents the median residual. If this line is at zero, the batch is perfectly centered on the Ground Truth. If it is consistently above or below zero, it indicates a systematic bias for that specific lot.

  • The Box (Interquartile Range - IQR): Represents the middle 50% of your data. A "taller" box indicates higher variability (lower precision) within that batch.

  • The Whiskers: Extend to show the spread of the rest of the data.

  • Outliers (Individual Dots): Any results that fall significantly outside the expected range are plotted as individual points, helping you identify specific samples that may have had physical defects or interpretation anomalies.

Batch Comparison Metrics

RARE calculates how each "Test" batch deviates from the designated Reference Batch using the following formulae:

Mean Residual (Bias):
The average difference between the measured values and the Ground Truth for the batch.

Batch-Specific Coefficient of Variation (%CV):
This metric quantifies the precision of the residuals for a specific batch, allowing you to compare the "tightness" of results against your reference lot.

By analyzing these plots and tables, you can objectively determine if a new manufacturing lot is performing within your established QC tolerances or if it requires a unique calibration update.

7 · Phone Variability: Qualitative Performance and Threshold Stability

This section reports diagnostic classification performance separately by smartphone model. It includes per-phone confusion matrices, sensitivity, specificity, accuracy, PPV, NPV, false-positive rate and false-negative rate, with denominators and 95% confidence intervals where estimable. It can also show per-phone ROC curves, AUC with confidence intervals and Youden-optimal threshold stability.

The section is available when a classifier has been applied and at least two known phone models each have enough ground-truth results. Smaller additional groups may be shown but do not necessarily support every metric.

How to interpret it: Compare both point estimates and confidence intervals. A phone with higher false negatives may be more important than one with a small accuracy difference. Threshold CV% or a wide range of phone-specific optimal thresholds suggests that a pooled threshold may not behave uniformly across phone models.

Limitation: These outputs are descriptive phone-stratified comparisons. They do not, by themselves, prove paired inter-phone concordance, device equivalence or validation of a common threshold across all phone models.

Spreadsheet follow-up: Export per-phone metrics and review FN/FP counts by concentration or signal band. Investigate whether one phone is overrepresented near the threshold, because that can magnify apparent diagnostic differences.

Screenshot 2026-11-04 at 12.08.58.png
Phone-specific ROC overlay for iPhone 14 and iPhone 17 Pro with AUC values and threshold stability plot.


8 · Phone Variability (Quantitative)

This section evaluates quantitative signal behaviour by phone model within each batch and test line. It may include independent 4PL, 5PL or linear fits per phone, curve overlays, parameter summaries, concentration-level mean/SD/CV%, residual summaries, variance components and comparisons with the selected batch calibration reference.

How to interpret it: Look for phone-specific curve shape differences, concentration-level CV changes and residual bias. A phone can show acceptable qualitative performance while still having a measurable quantitative offset that matters for future calibration or correction strategies.

Limitation: Section 8 does not establish phone equivalence, validate a shared calibration curve or implement per-device curve correction. It provides evidence for review and further study design.

Spreadsheet follow-up: Plot signal, T/C ratio and residuals by phone and concentration. Compare before/after or processed-signal versus T/C-ratio CV where available, but do not treat a CV reduction as validation of a deployed correction unless that study was explicitly designed and controlled.

Screenshot 2026-11-04 at 12.10.45.png
Per-phone calibration curve overlay with concentration-level precision table and residual summary.


9 · Phone Effect & Equivalence

This section calculates phone-model effect estimates against either a configured reference phone model or the other phone models pooled. It reports the estimate, units, sample counts, 95% confidence interval and comparison with saved lower and/or upper metric bounds.

Status values may include:

  • within_margin: the reported interval lies within the configured numerical bounds.

  • exceeds_margin: the estimate or interval crosses a configured bound.

  • not_estimable: the estimate or interval cannot be calculated.

  • not configured: no applicable margin was saved before report generation.

How to interpret it: Treat this as numerical margin comparison evidence, not a standalone equivalence certificate. Check the reference group, units, comparison population, sample counts and whether margins were configured before generation.

Limitation: The Portal does not comprehensively validate every comparison-specific design eligibility condition. Terms such as “formal_validation”, “within_margin”, “Equivalence demonstrated” or a summary PASS must not be treated as sufficient proof of device equivalence without study-design review.

Spreadsheet follow-up: Recreate forest-plot tables and flag comparisons with sparse counts, one-sided margins, disjoint batches or unresolved not-estimable rows. Use these flags to plan a better crossed phone × batch × concentration study if formal equivalence evidence is needed.

Screenshot 2026-11-04 at 12.25.39.png
Phone-effect plot with confidence intervals and configured equivalence/margin bounds.


10 · Environmental Factors and Capture Robustness

This section explores associations between capture conditions and signal measurements. It can include lux versus signal plots, pitch/yaw versus signal plots, Pearson and Spearman correlations, lux-banded precision and counts of excluded records without usable telemetry.

How to interpret it: Use this section to identify potential robustness issues, not to claim validated environmental robustness. Correlations may reflect concentration, batch or phone differences rather than a direct environmental effect. Always check telemetry missingness first.

Limitation: Environmental analysis is exploratory and not adjusted for all batch or concentration effects. Default environmental thresholds may be applied by the backend when explicit configuration is absent; this does not prove that the limits were user-selected or validated acceptance criteria.

Spreadsheet follow-up: Export telemetry and signal fields, then stratify by concentration, batch and phone before interpreting a lux or orientation trend. If a high-lux band appears worse, check whether it also contains a different phone or batch mix.

Screenshot 2026-11-04 at 12.27.11.png
Lux versus signal scatter plot coloured by phone model.


11 · Temporal Capture Dynamics and Usability

This section uses PMF story timing and telemetry to visualise capture dynamics. It can show frame state, orientation and lux over capture time, along with capture identity, telemetry provenance, selected result counts, distinct capture counts and frame counts.

How to interpret it: Use temporal plots to spot capture instability, orientation drift, abrupt lux changes, repeated marginal frames or patterns that suggest usability issues. These metrics describe recorded captures, not abandoned attempts or independent user behaviour unless those data were explicitly captured.

Spreadsheet follow-up: For usability investigations, group frames by capture ID and compare duration, frame count, fit state, pitch/yaw variation and lux variation. Avoid treating every frame as an independent sample when it comes from the same capture story.

Screenshot 2026-11-04 at 12.31.13.png
Temporal capture plot showing frame state, lux and orientation over time for selected phone models.


Calibration fitting failures and inspection

When a recognised fit failure occurs, the report should isolate the failure to the affected phone model or analysis slice, preserve independent outputs, and mark fit-dependent outputs as unavailable with a reason.

If underlying calibrator concentration-versus-signal points are available, the user can inspect them as a scatter plot without a fitted curve. This helps distinguish data-quality or concentration-response anomalies from transport or request errors.

A converged fit with finite parameters is not automatically proof of model identifiability. Non-estimable parameter uncertainty and ill-conditioned fits still require expert review.

Viewing or editing an existing report configuration

Open an existing report configuration when you need to inspect settings, regenerate a report or create a new version.

  • View report: opens the generated report view for metrics, charts and result breakdowns.

  • Save as a new version: use this when you need to change dataset, classifier, batch group, sections or limits while preserving the previous configuration.

  • Generate or regenerate: runs the analysis using the saved settings.

  • Verify lineage: confirm the configuration version, dataset ID, classifier version, batch version and RARE Portal version before using results in a validation or QC record.

After generating a report

After a report is generated:

  • Start with scope: verify provenance, dataset, test-line, classifier split, phone models and generation time.

  • Review population accounting: ensure analysed counts, missingness and sparse design cells support the intended interpretation.

  • Read section limitations: especially for phone equivalence, environmental analysis and temporal usability outputs.

  • Export the PDF: keep a human-readable record with charts, tables, contents and running headers.

  • Export tables where available: use spreadsheet analysis to investigate denominators, outliers, sparse cells and subgroup effects.

  • Protect data integrity: report generation does not rewrite original stored measurements; it creates an analysis snapshot from selected data and configuration versions.


Appendices

The following appendices provide the proposed calculation models for the report sections added or expanded for population accounting, phone variability, phone-effect margin comparison, environmental robustness and temporal usability. They are intended to make the report calculations transparent for LF manufacturing, QC and laboratory users, and to support independent review in exported tabular data where those fields are available.

Usage note: When using exported data, keep report ID, dataset ID, result ID, phone model, batch ID, classifier version and configuration version with every calculation so that any spreadsheet-derived result remains traceable to the generated report.

Appendix A · Section 1.1 Population Accounting calculations

Purpose: Use these calculations to audit how many records move from capture to analysis, and to identify whether missing metadata or sparse design cells limit interpretation.

Funnel stage yield and overall retention

For each population-accounting stage:

Overall retention from capture to analysis is:

Where: N_k is the record count at funnel stage k, for example Captured → Selected → Ground-truth linked → Technically valid → Analysed.

Interpretation: A large fall between Captured and Selected usually indicates filter or configuration exclusion. A fall between Technically valid and Analysed can indicate missing analytical signal, failed peak detection or another fit-dependent exclusion. Do not interpret later performance metrics without first checking which records were removed.

Spreadsheet check: =(CurrentStageCount / PreviousStageCount) * 100

Field-level missingness rate

Where: M_j is the missingness rate for field j, such as ground truth, signal, image, capture status, lux, pitch or yaw.

Interpretation: High missingness in telemetry fields weakens environmental and usability conclusions. For example, if lux is missing for many records, Section 10 should be treated as exploratory even when a plot is shown.

Spreadsheet check: =(MissingFieldCount / TotalRecordCount) * 100

Design-crossing sparse-cell flag

Where: N_ijk is the number of records in a phone model × batch × concentration or class cell.

Interpretation: Cells with fewer than three observations should normally be treated as descriptive rather than confirmatory. A phone comparison is especially fragile if each phone appears in different batches or concentration levels.

Spreadsheet check: =IF(CellCount<3,"Sparse cell","OK")

Appendix B · Section 7 Phone Variability: Qualitative Performance and Threshold Stability calculations

Purpose: Use these calculations to compare qualitative diagnostic behaviour by phone model and to assess whether the decision threshold appears stable across supported devices.

Per-phone false-positive and false-negative rates

Where: subscript p denotes a specific phone model. TP, TN, FP and FN are counted separately for that phone.

Interpretation: For screening uses, compare FNR_p carefully because false negatives may be the critical risk. For confirmatory uses, compare FPR_p. Always read the numerator and denominator, not only the percentage.

Spreadsheet check: =FP/(TN+FP) and =FN/(TP+FN)

Wilson score confidence interval for proportions

Where: p̂ is the observed proportion, such as sensitivity, specificity or accuracy; n is the relevant denominator; and z = 1.96 for an approximate 95% confidence interval.

Interpretation: Wilson intervals are more reliable than simple normal intervals when denominators are small or proportions are close to 0% or 100%. If phone confidence intervals are wide and overlapping, apparent rank ordering between phones may not be meaningful.

Optimal-threshold instability across phone models

Where: T_opt,p is the Youden-optimal threshold calculated independently for phone model p, and P is the number of phone models.

Interpretation: A high threshold CV suggests that the pooled operating threshold may not sit in an equally stable part of the response distribution for every phone. This does not by itself prove that phone-specific thresholds are required, but it is a useful trigger for review.

Spreadsheet check: =(STDEV.S(threshold_range)/AVERAGE(threshold_range))*100

Appendix C · Section 8 Phone Variability (Quantitative) calculations

Purpose: Use these models to describe phone-specific calibration behaviour, quantitative residuals and concentration-level precision.

Four-parameter logistic standard curve

Five-parameter logistic standard curve

Where: x is analyte concentration; y is optical density, peak height or T/C ratio; a is the minimum asymptote; d is the maximum asymptote; c is the inflection concentration or EC50; b is the slope parameter; and g is the asymmetry parameter. When g = 1, the 5PL model reduces to a symmetric 4PL form.

Interpretation: Differences in a can suggest phone-specific optical baseline offsets. Differences in b or c may indicate different dynamic range scaling or response steepness. Treat fitted parameters cautiously when the concentration range is sparse or when the fit is not identifiable.

Concentration-specific residual relative to a reference curve

Where: y_p,k is the observed response for phone p at concentration x_k, and ŷ_ref(x_k) is the predicted response from the selected reference batch or reference calibration curve.

Interpretation: Positive residuals show over-reading relative to the reference curve; negative residuals show under-reading. Review residuals by concentration because a phone may be biased only near the decision threshold or at the edge of the measuring range.

Spreadsheet check: =ObservedResponse - ReferencePredictedResponse

Appendix D · Section 9 Phone Effect & Equivalence calculations

Purpose: Use these calculations to understand how the report estimates a phone-model effect and compares it with configured numerical margins. These calculations support review but do not replace study-design assessment for formal equivalence claims.

Estimated phone-model effect

The same structure may be used when the comparison is against the pooled mean of the other phone models, depending on the report configuration.

Standard error of the difference

Where: s_test and s_ref are the standard deviations for the test and reference phone groups, and n_test and n_ref are their sample counts.

Spreadsheet check: =SQRT((STDEV.S(test_range)^2/COUNT(test_range))+(STDEV.S(ref_range)^2/COUNT(ref_range)))

Confidence interval for the effect estimate

Where: t_(1-α,ν) is the Student t critical value and ν is the applicable degrees of freedom, often using a Welch/Satterthwaite approximation when variances differ.

Margin comparison decision rule

Where: -θ_L and +θ_U are the configured lower and upper acceptance margins.

Interpretation: A point estimate inside the margin is not enough. For an interval-based margin comparison, the full confidence interval must fit inside the configured bounds. Sparse counts, non-crossed designs and unbalanced phone × batch × concentration coverage should be flagged before using the result as evidence.

Appendix E · Section 10 Environmental Factors and Capture Robustness calculations

Purpose: Use these calculations to explore whether capture telemetry such as lux, pitch or yaw is associated with signal or precision. These outputs are exploratory unless the study was designed to test environmental robustness.

Pearson correlation coefficient

Where: E_i is an environmental parameter, such as lux, pitch or yaw, and y_i is the signal or T/C ratio.

Interpretation: Pearson correlation measures linear association. A visible trend may be caused by confounding if one phone, batch or concentration dominates a particular environmental range.

Spreadsheet check: =CORREL(environment_range, signal_range)

Spearman rank correlation


Where: d_i is the difference between the rank of the environmental value and the rank of the signal value for observation i.

Interpretation: Spearman correlation detects monotonic relationships that are not necessarily linear. It is useful when signal changes steadily with lux or angle but not at a constant rate.

Lux-banded precision

Where: observations are grouped into lux brackets, for example low, medium and high lux, and s_tier is the standard deviation within a tier at a fixed concentration or other controlled stratum.

Interpretation: Compare lux-banded CV only within comparable concentration, batch and phone strata. A high CV in one lux tier may reflect different sample composition rather than an environmental effect.

Spreadsheet check: =(STDEV.S(tier_signal_range)/AVERAGE(tier_signal_range))*100

Appendix F · Section 11 Temporal Capture Dynamics and Usability calculations

Purpose: Use these calculations to describe capture timing, orientation stability and frame-fit behaviour from PMF story or frame telemetry. These are capture-dynamics metrics, not a complete measure of abandoned attempts or independent user behaviour unless those data are explicitly recorded.

Total capture duration or lock time

Interpretation: Longer capture durations may indicate difficulty aligning the strip, glare, focus issues or unstable cassette tracking. Compare by phone model and capture context before assigning a cause.

Spreadsheet check: =FinalLockTimestamp - FirstFrameTimestamp

Combined angular tilt

Interpretation: This combines vertical and horizontal orientation into one angle-like stability measure. High values can indicate perspective distortion risk, especially for multi-strip layouts or small regions of interest.

Spreadsheet check: =SQRT(Pitch^2 + Yaw^2)

Frame-fit stability rate

Interpretation: This is the proportion of preview frames where the cassette or strip tracking algorithm reports a valid fit state. Low stability can indicate user hand movement, glare, boundary instability or device/camera focus issues.

Spreadsheet check: =(ValidFrameCount / TotalFrameCount) * 100