Overview
MelanoDet develops and clinically validates a multimodal imaging system for cutaneous melanoma screening. The system combines three synchronized cameras—visible (VIS), near-infrared (NIR) and long-wave infrared (thermal, TH)—to capture a static triad of co-registered images for each suspicious lesion, followed by a short dynamic thermal sequence after a mild cooling stimulus. The project couples this hardware with a standardized acquisition protocol, a file-based database, and automated quality-control tools, aiming to create a high-quality multimodal dataset that can support future diagnostic and risk-stratification algorithms.
System-level results
The v1.0 MelanoDet setup has been designed, assembled and bench-tested according to a set of explicit hardware requirements. The system provides a common field of view of approximately 25 × 25 mm at a working distance around 250 mm, with a shared region containing the lesion and a small fiducial marker used for multimodal registration. VIS and NIR exposures are synchronized within 10 ms, while the thermal camera records at ≥10 fps with drift below 0.1 °C during the dynamic sequence. All cameras save lossless data with sufficient bit-depth for downstream analysis. These tests confirm that the setup can reliably deliver well-aligned VIS/NIR/TH data under controlled illumination and geometry, in line with the project’s TRL-4 hardware goals.
Acquisition protocol and clinical workflow
A complete acquisition protocol and a practical guideline for operators have been drafted and deployed in clinic. Each session follows the same structure: clinical examination and dermoscopy, patient preparation, static VIS/NIR/TH triad, then a two-to-three-minute thermal recovery sequence after a standardized cooling stimulus. The protocol includes detailed pre-session checks (environment, illumination, calibration), in-session retake logic, and post-session QC and data handover. From the clinician’s point of view, the procedure remains fully non-invasive, with no contact, radiation, or contrast agents, and complies with GDPR and local ethical regulations for patient consent and data handling.
Data quality and validation
An automated validation framework has been implemented to check each VIS/NIR/TH triplet before inclusion in the central database. For an initial pilot set of 10 triplets (30 images), the pipeline quantified focus, contrast, saturation, and illumination uniformity per modality, and evaluated cross-modal similarity between VIS, NIR and TH. All triplets met the predefined thresholds for sharpness and contrast, with very low saturation and good illumination uniformity across the field of view. In parallel, the project has defined monitoring metrics and early-warning indicators for the full acquisition workflow, including FOV overlap, retake rate, dynamic thermal protocol adherence, and registration health (SSIM/MI/NMI). These indicators are used to track the stability of the system as new data are acquired.
Dataset and current status
The MelanoDet system has been experimentally validated on an initial cohort of cases imaged in clinical conditions, confirming the practical feasibility of capturing static VIS/NIR/TH triads and dynamic thermal sequences under realistic workflows. Project-level targets forecast a cohort on the order of several hundred analyzed cases, including a substantial number of biopsy-confirmed melanomas, subject to clinical throughput and pathology timelines. The core building blocks are now in place: a clinic-ready imaging rig, a robust acquisition protocol, a file-centric database layout, and an automated QC/validation pipeline. Ongoing work focuses on scaling data acquisition, refining registration and analysis methods, and preparing the ground for downstream algorithm development and external validation.
Multispectral Image Registration
Within the current phase of the project, we have not only designed and validated the multimodal VIS/NIR/TH acquisition setup, but also implemented an image registration pipeline that makes these modalities usable together at lesion level. Using a small fiducial visible in all channels, we first achieve a stable geometric alignment between VIS and NIR, and then register thermal images to the VIS frame using a combination of calibrated homographies and robust corner/marker detection, with semi-automatic support where LWIR contrast is weak. On our pilot dataset, this pipeline produces well-aligned triplets that are further checked by automatic quality metrics (SSIM, mutual information and normalized mutual information) and by visual inspection when thresholds are not met, so only triplets with reliable multimodal alignment are promoted into the central MelanoDet database and downstream analysis.