MatSense: A Multimodal Dataset and Benchmark for Material Classification

Multimodal material understanding with RGB, NIR, Polar, and Thermal imagery.

Illustration of material samples viewed through multiple sensing modalities

Overview

MatSense is a multimodal dataset and benchmark for material classification. It brings together RGB, NIR, Polar, and Thermal imagery to support research on visual material understanding under complementary sensing conditions.

The benchmark is designed to make it easier to study how different sensing modalities contribute to robust material recognition, while providing a clear foundation for comparable future methods and evaluations.

Material Classes

MatSense covers 20 material classes spanning plastics, metals, natural materials, textiles, and manufactured surfaces.

PS Clay HDPE LDPE Steel Ceramic PP Paper Organic Aluminium PVC PET Copper Glass Rubber Wood Cotton EVA Leather Rock

Multimodal Capture

RGB

Visible-spectrum color and appearance information.

NIR

Near-infrared sensing for complementary material cues.

Polar

Polarimetric information that captures surface behavior.

Thermal

Thermal imagery for temperature-related visual structure.

Downloads

MatSense release package

MatSense includes both the raw and processed data used in the paper. Processed files are currently available for download; owing to their large size, the raw files are available upon request.

Download dataset

Citation

Citation details will be published together with the official MatSense release.

@misc{matsense,
  title = {MatSense: A Multimodal Dataset and Benchmark for Material Classification},
  note = {Dataset release in preparation}
}