Skip to content
ModelListedExternal
Browse files

BCAF 2026

Official model weights for the paper: "Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting" Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg...

Sourcehuggingface.co/jonasvilhofunk/BCAF_2026/tree/2b93c9adc23ff1f8e46c0d7e1ff75ffd4c1302fd
KindOtherVersionRevision 2b93c9a

Model source

Kind
Other
File size
123–387 MB
Version
vRevision 2b93c9a
Updated
Sep 21, 2026
Downloads
0
Source
Hugging Face

Source description

BCAF — Bidirectional Cross-Attention Fusion

Official model weights for the paper:
"Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting"
Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg Maier, Thomas Längle, Markus Klute

arXiv GitHub

Abstract

We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding spatial pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention.

Usage

Please refer to the detailed instructions and example notebooks (Preprocess, Evaluate, and Training) available on the official GitHub repository.

Checkpoints

Fusion Models

FilemIoU ↑Img./s ↑
BCAF_SpectralWaste_rgb1024_hsi5_best.pth76.4 ± 0.4%31
BCAF_SpectralWaste_rgb512_hsi5_best.pth75.4 ± 0.2%55
BCAF_SpectralWaste_rgb256_hsi5_best.pth71.1 ± 0.4%54
logitfusion_SpectralWaste_rgb1024_hsi5_best.pth72.6 ± 0.8%39

Unimodal RGB

FilemIoU ↑Img./s ↑
swin_t_SpectralWaste_rgb_256_best.pth65.8 ± 1.2%141
swin_t_SpectralWaste_rgb_512_best.pth71.1 ± 0.6%135
swin_t_SpectralWaste_rgb_1024_best.pth71.6 ± 0.3%60
swin_t_SpectralWaste_rgb_2048_best.pth68.4 ± 0.8%15

Unimodal HSI

FilemIoU ↑Img./s ↑
swin_t_SpectralWaste_hsi_1_best.pth60.9 ± 0.2%141
adapted_swin_t_SpectralWaste_hsi_3_best.pth59.7 ± 0.7%114
adapted_swin_t_SpectralWaste_hsi_5_best.pth60.3 ± 0.9%119
adapted_swin_t_SpectralWaste_hsi_7_best.pth59.0 ± 1.5%91
adapted_swin_t_SpectralWaste_hsi_10_best.pth57.8 ± 1.2%68

Citation

@article{funk2026bcaf,
  title={Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI
         for Multimodal Automated Waste Sorting},
  author={Jonas V. Funk and Lukas Roming and Andreas Michel and Paul B{\"{a}}cker
          and Georg Maier and Thomas L{\"{a}}ngle and Markus Klute},
  year={2026},
  eprint={2603.13941},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.13941}
}

Acknowledgements

Sources

1 source
Hugging FacePrimaryVersion 2b93c9adc23ff1f8e46c0d7e1ff75ffd4c1302fd
huggingface.co/jonasvilhofunk/BCAF_2026/tree/2b93c9adc23ff1f8e46c0d7e1ff75ffd4c1302fd

Verified Sep 8

Download files

Model artifacts

13 artifacts

adapted_swin_t_SpectralWaste_hsi_10_best.pth

pth · 174 MB · SHA-256 6ec1a2775bd8…c206 · Hugging Face

Download file

adapted_swin_t_SpectralWaste_hsi_3_best.pth

pth · 174 MB · SHA-256 026def8acd66…eefa · Hugging Face

Download file

adapted_swin_t_SpectralWaste_hsi_5_best.pth

pth · 174 MB · SHA-256 99a1440b1512…0c93 · Hugging Face

Download file

adapted_swin_t_SpectralWaste_hsi_7_best.pth

pth · 174 MB · SHA-256 8247d68a7d48…76a3 · Hugging Face

Download file

BCAF_SpectralWaste_rgb1024_hsi5_best.pth

pth · 387 MB · SHA-256 6f0894ad616d…2e0e · Hugging Face

Download file

BCAF_SpectralWaste_rgb256_hsi5_best.pth

pth · 387 MB · SHA-256 e3fb7b169297…9a30 · Hugging Face

Download file

BCAF_SpectralWaste_rgb512_hsi5_best.pth

pth · 387 MB · SHA-256 226715f47c1a…e9e2 · Hugging Face

Download file

logitfusion_SpectralWaste_rgb1024_hsi5_best.pth

pth · 297 MB · SHA-256 e5d142bc1d89…e8c8 · Hugging Face

Download file

swin_t_SpectralWaste_hsi_1_best.pth

pth · 123 MB · SHA-256 56290d088d03…809b · Hugging Face

Download file

swin_t_SpectralWaste_rgb_1024_best.pth

pth · 123 MB · SHA-256 6d503bb07f23…f3a6 · Hugging Face

Download file

swin_t_SpectralWaste_rgb_2048_best.pth

pth · 123 MB · SHA-256 c40a5061a690…c1a3 · Hugging Face

Download file

swin_t_SpectralWaste_rgb_256_best.pth

pth · 123 MB · SHA-256 00f8b05f9c63…574d · Hugging Face

Download file

swin_t_SpectralWaste_rgb_512_best.pth

pth · 123 MB · SHA-256 d403c5a2afc6…d6c8 · Hugging Face

Download file

Source excerpts

2 excerpts
ReadmeSep 8huggingface.co/jonasvilhofunk/BCAF_2026

--- datasets: - spectralwaste license: mit metrics: - mean_iou pipeline_tag: image-segmentation tags: - semantic-segmentation - computer-vision - multimodal - hyperspectral-imaging - waste-sorting - swin-transformer - pytorch model-index: - name: BCAF_SpectralWaste_rgb1024_hsi5 results: - task: type: image-segmentation dataset: name: SpectralWaste type: spectralwaste metrics: - type: mean_iou value: 76.4 --- # BCAF — Bidirectional Cross-Attention Fusion Official model weights for the paper: **"Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting"** Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg Maier, Thomas Längle, Markus Klute [![arXiv](https://img.shields.io/badge/arXiv-2603.13941-b31b1b.svg)](https://arxiv.org/abs/2603.13941)  [![GitHub](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/jonasvilhofunk/BCAF_2026) ## Abstract We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding spatial pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. --- ## Usage Please refer to the detailed instructions and example notebooks (Preprocess, Evaluate, and Training) available on the [official GitHub repository](https://github.com/jonasvilhofunk/BCAF_2026). --- ## Checkpoints ### Fusion Models | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `BCAF_SpectralWaste_rgb1024_hsi5_best.pth` | **76.4 ± 0.4%** | 31 | | `BCAF_SpectralWaste_rgb512_hsi5_best.pth` | 75.4 ± 0.2% | 55 | | `BCAF_SpectralWaste_rgb256_hsi5_best.pth` | 71.1 ± 0.4% | 54 | | `logitfusion_SpectralWaste_rgb1024_hsi5_best.pth` | 72.6 ± 0.8% | 39 | ### Unimodal RGB | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_rgb_256_best.pth` | 65.8 ± 1.2% | 141 | | `swin_t_SpectralWaste_rgb_512_best.pth` | 71.1 ± 0.6% | 135 | | `swin_t_SpectralWaste_rgb_1024_best.pth` | 71.6 ± 0.3% | 60 | | `swin_t_SpectralWaste_rgb_2048_best.pth` | 68.4 ± 0.8% | 15 | ### Unimodal HSI | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_hsi_1_best.pth` | 60.9 ± 0.2% | 141 |...

NoteSep 8huggingface.co/jonasvilhofunk/BCAF_2026

Source context: 0 downloads · 0 likes · Pipeline image-segmentation · Repo jonasvilhofunk/BCAF_2026