Skip to content
ModelListedExternal
Download

Disasterlens v2 Multimodal Disaster Response

DisasterLens V2 is a two-stage multimodal disaster-response classifier built using computer vision, transfer learning, NLP, and late probability fusion.

Sourcehuggingface.co/avnisinghal001/disasterlens-v2-multimodal-disaster-response/tree/5ce9acce61aa8923e3ec806375b515a947fcc303
KindOtherVersionRevision 5ce9acc

Model source

Kind
Other
File size
106–255 MB
Version
vRevision 5ce9acc
Updated
Sep 17, 2026
Downloads
0
Source
Hugging Face

Source description

DisasterLens V2: Multimodal Disaster-Response Classifier

DisasterLens V2 is a two-stage multimodal disaster-response classifier built using computer vision, transfer learning, NLP, and late probability fusion.

It analyzes a disaster-related image and optional social media text to predict:

  1. Whether the post is informative for disaster response.
  2. If informative, the humanitarian category.

Architecture

Stage 1: Informativeness Detection

Predicts:

  • informative
  • not_informative

Models:

  • ConvNeXt-Tiny image classifier with test-time augmentation
  • DistilBERT text classifier
  • Late fusion when text is available

Fusion weights:

  • image: 0.55
  • text: 0.45

Stage 2: Humanitarian Category Classification

Predicts one of 5 grouped humanitarian categories:

  • not_humanitarian
  • damage
  • rescue_or_donation
  • affected_people
  • other_relevant_information

Models:

  • ConvNeXt-Tiny image classifier with test-time augmentation
  • DistilBERT text classifier
  • Late fusion when text is available

Fusion weights:

  • image: 0.65
  • text: 0.35

Final Results

ComponentInputAccuracyF1 TypeF1
Informativeness image-onlyImage0.8494Binary F10.8565
Informativeness fusionImage + Text0.8632Binary F10.8701
Humanitarian image-onlyImage0.7948Macro F10.7375
Humanitarian fusionImage + Text0.7993Macro F10.7436
Full V2 cascadeImage + Text0.8485Macro F10.7837

Final full V2 cascade:

  • Accuracy: 84.85%
  • Macro F1: 78.37%
  • Weighted F1: 84.44%

Usage

Run from project root:

python scripts/34_predict_disasterlens_v2.py \
  --image_path "path/to/disaster_image.jpg" \
  --text "optional tweet or post text" \
  --json

Image only mode:

python scripts/34_predict_disasterlens_v2.py \
  --image_path "path/to/disaster_image.jpg" \
  --json

Example Output:

{
  "mode": "multimodal",
  "informativeness": {
    "prediction": "informative",
    "confidence": 0.9507
  },
  "humanitarian": {
    "mode": "image_text_fusion",
    "prediction": "other_relevant_information",
    "confidence": 0.9878
  }
}

Dataset

This model was trained on CrisisMMD v2.0-style disaster image/text data.

The original dataset is not redistributed in this repository. Please obtain the dataset from the official source and follow its license and terms.

Intended Use

This project is intended for:

  • disaster-response research
  • crisis informatics experiments
  • multimodal ML demonstrations
  • portfolio and educational use

It should not be used as the sole decision-making system in real emergency response workflows.

Limitations

  • Performance varies by disaster type and visual clarity.
  • Text-only humanitarian classification is weak alone, but useful in fusion.
  • Some categories are visually and semantically overlapping.
  • The dataset is imbalanced, especially for affected people-related labels.

Sources

1 source
Hugging FacePrimaryVersion 5ce9acce61aa8923e3ec806375b515a947fcc303
huggingface.co/avnisinghal001/disasterlens-v2-multimodal-disaster-response/tree/5ce9acce61aa8923e3ec806375b515a947fcc303

Verified Sep 17

Model artifacts

4 artifacts

models/convnext_tiny_humanitarian_5class_best.pt

pt · 106 MB · SHA-256 f33bccf52e92…1f99 · Hugging Face

Download

models/convnext_tiny_informative_best.pt

pt · 106 MB · SHA-256 9dc488b50e3c…9ce3 · Hugging Face

Download

models/distilbert_humanitarian_text_5class_best.pt

pt · 255 MB · SHA-256 5309e37241e2…5411 · Hugging Face

Download

models/distilbert_text_informative_best.pt

pt · 255 MB · SHA-256 233bcce08b24…409b · Hugging Face

Download

Source excerpts

2 excerpts
ReadmeSep 17huggingface.co/avnisinghal001/disasterlens-v2-multimodal-disaster-response

--- license: mit tags: - computer-vision - transfer-learning - disaster-response - multimodal - image-classification - text-classification - convnext - distilbert - late-fusion - crisis-informatics pipeline_tag: image-classification --- # DisasterLens V2: Multimodal Disaster-Response Classifier DisasterLens V2 is a two-stage multimodal disaster-response classifier built using computer vision, transfer learning, NLP, and late probability fusion. It analyzes a disaster-related image and optional social media text to predict: 1. Whether the post is informative for disaster response. 2. If informative, the humanitarian category. ## Architecture ### Stage 1: Informativeness Detection Predicts: - `informative` - `not_informative` Models: - ConvNeXt-Tiny image classifier with test-time augmentation - DistilBERT text classifier - Late fusion when text is available Fusion weights: - image: `0.55` - text: `0.45` ### Stage 2: Humanitarian Category Classification Predicts one of 5 grouped humanitarian categories: - `not_humanitarian` - `damage` - `rescue_or_donation` - `affected_people` - `other_relevant_information` Models: - ConvNeXt-Tiny image classifier with test-time augmentation - DistilBERT text classifier - Late fusion when text is available Fusion weights: - image: `0.65` - text: `0.35` ## Final Results | Component | Input | Accuracy | F1 Type | F1 | |---|---|---:|---|---:| | Informativeness image-only | Image | 0.8494 | Binary F1 | 0.8565 | | Informativeness fusion | Image + Text | 0.8632 | Binary F1 | 0.8701 | | Humanitarian image-only | Image | 0.7948 | Macro F1 | 0.7375 | | Humanitarian fusion | Image + Text | 0.7993 | Macro F1 | 0.7436 | | Full V2 cascade | Image + Text | 0.8485 | Macro F1 | 0.7837 | Final full V2 cascade: - Accuracy: **84.85%** - Macro F1: **78.37%** - Weighted F1: **84.44%** ## Usage Run from project root: ```bash python scripts/34_predict_disasterlens_v2.py \ --image_path "path/to/disaster_image.jpg" \ --text "optional tweet or post text" \ --json ``` Image only mode: ```bash python scripts/34_predict_disasterlens_v2.py \ --image_path "path/to/disaster_image.jpg" \ --json ``` Example Output: ```json { "mode": "multimodal", "informativeness": { "prediction": "informative", "confidence": 0.9507 }, "humanitarian": { "mode": "image_text_fusion", "prediction": "other_relevant_information", "confid...

NoteSep 17huggingface.co/avnisinghal001/disasterlens-v2-multimodal-disaster-response

Source context: 0 downloads · 0 likes · Pipeline image-classification · Repo avnisinghal001/disasterlens-v2-multimodal-disaster-response