Skip to content
ModelListedExternal
Download

Rico Blip LoRA Model

This model is a BLIP image captioning model fine-tuned using LoRA (Low Rank Adaptation) on the RICO Screen2Words dataset.

Sourcehuggingface.co/suhas2468/rico-blip-lora-model/tree/d5169016e1dba0488027ed87fb82c05914d65a5a
KindLoRAVersionadapter_modelCgrade

Model source

Kind
LoRA
File size
4.51 MB
Version
vadapter_model
Updated
Sep 18, 2026
Downloads
0
Source
Hugging Face

Source description

RICO BLIP LoRA Model

Model Description

This model is a BLIP image captioning model fine-tuned using LoRA (Low Rank Adaptation) on the RICO Screen2Words dataset.

The goal of the model is to generate textual descriptions of mobile user interface screenshots. The model takes an image of a UI screen as input and produces a caption describing the interface.

Model Details

  • Developed by: suhas2468
  • Model type: Vision-Language Model (Image Captioning)
  • Language(s): English
  • License: Apache 2.0
  • Finetuned from model: Salesforce/blip-image-captioning-base

Model Sources

Intended Use

Direct Use

This model can be used to generate captions describing mobile UI screenshots. It can help automatically summarize UI layouts and components present in mobile application interfaces.

Example use cases include:

  • UI understanding
  • UI documentation
  • Automated interface description

Out-of-Scope Use

The model is not designed for:

  • general image captioning outside UI screenshots
  • safety-critical applications
  • decision-making systems

How to Use the Model

Example inference code:

from transformers import pipeline

pipe = pipeline(
    "image-to-text",
    model="suhas2468/rico-blip-lora-model"
)

result = pipe("example_image.png")
print(result)

Training Details

Training Data

The model was fine-tuned using the RICO Screen2Words dataset.

This dataset contains mobile UI screenshots paired with captions describing the interface.

Dataset link: https://huggingface.co/datasets/rootsautomation/RICO-Screen2Words

Training Setup

  • GPU: NVIDIA Tesla T4
  • Epochs: 1
  • Batch size: 4
  • Learning rate: 2e-5
  • Training samples: 500

Training was performed using Hugging Face Transformers, PEFT, and PyTorch.

Training Method

The model was fine-tuned using LoRA (Low Rank Adaptation).

LoRA enables efficient fine-tuning by adding small trainable adapters to the base model while keeping most of the original model parameters frozen. This reduces GPU memory usage and speeds up training.

Limitations

  • The model was trained on a relatively small subset of the dataset (500 samples).
  • Performance may vary for UI layouts not represented in the training data.
  • The model is primarily intended for educational and research purposes.

Hardware

Training was performed on:

  • GPU: NVIDIA Tesla T4
  • Platform: Google Colab

Additional Resources

GitHub repository containing the training notebook and code:

https://github.com/suhassh2/orange-multimodal-slm

Sources

1 source
Hugging FacePrimaryVersion d5169016e1dba0488027ed87fb82c05914d65a5a
huggingface.co/suhas2468/rico-blip-lora-model/tree/d5169016e1dba0488027ed87fb82c05914d65a5a

Verified Sep 3

Model artifacts

1 artifact

adapter_model.safetensors

safetensors · 4.51 MB · SHA-256 56a9cfeb7036…c512 · Hugging Face

Download file

Source excerpts

2 excerpts
ReadmeSep 3huggingface.co/suhas2468/rico-blip-lora-model

--- library_name: transformers tags: - transformers - image-to-text --- # RICO BLIP LoRA Model ## Model Description This model is a BLIP image captioning model fine-tuned using **LoRA (Low Rank Adaptation)** on the **RICO Screen2Words dataset**. The goal of the model is to generate textual descriptions of mobile user interface screenshots. The model takes an image of a UI screen as input and produces a caption describing the interface. --- ## Model Details * **Developed by:** suhas2468 * **Model type:** Vision-Language Model (Image Captioning) * **Language(s):** English * **License:** Apache 2.0 * **Finetuned from model:** Salesforce/blip-image-captioning-base --- ## Model Sources * **Repository:** https://github.com/suhassh2/orange-multimodal-slm * **Base Model:** https://huggingface.co/Salesforce/blip-image-captioning-base * **Dataset:** https://huggingface.co/datasets/rootsautomation/RICO-Screen2Words --- ## Intended Use ### Direct Use This model can be used to generate captions describing mobile UI screenshots. It can help automatically summarize UI layouts and components present in mobile application interfaces. Example use cases include: * UI understanding * UI documentation * Automated interface description ### Out-of-Scope Use The model is not designed for: * general image captioning outside UI screenshots * safety-critical applications * decision-making systems --- ## How to Use the Model Example inference code: ```python from transformers import pipeline pipe = pipeline( "image-to-text", model="suhas2468/rico-blip-lora-model" ) result = pipe("example_image.png") print(result) ``` --- ## Training Details ### Training Data The model was fine-tuned using the **RICO Screen2Words dataset**. This dataset contains mobile UI screenshots paired with captions describing the interface. Dataset link: https://huggingface.co/datasets/rootsautomation/RICO-Screen2Words --- ### Training Setup * **GPU:** NVIDIA Tesla T4 * **Epochs:** 1 * **Batch size:** 4 * **Learning rate:** 2e-5 * **Training samples:** 500 Training was performed using Hugging Face **Transformers**, **PEFT**, and **PyTorch**. --- ## Training Method The model was fine-tuned using **LoRA (Low Rank Adaptation)**. LoRA enables efficient fine-tuning by adding small trainable adapters to the base model while keeping most of the original model parameters frozen....

NoteSep 3huggingface.co/suhas2468/rico-blip-lora-model

Source context: 0 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo suhas2468/rico-blip-lora-model