Decluttr Clip Vit B32 Coreml

--- license: mit library_name: coreml pipeline_tag: zero-shot-image-classification tags: - clip - openai - ios - coreml - on-device - image-search - visual-search - decluttr-ai base_model: openai/clip-vit-base-patch32 --- # Decluttr AI · CLIP ViT-B/32 · Core ML On-device text↔image search for the [Decluttr AI](https://apps.apple.com/app/id6770379060) iOS app. This is a Core ML port of OpenAI's **CLIP ViT-B/32** (image and text encoders), exported in FP16. It is designed to be fetched on demand by an iOS client and executed entirely on the device's Neural Engine. The app ships *without* these weights so the App Store binary stays small (around 86 MB). When a user enables Visual Search inside the app, this repository is fetched, integrity-checked, compiled to `.mlmodelc`, and run locally. Nothing leaves the phone. ## Files | Path | Bytes | Purpose | |------|------:|---------| | `CLIPImage.mlpackage/Data/com.apple.CoreML/weights/weight.bin` | 175,712,384 | FP16 image-encoder weights | | `CLIPImage.mlpackage/Data/com.apple.CoreML/model.mlmodel` | 136,412 | Image-encoder graph | | `CLIPImage.mlpackage/Manifest.json` | 617 | Core ML package manifest | | `CLIPText.mlpackage/Data/com.apple.CoreML/weights/weight.bin` | 126,878,848 | FP16 text-encoder weights | | `CLIPText.mlpackage/Data/com.apple.CoreML/model.mlmodel` | 171,503 | Text-encoder graph | | `CLIPText.mlpackage/Manifest.json` | 617 | Core ML package manifest | | `clip_tokenizer/vocab.json` | 961,143 | BPE token to id map | | `clip_tokenizer/merges.txt` | 524,619 | BPE merge rules | **Total: 303 MB.** ## Encoder I/O contract Both encoders are L2-normalised at the output, so cosine similarity reduces to a dot product. ### Image encoder · `CLIPImage.mlpackage` | | | |---|---| | **Input** | `image` | | **Type** | RGB `CGImage`, 224×224, 0–1 normalised pixels | | **Output** | `embedding` | | **Shape** | `MLMultiArray<Float16>` `(1, 512)` | The model bakes in CLIP's `(mean, std)` normalisation internally, so the Swift caller passes raw 0–1 pixels. ### Text encoder · `CLIPText.mlpackage` | | | |---|---| | **Input** | `input_ids` | | **Type** | `MLMultiArray<Int32>` `(1, 77)`, BPE-tokenized, padded with `0` | | **Output** | `embedding` | | **Shape** | `MLMultiArray<Float16>` `(1, 512)` | Tokenization is **not** baked into the Core ML graph. The consuming app runs a Swift BPE that loads `vocab.jso...

Source context: 184 downloads · 0 likes · Pipeline zero-shot-image-classification · Library coreml · Repo ddtfzco/decluttr-clip-vit-b32-coreml