# Using AI Models in Xcode/iOS

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060%3E%5C
**Category:** Beginners\
**Created:** [December 2, 2025, 10:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060 "2025-12-02T22:28:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dren77](https://avatars.discourse-cdn.com/v4/letter/d/c77e96/32.png) [@Dren77](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDren77)%5C
**Post date:** [December 2, 2025, 10:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060%2F1 "2025-12-02T22:28:26Z")

</div>

I am having issues running llama 3.1 8B CoreML/StableDiffusion models using Xcode on iPad M3 device such as:

AI: “the only accessible CoreML package (andmev/Llama-3.1-8B-Instruct-CoreML) still hits ios18.constexpr\_blockwise\_shift\_scale → std::bad\_cast inside Apple’s BNNS compiler, which ultimately throws the EXC\_BAD\_ACCESS you’re seeing before we can catch it. Until we can install an int8/fp16 flavor (or re-export the model ourselves), the app will keep crashing during the Core ML load phase.”

1. Does anyone know or have links to a working llama(text) /Stable Diffusion(image) model that already works well with Xcode/iOS, and the steps to install/make it work?

Thank you so much for your help

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [December 3, 2025, 2:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060%2F2 "2025-12-03T02:06:22Z")

</div>

I’m not familiar with iOS…

* * *

I’ll go step by step:

1. Why your current Llama-3.1-8B CoreML model is crashing.
2. Concrete, known-working llama-family (text) and Stable Diffusion (image) models for iOS, with install steps.
3. How to systematically find iOS/Xcode-friendly models on Hugging Face.

* * *

## 1. Why `andmev/Llama-3.1-8B-Instruct-CoreML` crashes on iPad

You are using:

- `andmev/Llama-3.1-8B-Instruct-CoreML`, which contains a single large `llama_3.1_coreml.mlpackage` (~4.5 GB). ([Hugging Face](https://huggingface.co/andmev/Llama-3.1-8B-Instruct-CoreML/tree/bc5ecb35dc98b497d94588630a7a16bbc0842be1 "andmev/Llama-3.1-8B-Instruct-CoreML at ..."))

Your runtime error:

> `ios18.constexpr_blockwise_shift_scale → std::bad_cast` inside BNNS, then `EXC_BAD_ACCESS` while loading the Core ML model.

What this means, in plain terms:

- **Core ML 8 / iOS 18 added new “constexpr\_\*” ops** (like `constexpr_blockwise_shift_scale`) to support advanced quantization schemes (4-bit blockwise, palettization, etc.) and large-model optimizations. These are used at compile time by the Core ML “ML Program” backend before BNNS/ANE kernels are created.
- The `andmev` model is a **converted/quantized Llama-3.1-8B**. From the size and packaging, it is highly likely it uses such compression / blockwise-quantized weights so that 8B parameters fit in ~4.5GB. ([Hugging Face](https://huggingface.co/andmev/Llama-3.1-8B-Instruct-CoreML/tree/bc5ecb35dc98b497d94588630a7a16bbc0842be1 "andmev/Llama-3.1-8B-Instruct-CoreML at ..."))
- On your device / OS build, BNNS is hitting an internal bug while compiling or running that op; it throws `std::bad_cast` in Apple’s code, and you only see the resulting `EXC_BAD_ACCESS`. There’s nothing you can “fix” from Swift.

So: you’re not doing anything wrong in Xcode. That particular CoreML export is hitting a relatively fresh, iOS-18-only code path that is still fragile.

The practical consequence:

- You should **not treat `andmev/Llama-3.1-8B-Instruct-CoreML` as a reference of “how CoreML LLMs behave”**.

- Start from models that:

* * *

## 2. Working models for Xcode/iOS, and how to install them

I’ll split this into:

- Text LLMs (llama-family-ish)
- Stable Diffusion (image generation)

### 2.1 Text: good CoreML LLMs that actually work on Apple devices

#### 2.1.1 TinyLlama CoreML – small, simple, good for smoke tests

Hugging Face: `TKDKid1000/TinyLlama-1.1B-Chat-v0.3-CoreML` ([Hugging Face](https://huggingface.co/TKDKid1000/TinyLlama-1.1B-Chat-v0.3-CoreML "TKDKid1000/TinyLlama-1.1B-Chat-v0.3-CoreML"))

- What it is:

- Why it’s a good start:

**Install / use steps (Mac → Xcode → iPad)**

1. **Download the CoreML package (on your Mac)**

2. **Add `.mlpackage` to Xcode**

3. **Load the model in Swift**

4. **Add tokenization / decoding**

Once this works on your iPad M3, you know the basic Core ML runtime is healthy.

* * *

#### 2.1.2 Apple Mistral CoreML – strong 7B, officially maintained

Hugging Face: `apple/mistral-coreml` ([Hugging Face](https://huggingface.co/apple/mistral-coreml "apple/mistral-coreml"))

- What it is:

- Why it’s special:

**Install / use steps**

1. **Download a `.mlpackage`**

2. **Add the `.mlpackage` to Xcode** exactly as for TinyLlama.

3. **Use Swift Transformers or the sample chat app**

This is the “serious” baseline text model I’d pick for iOS once TinyLlama is working.

* * *

#### 2.1.3 Smaller Llama 3.x CoreML variants

If you want to stay closer to Llama instead of Mistral, look at smaller CoreML-ready Llama 3.2 models, for example:

- Model: `smpanaro/Llama-3.2-1B-Instruct-CoreML` (part of an “Apple Neural Engine LLMs” collection, optimized for ANE). ([Hugging Face](https://huggingface.co/apple/mistral-coreml/blame/refs%2Fpr%2F4/README.md "README.md · apple/mistral-coreml at refs/pr/4"))

These splits the model into multiple `.mlmodelc` components (embedding, main, head). They are more work to integrate (you must stitch them together in Swift), but they are explicitly designed for Core ML on Apple devices and are much lighter than 8B.

* * *

### 2.2 Stable Diffusion: CoreML models that work in iOS apps

For image generation, you are much better served by Apple’s official CoreML Stable Diffusion models.

#### 2.2.1 `apple/coreml-stable-diffusion-v1-5`

Hugging Face: `apple/coreml-stable-diffusion-v1-5` ([Hugging Face](https://huggingface.co/apple/coreml-stable-diffusion-v1-5 "apple/coreml-stable-diffusion-v1-5"))

- Contains several variants: `original`, `split_einsum`, each with `compiled` and `packages` subfolders.
- The `compiled` variants are ready-to-use `.mlpackage` combinations (UNet, VAE, text encoders) for Apple’s Swift Stable Diffusion pipeline. ([Hugging Face](https://huggingface.co/apple/coreml-stable-diffusion-v1-5 "apple/coreml-stable-diffusion-v1-5"))
- Apple’s blog post on “Stable Diffusion with Core ML on Apple Silicon” and the `apple/ml-stable-diffusion` GitHub repo show exactly how to use these in Swift on macOS and iOS. ([Hugging Face](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.3 "TinyLlama/TinyLlama-1.1B-Chat-v0.3"))

**Install / use steps**

1. **Download a compiled model**

2. **Add the downloaded folder to Xcode**

3. **Add the Swift pipeline**

4. **Minimal Swift usage**

* * *

## 3. How to find “Xcode/iOS-friendly” models on Hugging Face

Here is a practical approach to find models that already works well with Xcode/iOS on Hugging Face.

### 3.1 Use the model search filters

On the Hugging Face “Models” page:

1. In the **Library** filter, select **`coreml`**.

2. Optionally filter by:

The “Core ML” library tag means:

- The model is either directly a `.mlmodel/.mlpackage`, or
- There is explicit code / metadata on the card about converting to Core ML.

### 3.2 Look for CoreML-specific keywords on the model card

On a Hugging Face model page, scan for:

- “CoreML” or “Core ML” in:

- Concrete file names:

These tell you the model is already packaged in the format Xcode/CoreML can load directly.

### 3.3 Prefer repos with usage instructions or linked Swift code

Good signs on a model page:

- A “Usage” or “CoreML” section that explicitly mentions:

- Links to GitHub repositories or sample iOS/macOS apps:

If a model card just says “here is a `.mlpackage`” with no Swift instructions, it can still work, but you will have to explore the inputs/outputs yourself. Models with sample Swift code are much easier to get working quickly.

### 3.4 Prefer official or widely-used orgs

For iOS/Xcode, these orgs are good bets:

- `apple` (e.g. `apple/coreml-stable-diffusion-*`, `apple/mistral-coreml`) ([Hugging Face](https://huggingface.co/apple/coreml-stable-diffusion-v1-5 "apple/coreml-stable-diffusion-v1-5"))

- Well-known CoreML experimenters:

### 3.5 Avoid bleeding-edge conversions for now

Models like `andmev/Llama-3.1-8B-Instruct-CoreML` are valuable experiments, but:

- They may use the newest ML Program + quantization features (4-bit blockwise, palettization) that rely heavily on `constexpr_*` ops.
- iOS 18 support for those code paths is still maturing, which is exactly why you see `constexpr_blockwise_shift_scale` related crashes.

Until Apple documents that a specific large Llama 3.1 CoreML model is tested on iOS, you’re safer with:

- Smaller Llama 3.2 CoreML exports (1B/3B), or
- Officially supported CoreML conversions (Mistral 7B, SD models).

* * *

## 4. Putting it all together

To directly answer your questions:

1. **Working llama(text) / StableDiffusion(image) models with Xcode/iOS + install steps**

2. **How to find such models on Hugging Face**

* * *

### Short bullet summary

- Your crash with `andmev/Llama-3.1-8B-Instruct-CoreML` is due to a CoreML/BNNS bug in an advanced quantization op (`constexpr_blockwise_shift_scale`) on iOS 18, not a mistake in your Xcode code. ([Hugging Face](https://huggingface.co/andmev/Llama-3.1-8B-Instruct-CoreML/tree/bc5ecb35dc98b497d94588630a7a16bbc0842be1 "andmev/Llama-3.1-8B-Instruct-CoreML at ..."))

- For **text LLMs** that work well in iOS apps:

- For **Stable Diffusion** :

- To find more models:

---

<div class="post-metadata">

**Author:** ![Dren77](https://avatars.discourse-cdn.com/v4/letter/d/c77e96/32.png) [@Dren77](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDren77)%5C
**Post date:** [December 3, 2025, 3:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060%2F3 "2025-12-03T03:12:04Z")

</div>

Thanks so much. I appreciate the effort.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [December 3, 2025, 3:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-ai-models-in-xcode-ios%2F171060%2F4 "2025-12-03T15:12:26Z")

</div>

Closed on December 3, 2025, 3:12pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.
