On-Device AI Shift: Simple Prompting

1. The Problem: Privacy vs. Intelligence

Integrating LLMs usually forces a trade-off between user privacy and feature set, as cloud-based services require sending sensitive data to remote servers. Furthermore, these services incur network latency and external costs that can degrade the user experience.

2. Overview of Foundation Models

Apple’s Foundation Models framework, optimized for iOS 26, allows us to run a 3-billion parameter LLM directly on-device. This framework requires zero setup, zero costs, and ensures complete user privacy by keeping all data in RAM. While smaller than cloud models like ChatGPT, it excels at language processing and text generation tasks.

3. Step-by-Step Integration

To maintain a conversation’s context, you must persist your session object, typically within a @State property.

Initialize the Session:

import FoundationModels
import SwiftUI

@State private var session = LanguageModelSession()
@State private var responseText = ""

Execute a Prompt: Use the respond(to:) method for background tasks where you need the full string at once, or streamResponse(to:) for real-time UIs.

func generateContent(input: String) {
  Task {
    do {
      let response = try await session.respond(to: input)
      self.responseText = response.content
    } catch {
      print("Generation failed: \(error.localizedDescription)")
    }
  }
}

4. Pitfalls and Best Practices

• Token Limits: Every session has a "Context Window" of 4,096 tokens (roughly 3,000 words). If this cumulative limit is exceeded, the model throws an error.

• Prewarming: To avoid "cold start" delays, call session.prewarm() when the view appears to proactively load model resources into memory.

• Hallucination: These models predict tokens based on probability, not logic; they are notably poor at complex math and will assert incorrect facts confidently.

• Guardrails: Apple’s safety filters are extremely strict; even innocent prompts like "tell me a joke" can occasionally trigger a refusal or exception if repeated.

5. On-Device vs. Server Models

Choose on-device when privacy, offline availability, and speed are the priority. Opt for cloud models for deep abstract reasoning, complex math, or massive-scale world knowledge. A hybrid approach—using a cloud model for reasoning and an on-device model for parsing the result into native Swift objects—often provides the best results.

6. Sample Single-File Implementation:

import SwiftUI
import FoundationModels

struct IdeationAssistantView: View {
  @State private var session = LanguageModelSession()
  @State private var userInput = ""
  @State private var aiResponse = ""
  @State private var isProcessing = false
  
  var body: some View {
    NavigationStack {
      VStack {
        ScrollView {
          Text(.init(aiResponse))
            .padding()
        }
        
        HStack {
          TextField("Share an idea...", text: $userInput, axis: .vertical)
            .textFieldStyle(.roundedBorder)
          
          Button("Generate") {
            generateResponse()
          }
          .disabled(isProcessing || userInput.isEmpty)
        }
        .padding()
      }
      .navigationTitle("Ideation Partner")
      .onAppear {
        session.prewarm()
      }
    }
  }
  
  private func generateResponse() {
    isProcessing = true
    Task {
      do {
        let result = try await session.respond(to: userInput)
        aiResponse = result.content
      } catch LanguageModelSession.GenerationError.guardrailViolation {
        aiResponse = "Error: Prompt triggered safety filters."
      } catch {
        aiResponse = "Unexpected error: \(error.localizedDescription)"
      }
      isProcessing = false
      userInput = ""
    }
  }
}