iOS SDK

The AppLoomaRTC Swift module adds voice, video, live streaming and audio rooms to an iOS app. You create an engine, join a channel with a token from your own server, and render video in an AppVideoView.

Requirements#

Installation#

The SDK is a Swift package, published at github.com/AppLooma-RTC/iOS-SDK.

https://github.com/AppLooma-RTC/iOS-SDK

dependencies: [
    .package(url: "https://github.com/AppLooma-RTC/iOS-SDK", from: "0.6.1")
],
targets: [
    .target(
        name: "YourApp",
        dependencies: [
            .product(name: "AppLoomaRTC", package: "iOS-SDK")
        ]
    )
]

Info.plist keys

iOS terminates the app the moment the camera or microphone is used without a usage description. Add both keys before your first call, even if you only plan to use audio.

<key>NSCameraUsageDescription</key>
<string>Used for video calls.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Used for voice and video calls.</string>

The strings are shown to the user in the permission prompt, so write them for the user rather than for yourself. To keep audio running while the app is in the background, also enable the Audio, AirPlay, and Picture in Picturebackground mode in your target's Signing & Capabilities tab.

Bluetooth headsets need no extra permission or plist key: the SDK configures the audio session with Bluetooth allowed (HFP for headsets with a microphone, A2DP for listen-only) and a connected headset is preferred automatically, then wired headphones, then the loudspeaker (.media) or the earpiece (.call). For an in-call route picker use engine.audioRoutes, engine.setAudioRoute(_:) and the delegate's audioRouteChanged; engine.setSpeakerphone(_:)switches between the phone's own outputs when no headset is connected.

A minimal call screen#

The view controller below fetches a token from your own server, joins, renders remote video and leaves. Your server holds the API secret; the app never does. See Quickstart for the server side.

import UIKit
import AppLoomaRTC

final class CallViewController: UIViewController, AppEngineDelegate {

    private var engine: AppEngine!
    private var views: [String: AppVideoView] = [:]
    private let localView = AppVideoView()

    override func viewDidLoad() {
        super.viewDidLoad()
        view.addSubview(localView)          // position with your own constraints

        engine = AppEngine.create(appId: "YOUR_APP_ID", delegate: self)

        Task {
            do {
                // Token comes from YOUR server.
                let res = try await api.rtcToken(channel: "lobby")
                try await engine.joinChannel(
                    token: res.token,
                    wsUrl: res.wsUrl,
                    options: AppJoinOptions(role: .host, camera: true, microphone: true)
                )
                localView.attachLocal(engine: engine)
            } catch {
                print("join failed: \(error)")
            }
        }
    }

    // A user is announced BEFORE their camera track arrives, so the tile
    // created here has nothing to render yet.
    func appEngine(_ engine: AppEngine, userJoined user: AppRemoteUser) {
        let tile = AppVideoView()
        views[user.uid] = tile
        view.addSubview(tile)
    }

    // The track is ready — this is where video actually starts.
    func appEngine(_ engine: AppEngine, trackSubscribedFor user: AppRemoteUser) {
        views[user.uid]?.attach(user: user)
    }

    func appEngine(_ engine: AppEngine, userLeft user: AppRemoteUser) {
        views.removeValue(forKey: user.uid)?.removeFromSuperview()
    }

    func appEngine(_ engine: AppEngine, connectionStateChanged state: AppConnectionState) {
        if state == .disconnected { dismiss(animated: true) }
    }

    private func hangUp() {
        Task {
            await engine.leaveChannel()
            dismiss(animated: true)
        }
    }
}

AppEngine#

AppEngine is the entry point. Create one per call screen.

let engine = AppEngine.create(appId: "YOUR_APP_ID", delegate: self)

// A live stream or a voice room with music: loudspeaker, music-grade audio,
// 720p with the hardware encoder.
let engine = AppEngine.create(
    appId: "YOUR_APP_ID",
    delegate: self,
    options: AppEngineOptions(
        audioScenario: .media,
        video: AppVideoConfig(height: 720, fps: 30)
    )
)

The delegate is optional and held weakly, so keep a strong reference to whatever object you pass. create traps at runtime if the App ID is empty.

AppEngineOptions

AppVideoConfig.codec defaults to .auto: H.264, which every iPhone encodes in hardware, with a VP8 backup layer so a viewer that cannot decode the H.264 stream is served VP8 by the server instead of a black frame. .h265 (since 0.5.1) is hardware on every iPhone since the 7 and gives the same picture in roughly 30 % fewer bits; it keeps the VP8 backup layer. .av1 falls back to .auto with a warning — no iPhone encodes AV1 in hardware. engine.publishedCodec tells you what was chosen.

AppVideoConfig.degradation (since 0.5.1) is what the encoder gives up first under pressure: .auto keeps the frame rate and drops resolution, .keepResolution keeps the picture sharp and drops frames, .keepFramerate, .balanced.

Video modes (AppVideoMode)

mode sets the overall tuning of your camera. Any field you set yourself (height, maxBitrate, simulcast, degradation) still wins over the mode. Since 0.5.2.

4K is a premium mode. It needs a strong network at both ends (20 Mbps or more upload for the sender and download for each viewer is recommended), a device whose encoder can do 3840×2160 — checkAppEngine.isUltraHdSupported() first — and it costs more battery and heat than HD. On a device that fails the check the SDK uses .stableHd instead and logs a warning.

// Default: nothing to set — .stableHd.
let engine = AppEngine.create(appId: "YOUR_APP_ID", delegate: self)

// Premium 4K.
let uhd = AppEngineOptions(video: AppVideoConfig(mode: .ultraHd4k))

// The earlier behaviour (three layers, receivers switch by view size).
let adaptive = AppEngineOptions(video: AppVideoConfig(mode: .adaptive))

Getting crisp 1080p

.stableHd is now the default, so a plain engine already keeps the resolution and sends one steady 1080p layer where the encoder is hardware (720p on a software encoder). Publish from a strong connection, and watch appEngine(_:videoQualityChanged:): reason .bandwidthmeans the uplink is the limit, the CPU reason means the device is — drop to 720p then. To force 1080p on a device the tier would put on 720p, set height and maxBitrate explicitly as below.

let engine = AppEngine.create(
    appId: "YOUR_APP_ID",
    delegate: self,
    options: AppEngineOptions(
        video: AppVideoConfig(height: 1080, fps: 30, maxBitrate: 4_000_000, degradation: .keepResolution)
    )
)

func appEngine(_ engine: AppEngine, videoQualityChanged q: AppVideoQualityInfo) {
    print(q.width, q.height, q.fps, q.bitrateKbps, q.layer, q.reason)
}

Methods

Properties

Events#

Conform to AppEngineDelegate. Every method has a default empty implementation, so implement only the ones you need. All callbacks are delivered on the main actor, so you may touch views directly.

Automatic codec recovery#

A stream that viewers receive but cannot show is healed by the SDK by itself. There is nothing to turn on. Since 0.5.4.

• When your app shows someone’s camera and it arrives but no picture can be decoded for more than six seconds while their camera is on, the SDK tells that publisher (and only them) with a small internal message. Internal messages never reach your message or data callbacks.
• On the publishing side, reports from two different viewers within 20 seconds (or from the only other person in a one-to-one call) make the SDK republish your camera on a safer path: VP8 if it was on another codec, otherwise once more as a single layer. Viewers see about half a second of interruption; audio is not touched. This happens at most once per join.
• The platform also learns from anonymous device reports (make and model, OS and SDK version, the codec and whether it worked; no user, identity or IP address) and tunes video settings per device model, such as the codec, a resolution or frame rate cap, or a single layer.

func appEngine(_ engine: AppEngine, videoCodecFallbackFrom from: String, to: String, reason: String) {
    // e.g. from = "h264", to = "vp8", reason = "viewers_cannot_decode"
    print("camera switched \(from) -> \(to) (\(reason))")
}

reason is no_frames, encoder_error or viewers_cannot_decode. The callback is for your logs and support tooling; the app has nothing to do.

Remote diagnostics#

So AppLooma support can look into a problem on your users' devices without asking them to reproduce it, the SDK keeps the SDK's own recent log lines (join, connection state, reconnects, codec and codec recovery) and a stats snapshot every 10 s in memory. They are uploaded, at most about once per call when you leave or are disconnected (and once 60 s into a call), only when:

• the call had a problem: a codec fallback, a viewer reporting no picture, an encoder failure, a failed join, or repeated reconnects;
• AppLooma support turned log collection on for this device model or your project (for a limited time);
• your app asked for it, for example from a "Report a problem" button.

engine.uploadDiagnostics(reason: "user_report")

Never collected: audio, video, screen content, chat or data messages, display names, metadata or attribute values. User ids are replaced by a short one-way hash, tokens and URL parameters are removed, and only lines the SDK itself produced are included. If the phone is offline, the latest upload is kept in memory and sent at the next join in the same app session. Uploads are stored for 14 days and then deleted. Since 0.5.5.

To turn every upload off, set AppEngineOptions.remoteDiagnostics:

let engine = AppEngine.create(appId: "YOUR_APP_ID", delegate: self, options: AppEngineOptions(remoteDiagnostics: false))

Smart routing#

joinChannel asks AppLooma which media server is nearest the device and connects there automatically. There is no backend change to make: keep passing the token and URL your server returns. A room stays on one server, so everyone in it meets in the same place even when they join from far apart. It only applies to applooma.dev URLs; any other URL is used as given. Since 0.6.1.

To connect to exactly the URL you passed, opt out:

AppJoinOptions(role: .host, camera: true, smartRouting: false)

Network test#

startNetworkTest(serverUrl:token:completion:) opens a short, separate connection (mint the token for a throwaway channel such as nettest-<uid>; nothing is published), samples the media server's quality grade for about 5 s and calls the completion once on the main thread with an AppNetworkTestResult. It gives up after 10 s with error set; stopNetworkTest() cancels it without a callback. The channel you are in is not touched. On iOS the result carries the quality only: rttMs and jitterMs are -1 and the loss fields 0. There is no echo test on iOS yet. Since 0.5.6.

engine.startNetworkTest(serverUrl: wsUrl, token: testToken) { result in
    if let error = result.error { print("test failed: \(error)"); return }
    print("uplink \(result.uplinkQuality), downlink \(result.downlinkQuality)")
}

Network fallback#

Choose what to give up when the network is weak with an AppFallbackOption: .none (default), .videoLowQuality or .audioOnly. The media server's quality grade for this device at poor or lost for about 4 s falls back; good or excellent for about 10 s restores. Since 0.5.6.

• setRemoteSubscribeFallback(_:) — remote video. .videoLowQuality asks for every remote camera's low layer (only where the sender publishes layers and adaptive stream is off); .audioOnly stops receiving remote video.
• setLocalPublishFallback(_:) — your camera. .videoLowQuality republishes it at 30 % of its bitrate (at least 300 kbps; viewers see a short gap), .audioOnly pauses it.

appEngine(_:fallbackStateChanged:isLocal:) reports each change; .none means restored.

Device tuning#

AppLooma can tune video per iPhone model from the server (codec, height, frame rate, bitrate cap, simulcast off, and codecs whose hardware encoder should be avoided, which on iOS moves H.265 to H.264 and H.264 to VP8). The last answer is kept on the device and applied at join; a fresh one is fetched in the background for the next join. It only lowers what your AppVideoConfig asks for. Since 0.5.6.

Restrictive networks#

Some office, hotel and school networks block UDP and every port but 443. The cloud proxy carries the call through AppLooma's relay over TLS on port 443 so it still connects. Set it with AppEngineOptions.cloudProxy:

• .auto (default): connect directly; if that join fails with a connection or network timeout, retry once through the relay on TLS 443. A refused token is not retried.
• .forceTls443: always relay on TLS 443. Use it when you already know the network is locked down; it adds a little latency.
• .off: never relay.

appEngine(_:proxyStateChanged:autoRetry:) reports the state: .direct (not relayed, also after leaving), .connecting (joining through the relay) and .connected (in the channel through the relay), with a flag that is true when the relay is used because the direct join failed. The SDK also logs the retry. Since 0.5.6.

let engine = AppEngine.create(appId: "YOUR_APP_ID", delegate: self,
                              options: AppEngineOptions(cloudProxy: .forceTls443))

func appEngine(_ engine: AppEngine, proxyStateChanged state: AppProxyState, autoRetry: Bool) {
    if state == .connected { showBadge("Connected through the cloud proxy") }
}

Video enhancement#

engine.setVideoEnhance(AppVideoEnhance(...)) improves your camera before it is sent, with Core Image on the capture path: beauty (0 to 1, noise-reduction smoothing with a slight lift), lowLight (exposure and shadow lift, adapting to the scene), sharpen (0 to 1) and warmth (-1 cooler to 1 warmer). Pass nil to turn it off. It follows the camera through off/on, switches and republishes. Since 0.5.7.

engine.setVideoEnhance(AppVideoEnhance(beauty: 0.6, lowLight: true))
engine.setVideoEnhance(.portrait)
engine.setVideoEnhance(nil) // off

Receivers and super-resolution

Enhancement runs on the sender, so every viewer sees the improved picture at no cost to them. Upscaling a low-resolution stream on the receiving phone (super-resolution) is not offered: doing it well needs machine-learning models that are too heavy for an in-call path on mid-range phones. For viewers on weak networks, use the existing receiver quality controls instead (network fallback and the video quality layers) so they get the best layer their connection carries.

Keep audio running in background#

A call or voice room stops as soon as the app leaves the screen unless the app declares background audio. Add UIBackgroundModes with audio (and voip if your app rings for incoming calls) to your Info.plist, or tick Background Modes → Audio, AirPlay, and Picture in Picture in Signing & Capabilities:

<key>UIBackgroundModes</key>
<array>
  <string>audio</string>
  <string>voip</string>
</array>

Only declare voip if you really handle calls: App Review rejects it otherwise.

Audience latency#

An audience member does not talk back, so a little extra delay buys smoother playback on a bad network. AppEngineOptions.audienceLatency sets the tier; it only applies while the local user is audience (publishing nothing) and switches back as soon as they go on stage. Since 0.5.7.

• .ultraLow (default): the lowest delay, the behaviour before this option.
• .low: same as .ultraLow on iOS: the engine here has no receive-buffer control.
• .standard: prefers lower video layers under congestion: turns on setRemoteSubscribeFallback(.videoLowQuality) unless you set a fallback yourself. No extra receive buffer on iOS.

let engine = AppEngine.create(appId: APP_ID, options: AppEngineOptions(audienceLatency: .standard))

Clear Voice#

AppAudioOptions.noiseSuppressionMode chooses the noise suppression mode:.standard (default) is the standard suppressor,.clear is Clear Voice, .off turns suppression off. Since 0.5.7.

On iOS, .clear runs the microphone through Apple's voice processing in the voice-chat mode in every audio scenario (the .media scenario otherwise uses the video-chat mode). The strongest noise removal on iOS is the system's Voice Isolation, which only the user can switch on: AppEngine.showMicrophoneModes() opens the picker (iOS 15+), and AppEngine.isVoiceIsolationActive tells you whether it is on.

AppEngineOptions(audio: AppAudioOptions(noiseSuppressionMode: .clear))
AppEngine.showMicrophoneModes()

Codec detection#

AppEngine.getSupportedVideoCodecs() lists the video codecs this device can send and receive, each as mime, hardware, encoder anddecoder. H.264 is always hardware; H.265 is listed when VideoToolbox can decode it in hardware, and as an encoder when the device can also export HEVC. VP8 and VP9 run in software. Since 0.5.7.

for c in AppEngine.getSupportedVideoCodecs() { print(c.mime, c.hardware) }

Token renewal#

About 30 seconds before the join token expires (immediately if less was left at join) the SDK calls tokenWillExpire. Fetch a fresh token from your server and hand it to renewToken. The live connection is not interrupted: while connected, its credential is refreshed by the media server, and the renewed token is used for the SDK’s own later calls. Calling renewToken re-arms the reminder; when not joined the token is just stored. Since 0.5.5.

func appEngine(_ engine: AppEngine, tokenWillExpire token: String) {
    Task {
        let fresh = try await myServer.fetchToken()
        engine.renewToken(fresh)
    }
}

Rendering video#

AppVideoView is a UIView subclass. Add it to your layout with your own constraints and point it at a track.

let tile = AppVideoView()

// A remote user.
tile.attach(user: user)

// The local camera preview. Sets isMirrored for you.
tile.attachLocal(engine: engine)

// Clear a tile.
tile.track = nil

Types#

AppJoinOptions

AppRole

AppConnectionState

.connecting, .connected, .reconnecting, .disconnected.

AppRemoteUser

AppGiftEvent

Delivered on appEngine(_:giftReceived:) when your server sends a gift with POST /v1/gifts/send. It is also parseable directly from a data payload with AppGiftEvent.tryParse(data), which returns nil when the payload is not a gift.

AppError

AppError.alreadyJoined — thrown by joinChannel when the engine is already in a channel.

Lifecycle and cleanup#

leaveChannel is the cleanup call on iOS. It disconnects, releases the camera and microphone, and clears the roster; the engine stays usable and can join again. There is no separate destroy method — drop your reference to the engine once you have left.

override func viewWillDisappear(_ animated: Bool) {
    super.viewWillDisappear(animated)
    Task { await engine.leaveChannel() }
}

• leaveChannel is async. Await it before you tear down the screen so the disconnect is sent.
• Clear each AppVideoView.track (or remove the view) when a user leaves, so a stale last frame does not stay on screen.
• The delegate is weak. If your delegate object is deallocated while the call runs, callbacks stop arriving silently.

Data messages#

try await engine.sendData(Data(#"{"text":"hello"}"#.utf8))

func appEngine(_ engine: AppEngine, dataReceived data: Data, from user: AppRemoteUser?) {
    let text = String(decoding: data, as: UTF8.self)
    print("\(user?.uid ?? "server"): \(text)")
}

Troubleshooting#

A remote tile stays black

You called attach(user:) inside appEngine(_:userJoined:). The user is announced before their camera track is subscribed, so videoTrack is still nil at that point. Attach in appEngine(_:trackSubscribedFor:).

The app crashes as the call starts

A missing NSCameraUsageDescription or NSMicrophoneUsageDescription terminates the app the moment the device is accessed. The crash log names the missing key. Add both to Info.plist.

Camera or microphone never turns on

The user denied the prompt, or denied it in an earlier build. iOS does not ask twice — the user must re-enable it in Settings. Handle the error thrown by enableCamera and enableMicrophone and show that instruction.

Audience members cannot publish

A token issued with the audience role carries no publish rights, and AppJoinOptions skips publishing entirely for that role. To let a viewer speak, have your server issue a new token with cohost, then rejoin with it.

The other participant disappears when a second device joins

Two connections that share one identity are treated as the same participant, and the newer connection replaces the older one. Give every participant a unique identity when your server issues the token.

AppError.alreadyJoined

joinChannel throws when the engine is already connected. Check engine.isJoined, or await engine.leaveChannel() first. A common cause is joining from viewDidAppear, which runs again each time the screen returns.

No callbacks arrive at all

The delegate is weak. If the only reference to your delegate object was the engine, it has been deallocated. Hold it yourself for the life of the call.

The join is rejected

Channel names must match /^[a-zA-Z0-9_-]{1,64}$/. Tokens also expire, so fetch a fresh one for each join rather than caching it across sessions.

The other side hears themselves

You are on the loudspeaker without the echo canceller. Use audioScenario: .media: it keeps the loudspeaker and still runs the session in playAndRecord + videoChat, where the canceller lives.

What changed#

0.6.1

• Smart routing: joinChannel connects to the nearest AppLooma media server for the device; the room stays on one server. Only applies to applooma.dev URLs, with a 2 s limit and a silent fallback to the URL you passed. Opt out with AppJoinOptions(smartRouting: false). See Smart routing.

0.6.0

• Camera frame processing for apps: AppEngine.setVideoProcessor(_:) with the new AppVideoFrameProcessor protocol. It runs on the local camera after the built-in enhancement, before encoding and the local preview, and survives camera off/on, switches and republishes. Looma Beauty Camera for iOS uses it.

0.5.10

• .stableHd now degrades .balanced by default, so a live stream is smooth from its first seconds instead of starting at a few frames per second while bandwidth ramps up. ultra-hd-4k still keeps resolution, and an explicit degradation setting is honoured.

0.5.9

• Cloud proxy auto-retry no longer reports a disconnect to the app while it switches to the relay.

0.5.8

• UIKit: an Enhance button (beauty + low light) in the call screen video controls and the live stream host controls.
• Docs: startNetworkTest explains why round-trip time, jitter and loss read -1 / -1 / 0 on iOS.

0.5.7

• Video enhancement (setVideoEnhance) with Core Image — see the sections above.
• Clear Voice (noiseSuppressionMode, Voice Isolation picker) — see the sections above.
• Audience latency tiers (audienceLatency) — see the sections above.
• Codec detection (AppEngine.getSupportedVideoCodecs()) — see the sections above.

0.5.6

• Cloud proxy for restrictive networks (AppEngineOptions.cloudProxy, appEngine(_:proxyStateChanged:autoRetry:)), pre-call network test (startNetworkTest), network fallback (setRemoteSubscribeFallback, setLocalPublishFallback, appEngine(_:fallbackStateChanged:isLocal:)) and server device tuning — see the sections above.

0.5.5

• Remote diagnostics (uploadDiagnostics(reason:), AppEngineOptions.remoteDiagnostics) and token renewal (appEngine(_:tokenWillExpire:), renewToken(_:)) — see the sections above.

0.5.4

• Automatic codec recovery: a camera that viewers cannot show is republished on a safer path by itself. New appEngine(_:videoCodecFallbackFrom:to:reason:) delegate method.

0.5.1

• AppVideoConfig.degradation — .keepResolution / .keepFramerate / .balanced / .auto; minBitrate (reserved).
• Default caps raised: 4 Mbps at 1080p, 2.2 Mbps at 720p when maxBitrate is 0. New height: 1440 preset (2560×1440, 5 Mbps, 30 fps).
• AppVideoCodec.h265 (hardware on every iPhone since the 7) and .av1 (falls back to auto), with the VP8 backup layer kept.
• appEngine(_:videoQualityChanged:) — what your own camera is sending (size, fps, bitrate, layer, reason).

0.4.0

• AppEngineOptions — audioScenario, video (codec: .auto publishes H.264 with a VP8 backup layer), audio. engine.publishedCodec.
• Every publisher always sends and every joiner receives all tracks — fixes co-hosts seeing a black frame on flaky links.
• The .media scenario keeps the echo canceller engaged on the loudspeaker.
• switchCamera(). Views attached with attachLocal follow the camera through restarts; appEngineLocalVideoChanged.
• appEngine(_:remoteStats:), appEngine(_:firstRemoteVideoFrameFor:elapsedMs:), appEngine(_:connectionStage:uid:elapsedMs:).
• Guardrails: maxBitrate under 10 000 logs a units warning; a non-preset height snaps to the nearest preset with a warning.

Next steps#