Flutter SDK

The applooma_rtc package adds voice, video, live streaming and audio rooms to a Flutter app. You create an engine, join a channel with a token from your own server, and render video widgets.

Requirements#

Installation#

Add the package to your pubspec.yaml.

dependencies:
  flutter:
    sdk: flutter
  applooma_rtc: ^0.3.14

Then fetch it.

flutter pub get

Android setup

Declare the permissions in android/app/src/main/AndroidManifest.xml.

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />

On Android 6.0 and above, CAMERA and RECORD_AUDIO are runtime permissions. Declaring them is not enough — request them from the user before you join a channel with camera or microphone enabled, for example with the permission_handler package.

Bluetooth headsets. On Android 12 and above, BLUETOOTH_CONNECT is a runtime permission too, and without it audio never reaches a Bluetooth headset. Request Permission.bluetoothConnecttogether with the microphone (older Android grants it without a dialog) and treat a refusal as non-fatal — the call continues on the phone's own speaker or earpiece. A connected headset is preferred automatically; engine.setSpeakerphone(bool)only chooses between the phone's outputs when no headset is present. On iOS nothing extra is needed. The UIKit screens ask for it for you.

iOS setup

Add the two usage descriptions to ios/Runner/Info.plist. iOS terminates the app if a camera or microphone is used without them.

<key>NSCameraUsageDescription</key>
<string>Used for video calls.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Used for voice and video calls.</string>

A minimal call screen#

The example below fetches a token from your own server, joins, renders remote video, and leaves. Your server holds the API secret; the app never does.

import 'dart:async';
import 'dart:convert';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
import 'package:applooma_rtc/applooma_rtc.dart';

class CallScreen extends StatefulWidget {
  const CallScreen({super.key, required this.channel});
  final String channel;

  @override
  State<CallScreen> createState() => _CallScreenState();
}

class _CallScreenState extends State<CallScreen> {
  final AppEngine _engine = AppEngine.create(appId: 'YOUR_APP_ID');
  final List<StreamSubscription> _subs = [];

  @override
  void initState() {
    super.initState();
    // A user is announced before their camera track arrives, so rebuild on
    // both events. Rendering only on onUserJoined shows an empty tile.
    _subs.add(_engine.onUserJoined.listen((_) => setState(() {})));
    _subs.add(_engine.onTrackSubscribed.listen((_) => setState(() {})));
    _subs.add(_engine.onUserLeft.listen((_) => setState(() {})));
    _join();
  }

  Future<void> _join() async {
    final res = await http.post(
      Uri.parse('https://your-server.com/api/rtc-token'),
      headers: {'Content-Type': 'application/json'},
      body: jsonEncode({'channel': widget.channel}),
    );
    final data = jsonDecode(res.body) as Map<String, dynamic>;

    await _engine.joinChannel(
      token: data['token'] as String,
      wsUrl: data['wsUrl'] as String,
      options: const AppJoinOptions(
        role: AppRole.host,
        camera: true,
        microphone: true,
      ),
    );
    if (mounted) setState(() {});
  }

  @override
  void dispose() {
    for (final s in _subs) {
      s.cancel();
    }
    _engine.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Stack(
        children: [
          GridView.count(
            crossAxisCount: 2,
            children: [
              for (final user in _engine.remoteUsers)
                AppVideoView.remote(user, key: ValueKey(user.uid)),
            ],
          ),
          Positioned(
            right: 16,
            bottom: 16,
            width: 120,
            height: 160,
            child: AppVideoView.local(_engine),
          ),
        ],
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () async {
          await _engine.leaveChannel();
          if (mounted) Navigator.of(context).pop();
        },
        child: const Icon(Icons.call_end),
      ),
    );
  }
}

AppEngine#

AppEngine is the entry point. Create one per call screen.

Creating an engine

final engine = AppEngine.create(appId: 'YOUR_APP_ID');

// A live stream or a voice room with music: loudspeaker, music-grade audio,
// 720p with the phone's hardware encoder.
final engine = AppEngine.create(
  appId: 'YOUR_APP_ID',
  options: const AppEngineOptions(
    audioScenario: AppAudioScenario.media,
    video: AppVideoConfig(height: 720, fps: 30),
  ),
);

Throws ArgumentError if appId is empty. The App ID is safe to ship in your app; the API secret is not.

AppEngineOptions

AppVideoConfig.codec defaults to AppVideoCodec.auto: H.264 on iOS and VP8 on Android, with a VP8 backup layer so a viewer that cannot decode the main stream is served VP8 by the server instead of a black frame. AppVideoCodec.h265 (since 0.3.3) is published on iOS — hardware on every iPhone since the 7 — and falls back to auto on Android; AppVideoCodec.av1 falls back to auto on both, with a warning. engine.publishedCodec tells you what was chosen.

AppVideoConfig.degradation (since 0.3.3) is what the encoder gives up first under pressure: AppVideoDegradation.auto keeps the frame rate and drops resolution, keepResolution keeps the picture sharp and drops frames, keepFramerate, balanced.

Video modes (AppVideoMode)

mode sets the overall tuning of your camera. Any field you set yourself (height, maxBitrate, simulcast, degradation) still wins over the mode. Since 0.3.4.

4K is a premium mode. It needs a strong network at both ends (20 Mbps or more upload for the sender and download for each viewer is recommended), a device whose encoder can do 3840×2160 — checkAppEngine.isUltraHdSupported() first — and it costs more battery and heat than HD. On a device that fails the check the SDK uses AppVideoMode.stableHd instead and logs a warning.

// Default: nothing to set — AppVideoMode.stableHd.
final engine = AppEngine.create(appId: 'YOUR_APP_ID');

// Premium 4K where the device can do it.
final video = AppEngine.isUltraHdSupported()
    ? const AppVideoConfig(mode: AppVideoMode.ultraHd4k)
    : const AppVideoConfig();

// The earlier behaviour (three layers, receivers switch by view size).
const adaptive = AppVideoConfig(mode: AppVideoMode.adaptive);

Getting crisp 1080p

AppVideoMode.stableHd is now the default, so a plain engine already keeps the resolution and sends one steady 1080p layer where the encoder is hardware (720p on a software encoder). Publish from a strong connection, and watch onVideoQualityChanged: reason bandwidthmeans the uplink is the limit, the CPU reason means the device is — drop to 720p then. To force 1080p on a device the tier would put on 720p, set height and maxBitrate explicitly as below.

final engine = AppEngine.create(
  appId: 'YOUR_APP_ID',
  options: const AppEngineOptions(
    video: AppVideoConfig(
      height: 1080, fps: 30, maxBitrate: 4000000,
      degradation: AppVideoDegradation.keepResolution,
    ),
  ),
);
engine.onVideoQualityChanged.listen((q) => debugPrint('$q'));

Methods

Properties

Events#

Every event is a broadcast Stream. Subscribe before you join so that participants already in the channel are not missed, and cancel the subscriptions when the widget is disposed.

final sub = engine.onTrackSubscribed.listen((user) {
  setState(() {});           // user.videoTrack is now available
});

// later
await sub.cancel();

Automatic codec recovery#

A stream that viewers receive but cannot show is healed by the SDK by itself. There is nothing to turn on. Since 0.3.5.

• When your app shows someone’s camera and it arrives but no picture can be decoded for more than six seconds while their camera is on, the SDK tells that publisher (and only them) with a small internal message. Internal messages never reach your message or data callbacks.
• On the publishing side, reports from two different viewers within 20 seconds (or from the only other person in a one-to-one call) make the SDK republish your camera on a safer path: VP8 if it was on another codec, otherwise once more as a single layer. Viewers see about half a second of interruption; audio is not touched. This happens at most once per join.
• The platform also learns from anonymous device reports (make and model, OS and SDK version, the codec and whether it worked; no user, identity or IP address) and tunes video settings per device model, such as the codec, a resolution or frame rate cap, or a single layer.

engine.onVideoCodecFallback.listen((e) {
  // e.from = 'h264', e.to = 'vp8', e.reason = AppVideoCodecFallbackReason.viewersCannotDecode
  debugPrint('camera switched ${e.from} -> ${e.to} (${e.reason.value})');
});

AppVideoCodecFallbackReason is noFrames, encoderError or viewersCannotDecode (wire values no_frames, encoder_error, viewers_cannot_decode). The stream is for your logs and support tooling; the app has nothing to do.

Rendering video#

AppVideoView is a plain widget with two factories. Give remote tiles a ValueKey(user.uid)so Flutter does not reuse one user's renderer for another.

// Local camera preview. Mirrored, as users expect of a self-view.
AppVideoView.local(engine)

// A remote user.
AppVideoView.remote(user, key: ValueKey(user.uid))

// Contain instead of the default cover.
AppVideoView.remote(user, fit: BoxFit.contain)

When there is no track yet the widget renders an empty box of zero size, so a tile with no video collapses rather than showing black. Wrap it in a sized container if you want a fixed placeholder.

Types#

AppJoinOptions

AppRole

AppConnectionState

connecting, connected, reconnecting, disconnected.

AppRemoteUser

AppGiftEvent

Delivered on onGiftReceived when your server sends a gift with POST /v1/gifts/send.

Lifecycle and cleanup#

leaveChannel ends the call but keeps the engine usable. When the screen goes away, cancel every stream subscription and then call dispose, which leaves the channel and releases the media resources. An engine that is not disposed keeps the camera and microphone open.

@override
void dispose() {
  for (final s in _subs) {
    s.cancel();
  }
  _engine.dispose();
  super.dispose();
}

Data messages#

sendData broadcasts bytes to everyone in the channel. Use it for chat and custom signalling.

await engine.sendData(utf8.encode(jsonEncode({'text': 'hello'})));

engine.onDataReceived.listen((event) {
  final (bytes, from) = event;
  final text = utf8.decode(bytes);
  debugPrint('${from?.uid ?? 'server'}: $text');
});

Remote diagnostics#

So AppLooma support can look into a problem on your users' devices without asking them to reproduce it, the SDK keeps the SDK's own recent log lines (join, connection state, reconnects, codec and codec recovery) and a stats snapshot every 10 s in memory. They are uploaded, at most about once per call when you leave or are disconnected (and once 60 s into a call), only when:

• the call had a problem: a codec fallback, a viewer reporting no picture, an encoder failure, a failed join, or repeated reconnects;
• AppLooma support turned log collection on for this device model or your project (for a limited time);
• your app asked for it, for example from a "Report a problem" button.

engine.uploadDiagnostics('user_report');

Never collected: audio, video, screen content, chat or data messages, display names, metadata or attribute values. User ids are replaced by a short one-way hash, tokens and URL parameters are removed, and only lines the SDK itself produced are included. Uploads are gzip-compressed. If the phone is offline, the latest upload is kept in memory and sent at the next join in the same app session. Uploads are stored for 14 days and then deleted. Since 0.3.6.

To turn every upload off, set AppEngineOptions.remoteDiagnostics:

final engine = AppEngine.create(appId: 'YOUR_APP_ID', options: const AppEngineOptions(remoteDiagnostics: false));

Smart routing#

joinChannel asks AppLooma which media server is nearest the device and connects there automatically. There is no backend change to make: keep passing the token and URL your server returns. A room stays on one server, so everyone in it meets in the same place even when they join from far apart. It only applies to applooma.dev URLs; any other URL is used as given. Since 0.3.14.

To connect to exactly the URL you passed, opt out:

await engine.joinChannel(
  token: token,
  wsUrl: wsUrl,
  options: const AppJoinOptions(role: AppRole.host, camera: true, smartRouting: false),
);

Network test#

startNetworkTest(serverUrl, token, callback) opens a short, separate test connection (about 5 s, 10 s at most), then calls back once with an AppNetworkTestResult: uplinkQuality and downlinkQuality (AppNetworkQuality.excellent / good / poor / lost / unknown), rttMs (-1 when unknown), jitterMs, uplinkLoss and downlinkLoss (0..1), and error (null on success). Mint the token for a throwaway channel such as nettest-<uid>. The test never touches the channel you are in, but it gives the cleanest numbers before joinChannel. stopNetworkTest() cancels it without a callback. Since 0.3.6.

await engine.startNetworkTest(wsUrl, testToken, (r) {
  if (r.error != null) return showError(r.error!);
  debugPrint('up=${r.uplinkQuality.name} rtt=${r.rttMs}ms');
});

Echo test

There is no echo test in the Flutter SDK: recording the microphone and playing it back needs an audio recorder plugin the package does not depend on. Use the network test above, then a short test call.

Network fallback#

Choose what to give up when the network is weak with an AppFallbackOption: none (default), videoLowQuality or audioOnly. Your own connection quality at poor or lost for 4 s falls back; good or excellent for 10 s restores. Since 0.3.6.

• setRemoteSubscribeFallback(option) — remote video. videoLowQuality asks for the low layer of every remote camera (best effort: only publishers sending several layers have one, and in AppVideoMode.adaptive a smaller view still wins). audioOnly stops receiving remote video; cameras your app turned off stay off on recovery.
• setLocalPublishFallback(option) — your camera. videoLowQuality lowers its bitrate cap to 30 % (at least 300 kbps); audioOnly pauses the camera (it stays published).

engine.setRemoteSubscribeFallback(AppFallbackOption.audioOnly);
engine.setLocalPublishFallback(AppFallbackOption.videoLowQuality);
engine.onFallbackStateChanged.listen((s) {
  // s.isLocal: your camera (true) or remote video (false)
  // s.state: the option in effect; none = restored
});

Restrictive networks#

Some office, hotel and school networks block UDP and every port but 443. The cloud proxy carries the call through AppLooma's relay over TLS on port 443 so it still connects. Set it with AppEngineOptions.cloudProxy:

• AppCloudProxy.auto (default): connect directly; if that join fails with a connection or network timeout, retry once through the relay on TLS 443. A refused token is not retried.
• forceTls443: always relay on TLS 443. Use it when you already know the network is locked down; it adds a little latency.
• off: never relay.

The onProxyStateChanged stream of AppProxyStateChange (state, autoRetry) reports the state: direct (not relayed, also after leaving), connecting (joining through the relay) and connected (in the channel through the relay), with a flag that is true when the relay is used because the direct join failed. The SDK also logs the retry. Since 0.3.7.

final engine = AppEngine.create(appId: 'YOUR_APP_ID',
    options: const AppEngineOptions(cloudProxy: AppCloudProxy.forceTls443));
engine.onProxyStateChanged.listen((c) {
  if (c.state == AppProxyState.connected) showBadge('Connected through the cloud proxy');
});

Device tuning#

AppLooma can tune video per phone model from the server (codec, height, frame rate, bitrate cap, simulcast off). The Flutter SDK fetches this at join in the background and applies it when the camera is published (it only lowers what your AppVideoConfig asks for). The answer is kept in memory, so the first join after a cold start uses it only if it arrived within a second. Choosing a software encoder per phone is not possible from Flutter. Since 0.3.7.

Video enhancement#

await engine.setVideoEnhance(options) turns on built-in camera enhancement: skin smoothing (beauty 0 to 1), lowLight boost, sharpen (0 to 1) and warmth (-1 cooler to 1 warmer). It runs on the capture path, so your own preview and what viewers receive are both enhanced. It survives camera off/on, camera switches and republishes; set it before the camera is on and it applies as soon as the camera starts. Pass null to turn it off. engine.videoEnhance reads the options in force. Android and iOS; on other platforms the call returns false. Rebuild the app (not a hot reload) after upgrading, since this adds native code. Since 0.3.9.

await engine.setVideoEnhance(AppVideoEnhance.portrait); // beauty 0.5 + low light
await engine.setVideoEnhance(const AppVideoEnhance(beauty: 0.3, sharpen: 0.4, warmth: 0.2));
await engine.setVideoEnhance(null); // off

Audience latency#

An audience member does not talk back, so a little extra delay buys smoother playback on a bad network. AppEngineOptions.audienceLatency sets the tier; it only applies while the local user is audience (publishing nothing) and switches back as soon as they go on stage. Since 0.3.8.

• AppAudienceLatency.ultraLow (default): the lowest delay, the behaviour before this option.
• AppAudienceLatency.low: same as ultraLow on Flutter: no receive-buffer control is reachable here.
• AppAudienceLatency.standard: prefers lower video layers under congestion: turns on setRemoteSubscribeFallback(AppFallbackOption.videoLowQuality) unless you set a fallback yourself.

AppEngine.create(appId: appId, options: const AppEngineOptions(audienceLatency: AppAudienceLatency.standard));

Clear Voice#

AppAudioOptions.noiseSuppressionMode chooses the noise suppression mode:standard (default) is the standard suppressor,clear is Clear Voice, off turns suppression off. Since 0.3.8.

clear adds a stronger spectral suppressor on the microphone's post-processing stage, after the standard one: steady noise (fans, hiss, hum, traffic) is removed far more strongly while speech passes. It adds about 10 ms of latency. Android and iOS since 0.3.9 (before that, clear ran the standard suppressor only); on other platforms it acts like standard.

const AppEngineOptions(audio: AppAudioOptions(noiseSuppressionMode: AppNoiseSuppression.clear));

Codec detection#

await AppEngine.getSupportedVideoCodecs() lists the video codecs this device can send and receive, each as mime, hardware, encoder anddecoder. The lists come from the media engine. hardware is true for H.264 and H.265, which are only offered on a hardware encoder; VP8, VP9 and AV1 report false because the SDK cannot tell from Flutter. Since 0.3.8.

final codecs = await AppEngine.getSupportedVideoCodecs();

Token renewal#

onTokenWillExpire fires about 30 s before the join token's exp (at once if less remains at join). Fetch a new token and pass it to renewToken(token), which re-arms the event. Since 0.3.6.

engine.onTokenWillExpire.listen((_) async {
  engine.renewToken(await myServer.fetchToken(channel));
});

Troubleshooting#

A remote tile stays blank

You are building tiles on onUserJoined only. The user is announced before their camera track is subscribed, so videoTrack is still null at that moment. Rebuild on onTrackSubscribed as well.

Camera or microphone never turns on

On Android 6.0 and above the permission must be granted at runtime, not only declared in the manifest. On iOS a missing NSCameraUsageDescription or NSMicrophoneUsageDescription terminates the app the moment the device is used. Request the permissions before joinChannel.

Audience members cannot publish

A token issued with the audience role carries no publish rights. Setting camera: true in AppJoinOptions does nothing for that role. To let a viewer speak, have your server issue a new token with cohost, then rejoin with it.

The other participant disappears when a second device joins

Two connections that share one identity are treated as the same participant, and the newer connection replaces the older one. Give every participant a unique identity when your server issues the token.

StateError: Already in a channel

joinChannel throws if the engine is already connected. Call leaveChannel first, or guard the call so a rebuild cannot trigger a second join.

The join is rejected

Channel names must match /^[a-zA-Z0-9_-]{1,64}$/. Tokens also expire, so fetch a fresh one for each join rather than caching it across sessions.

The other side hears themselves

You are on the loudspeaker without the phone's echo canceller. Use AppAudioScenario.media: it keeps the loudspeaker and still runs the session in the mode where the canceller lives.

What changed#

0.3.14

• Smart routing: joinChannel connects to the nearest AppLooma media server for the device; the room stays on one server. Only applies to applooma.dev URLs, with a 2 s limit and a silent fallback to the wsUrl you passed. Opt out with AppJoinOptions(smartRouting: false). See Smart routing.

0.3.13

• Errors from joinChannel, enableCamera, enableMicrophone, switchCamera, enableScreenShare, sendData and sendMessage are now thrown as AppLoomaException (message, code, cause). ArgumentError/StateError for API misuse are unchanged.
• Neutral error and log text throughout.

0.3.12

• Android: joining no longer fails with a null audio device module error.
• Android: on some phones the local preview stays blank under Flutter's Impeller renderer. Until that is fixed, turn Impeller off in AndroidManifest.xml (see Android setup).

0.3.11

• AppVideoMode.stableHd now degrades balanced by default, so a live stream is smooth from its first seconds instead of starting at a few frames per second while bandwidth ramps up. ultra-hd-4k still keeps resolution, and an explicit degradation setting is honoured.

0.3.10

• Cloud proxy auto-retry no longer reports a disconnect to the app while it switches to the relay.
• UIKit 0.1.10 requires 0.3.10; no UIKit API changes.

0.3.9

• Video Enhance (setVideoEnhance, videoEnhance) and Clear Voice (AppNoiseSuppression.clear) now work on Android and iOS — see the sections above. The package now contains native code: do a full rebuild (and pod install on iOS) after upgrading.
• Requires the fixed core release 2.12.1, which makes iOS builds compile.

0.3.8

• setVideoEnhance and Clear Voice modes added for API parity — see the sections above.
• Audience latency tiers (audienceLatency) — see the sections above.
• Codec detection (AppEngine.getSupportedVideoCodecs()) — see the sections above.

0.3.7

• Cloud proxy for restrictive networks (AppEngineOptions.cloudProxy, onProxyStateChanged) and server device tuning applied at camera publish — see the sections above.

0.3.6

• Remote diagnostics (uploadDiagnostics, AppEngineOptions.remoteDiagnostics), startNetworkTest, network fallback and token renewal — see the sections above.

0.3.5

• Automatic codec recovery: a camera that viewers cannot show is republished on a safer path by itself. New onVideoCodecFallback stream.

0.3.3

• AppVideoConfig.degradation — AppVideoDegradation.keepResolution / keepFramerate / balanced / auto; minBitrate (reserved).
• Default caps raised: 4 Mbps at 1080p, 2.2 Mbps at 720p when maxBitrate is 0. New height: 1440 preset (2560×1440, 5 Mbps, 30 fps).
• AppVideoCodec.h265 (iOS) and av1 (falls back to auto), with the VP8 backup layer kept.
• onVideoQualityChanged — what your own camera is sending (size, fps, bitrate, layer, reason).

0.3.2

• Android: AppVideoCodec.auto publishes VP8 (hardware or software) so MediaTek and other phones without an engine-supported H.264 encoder never publish a black camera. Pass AppVideoCodec.h264 explicitly on verified devices.

0.3.1

• switchCamera() flips between the front and back cameras; the local preview follows.

0.3.0

• AppEngineOptions — audioScenario, video (codec: auto publishes H.264 with a VP8 backup layer), audio. engine.publishedCodec.
• Every publisher always sends and every joiner receives all tracks — fixes co-hosts seeing a black frame on flaky links.
• The media scenario keeps the phone's echo canceller engaged on the loudspeaker.
• AppVideoView.local follows the camera through restarts; onLocalVideoChanged.
• onRemoteStats, onFirstRemoteVideoFrame, onConnectionStage.
• Guardrails: maxBitrate under 10 000 logs a units warning; a non-preset height snaps to the nearest preset with a warning.

0.2.0

• sendMessage, onMessage, audience, hosts. HD by default.

Next steps#