CactusBrain Signal

What it does

• Tagging. Give it a clip of audio and it returns labels for what it sounds like, each with a score — a dog barking, a smoke alarm, running water.
• Enhancement. Clean up speech from noisy audio. It rides the same signal capability as tagging, because both are “run a model over sampled audio”.
• Personalization. Turn a sound into a vector and hand it to CactusBrain Personalize, so a sound becomes just another thing a user can prefer.

Install on iOS

// Package.swift
dependencies: [
  .package(url: "https://github.com/cactusbrain/cactusbrain-swift", from: "0.1.0")
]

// Then add the product to your target:
.product(name: "CactusBrainSignalSDK", package: "cactusbrain-swift")

CactusBrainSignalSDK provides the audio types, session, and nativeconfigure entry point through one developer-facing module.

Configure and tag

import CactusBrainSignalSDK

let session = try await CactusBrainSignalSDK.configure(
    projectKey: Secrets.cactusBrainProjectKey
)

// A clip from a file, or PCM you captured yourself.
let result = try await session.tags(for: .file(recordingURL))

for tag in result.tags {
    print(tag.label, tag.score)
}

A SignalTag carries a label and a score; the result also carries timing metrics so you can see how long inference took. Pass SignalTagOptions to configure to cap the number of tags or set a score threshold.

Install on Android

// build.gradle.kts
dependencies {
    implementation("ai.cactusbrain:cactusbrain-signal:0.1.0")
}

import ai.cactusbrain.signal.CactusBrainSignal

val session = CactusBrainSignal.configure(
    context = applicationContext,
    projectKey = BuildConfig.CACTUSBRAIN_PROJECT_KEY,
)

// captureFormat() reports the rate and length the model needs.
val format = session.captureFormat()
val input = SignalInput.Pcm(
    samples = frames,
    sampleRate = format.sampleRate,
)

val result = session.tags(input)
result.tags.forEach { tag -> println("${tag.label} ${tag.score}") }

configure does no network work, so it is safe to call at launch on the main thread. Activation and the first download happen on the first prepare or tags call.

Capture at the rate the model needs

Signal does not resample, mix channels or decode media for you. Ask the loaded model for its captureFormat and open your recorder to match — a microphone opened at a guessed sample rate is the one mistake a tagger cannot recover from.

// Ask the model what geometry it needs, then capture to match.
// Guessing a sample rate is the one mistake a tagger cannot recover from.
let format = try await session.captureFormat
let input = SignalInput.pcm(
    samples: frames,
    sampleRate: format.sampleRate,
    channels: 1
)

let result = try await session.tags(for: input)

captureFormat also reports minimumSamples, so you can wait for a long enough clip before tagging rather than scoring silence.

Prepare before first use

The tagging model is small by the standards of the other modalities, but a first tags call on a device that has not installed it will still download it first. Prepare it behind a spinner where you can:

// The tagging model loads on first use. Prepare it where you can show
// progress rather than letting it happen inside the first tag.
session.prepare(SignalFeature.TAG).collect { event ->
    when (event) {
        is PreparationEvent.Progress -> setProgress(event.fraction)
        is PreparationEvent.Installed -> enableMicrophone()
        is PreparationEvent.Failed -> showFailure(event.cause)
    }
}

Preparation is the same delivery path documented for Vision: the server decides entitlement, the artifact is verified before install, and a model that fails verification is discarded rather than installed.

Offline and privacy

Once the model is installed, tagging runs with no network. Audio and results stay on the device — there is no per-clip server call, so we could not read them even if we wanted to.

See Security and model delivery for how entitlement, revocation and integrity verification work. They are identical across Signal, Text and Vision.

Known limitations

• Tagging reports what a sound resembles from a fixed label set. It is not transcription, and it does not identify a specific speaker or a specific song.
• Enhancement exposes streaming enhance(chunk:) on iOS today; the Android module ships tagging first.
• The native runtime is arm64-v8a / arm64 only, so an x86_64 emulator or Intel-Mac simulator will not run it.