Skip to content

Latest commit

 

History

History
333 lines (233 loc) · 9.06 KB

README.md

File metadata and controls

333 lines (233 loc) · 9.06 KB
Raw

Koala

GitHub release GitHub

Maven Central npm CocoaPods PyPI

Made in Vancouver, Canada by Picovoice

Twitter URL YouTube Channel Views

Koala is an on-device noise suppression engine. Koala is:

  • Private; All voice processing runs locally.
  • Cross-Platform:
    • Linux (x86_64), macOS (x86_64, arm64), Windows (x86_64)
    • Android and iOS
    • Chrome, Safari, Firefox, and Edge
    • Raspberry Pi (3, 4, 5)

Table of Contents

AccessKey

AccessKey is your authentication and authorization token for deploying Picovoice SDKs, including Koala. Anyone who is using Picovoice needs to have a valid AccessKey. You must keep your AccessKey secret. You would need internet connectivity to validate your AccessKey with Picovoice license servers even though the noise suppression is running 100% offline.

AccessKey also verifies that your usage is within the limits of your account. Everyone who signs up for Picovoice Console receives the Free Tier usage rights described here. If you wish to increase your limits, you can purchase a subscription plan.

Demos

Python Demos

Install the demo package:

pip3 install pvkoalademo
koala_demo_mic --access_key ${ACCESS_KEY} --output_path ${WAV_OUTPUT_PATH}
koala_demo_file \
    --access_key ${ACCESS_KEY} \
    --input_path ${WAV_INPUT_PATH} \
    --output_path ${WAV_OUTPUT_PATH}

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

Android Demo

Using Android Studio, open demo/android/Activity as an Android project and then run the application.

Replace "${YOUR_ACCESS_KEY_HERE}" in the file MainActivity.java with your AccessKey.

iOS Demo

Copy your AccessKey into the ACCESS_KEY variable inside ViewModel.swift.

Before building the demo app, run the following from KoalaDemo directory to install the Koala-iOS CocoaPod:

pod install

Open KoalaDemo.xcworkspace and run the demo.

C Demos

Build the demo:

cmake -S demo/c/ -B demo/c/build && cmake --build demo/c/build --target koala_demo_mic

To list the available audio input devices:

./demo/c/build/koala_demo_mic -s

To run the demo:

./demo/c/build/koala_demo_mic -l ${LIBRARY_PATH} -m ${MODEL_PATH} -a ${ACCESS_KEY} -o ${WAV_OUTPUT_PATH}

Replace ${LIBRARY_PATH} with path to appropriate library available under lib, ${MODEL_PATH} with path to the model file available under lib/common, ${ACCESS_KEY} with AccessKey obtained from Picovoice Console, and ${WAV_OUTPUT_PATH} with a path to a .wav file where the enhanced audio will be stored. Terminate the demo with Ctrl+C.

For more information about C demos go to demo/c.

Web Demo

From demo/web run the following in the terminal:

yarn
yarn start

(or)

npm install
npm run start

Open http://localhost:5000 in your browser to try the demo.

SDKs

Python

Install the Python SDK:

pip3 install pvkoala

Create an instance of the engine and enhance audio in real-time:

import pvkoala

koala = pvkoala.create(access_key='${ACCESS_KEY}')

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

def get_next_audio_frame():
    pass

while True:
    enhanced_audio = koala.process(get_next_audio_frame())

Finally, when done be sure to explicitly release the resources using koala.delete().

Android

To include the package in your Android project, ensure you have included mavenCentral() in your top-level build.gradle file and then add the following to your app's build.gradle:

dependencies {
    implementation 'ai.picovoice:koala-android:${LATEST_VERSION}'
}

Create an instance of the engine and enhance audio in real-time:

import ai.picovoice.koala.*;

final String accessKey = "${ACCESS_KEY}"; // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)

short[] getNextAudioFrame() {
    // .. get audioFrame
    return audioFrame;
}

try {
    Koala koala = new Koala.Builder()
        .setAccessKey(accessKey)
        .build(appContext);

    while true {
        short[] enhancedFrame = koala.process(getNextAudioFrame());
    };

} catch (KoalaException ex) { }

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

iOS

Create an instance of the engine and enhance audio:

import Koala

do {
  let koala = try Koala(accessKey: "${ACCESS_KEY}")
} catch {}

func getNextAudioFrame() -> [Int16] {
  // .. get a frame of audio
  return audioFrame;
}

while true {
  do {
    let enhancedAudio = try koala.process(getNextAudioFrame())
    // .. use enhanced audio
  } catch {}
}

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console.

In case the next audio frame does not directly follow the previous one, call koala.reset().

When done be sure to explicitly release the resources using koala.delete().

C

include/pv_koala.h header file contains relevant information. Build an instance of the object:

    pv_koala_t *handle = NULL;
    const char *model_path = "${MODEL_PATH}";
    pv_status_t status = pv_koala_init(${ACCESS_KEY}, model_path, &handle);
    if (status != PV_STATUS_SUCCESS) {
        // error handling logic
    }

Replace ${ACCESS_KEY} with the AccessKey obtained from Picovoice Console, and ${MODEL_PATH} with the path to the model file available under lib/common.

Now the handle can be used to enhance audio in real-time:

extern const int16_t *get_next_audio_frame(void);

const int32_t frame_length = pv_koala_frame_length();
int16_t *enhanced_pcm = (int16_t *) malloc(frame_length * sizeof(int16_t));

while (true) {
    const int16_t *pcm = get_next_audio_frame();
    const pv_status_t status = pv_koala_process(handle, pcm, enhanced_pcm);
    if (status != PV_STATUS_SUCCESS) {
        // error handling logic
    }
}

Finally, when done be sure to release the acquired resources:

pv_koala_delete(handle);

Web

Install the web SDK using yarn:

yarn add @picovoice/koala-web

or using npm:

npm install --save @picovoice/koala-web

Create an instance of the engine using KoalaWorker and enhance audio in real-time:

import { Koala } from "@picovoice/koala-web";
import koalaParams from "${PATH_TO_BASE64_KOALA_PARAMS}";

function processCallback(enhancedPcm) {
  // do something with enhancedPcm
}

function getAudioData(): Int16Array {
... // function to get audio data
  return new Int16Array();
}

const koala = await KoalaWorker.create(
  "${ACCESS_KEY}",
  processCallback,
  { base64: koalaParams },
);

await koala.reset();
for (;;) {
    await koala.process(getAudioData());
}

Replace ${ACCESS_KEY} with yours obtained from Picovoice Console. Finally, when done release the resources using koala.release().

Releases

v2.0.0 - November 24th, 2023

  • Improvements to error reporting
  • Upgrades to authorization and authentication system
  • Various bug fixes and improvements
  • Web min support bumped to Node 16
  • iOS support bumped to iOS 13

v1.0.0 - February 7th, 2023

  • Initial release