Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supporting bytes and strings as audio input to agents #1630

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

supporting bytes and strings as audio input to agents #1630

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d3c63ca
supporting bytes and strings as audio
alphamarket Dec 24, 2024
8865916
Merge branch 'main' into main
manthanguptaa Dec 30, 2024
2d9e1fb
Merge branch 'main' into main
dirkbrnd Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions phi/model/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import base64
import collections.abc

from types import GeneratorType
Expand Down Expand Up @@ -494,6 +495,14 @@ def add_audio_to_message(self, message: Message, audio: Optional[Any] = None) ->
if audio is None:
return message

# supporting raw audio data
if isinstance(audio, bytes):
audio = base64.b64encode(audio).decode("utf-8")

# supporting pre-formatted audio data
if isinstance(audio, str):
audio = {"data": audio, "format": "wav"}

# If `id` is in the audio, this means the audio is already processed
# This is used in multi-turn conversations
if "id" in audio:
Expand Down