How to display/change stream preview (Stream just started. Get in Here)

[sfeed1095]

I always get this when streaming something and it never changes

Is there a way to set a preview like in actual streams?

[longnguyen2004]

We haven't figured out how that works yet, so that's a no for now.
Also, even if we figured it out, setting a stream preview based on the actual video input is going to require decoding the video stream, and I'm not sure if I'm willing to do that yet. It'd require a libav.js build that has the 5 decoders for the codecs that we support.

[sfeed1095]

Setting the exact live stream frame as preview sounds quite complex and likely unnecessary. If it turns out to be possible, it would be great to have the option to set any static content as a preview, such as by providing an image_url or an image_path, but yeah I don't know neither how that works lol

[longnguyen2004]

Static image is definitely easier, since the image only needs to be decoded once, and can be done using something like sharp. Just need to figure out how the image is sent.

[longnguyen2004]

Stream preview is fetched by the client through a normal API endpoint (see this repo for more info). If the client also sends the image through a normal endpoint then it should be easy

[sfeed1095]

Setting this up via a standard API endpoint is pretty straightforward. Does the Discord-video-stream library stores the stream_key somewhere in a public variable where it could be retrieved if I wanted to make the request myself? If not, I could just wait for a PR in the future that includes a full feature to set the thumbnail.

[longnguyen2004]

Well...It doesn't seem to be that easy
- Web clients cannot send stream previews, only desktop clients
- DevTools on the desktop client also doesn't reveal any API request or WebSocket message related to sending stream previews

Both of these suggests that it's another proprietary thing in Discord's UDP protocol, which would take more effort to reverse engineer. I can't give you an estimate yet, maybe it'll come one day, maybe not.

All of the above is incorrect, see follow-up comments for more details.

[sfeed1095]

I tried setting up Burp (proxy) in my web browser and start a stream. The stream preview would get updated and I would see this request being captured:

POST /api/v9/streams/guild%3A1319695495202869391%3A1319695495672762381%3A770167875113320468/preview HTTP/2
Host: discord.com
Cookie: <skipped>
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json
Authorization: <skipped>
X-Super-Properties: <skipped>
Content-Length: 37019
Origin: https://discord.com
Referer: https://discord.com/channels/1319695495202869391/1319695495672762380
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Te: trailers

{"thumbnail":"data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQA......<skipped>"}

Isn't pretty much just this or did I miss something?

[longnguyen2004]

Oh, well that's a lot easier. Seems like it's the voice module that makes the request, not the Electron client, which would explain why it didn't show up in DevTools when I checked. Thanks for the finding.

EDIT: For some reason I still can't capture the POST request like yours, I can only capture the GET request from the client requesting the preview. I'll try to look further

EDIT EDIT: I have stream preview off :)...Got the POST request now

[sfeed1095]

Excellent.
I tried it out sending the request from my end, it seems to work.

Example:

import axios from 'axios';
import fetch from 'node-fetch';
import sharp from 'sharp';

async function imageToBase64(imageUrl) {
  const res = await fetch(imageUrl);
  const buffer = await res.buffer();
  const base64Image = await sharp(buffer)
    .resize(1000)
    .toBuffer()
    .then(data => data.toString('base64'));

  return `data:image/jpeg;base64,${base64Image}`;
}

async function sendPreviewRequest(stream_key, image_url, token) {
  try {
    const base64Image = await imageToBase64(image_url);
    const data = { thumbnail: base64Image };
    
    await axios.post(`https://discord.com/api/v9/streams/${stream_key}/preview`, data, {
      headers: {
        'Authorization': token,
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0',
      },
    });
  } catch (error) {
    // Handle error
  }
}

export { sendPreviewRequest };

Where stream_key has the format: guild:{guild_id}:{channel_id}:{self_bot_user_id}

Then calling the function and sending the request after streamer.createStream(..)

await streamer.joinVoice("guild_id", "channel_id");
const udp = await streamer.createStream({
    height:1080,
    width:1920,
    fps: 30,
    bitrateKbps: 4000,
    videoCodec: "H264",
    h26xPreset: "faster"
});

udp.mediaConnection.setSpeaking(true);
udp.mediaConnection.setVideoStatus(true);

const stream_key = `guild:<guild_id>:<channel_id>:<user_id>`;
const image_url = 'https://cdn.donmai.us/sample/80/3a/__suzumiya_haruhi_kyon_kyonko_and_suzumiya_haruhiko_suzumiya_haruhi_no_yuuutsu_drawn_by_hengebako_and_taiki_6240taiki__sample-803aa5c039298191e0217b0df83e156a.jpg';
const token = '<token>';

sendPreviewRequest(stream_key, image_url, token);

try {
    const cancellableCommand = await streamLivestreamVideo("<stream_url>", udp);
    const result = await cancellableCommand;
} catch (e) {
    console.log(e);
} finally {
    udp.mediaConnection.setSpeaking(false);
    udp.mediaConnection.setVideoStatus(false);
}