Aiden/add streaming demo

[aiden-ct]

Adds a streaming demo to the repo.

[aiden-ct]

This patch set adds a streaming demo to the repo. If there is interest in merging this, I will work on integrating my forked signalling server into the existing one.

[TheExDeus]

When running your code it stops at "Status: Sending SDP answer" and doesn't show anything.
Running web server with "python3 -m http.server".
Then signaling server with " ./streaming-signalling-server.py".
And media server with "./streaming-app".

Webserver log:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
192.168.88.253 - - [24/Aug/2019 20:50:02] "GET /webrtc-demo.html HTTP/1.1" 200 -
192.168.88.253 - - [24/Aug/2019 20:50:02] "GET /webrtc-demo.js HTTP/1.1" 200 -
192.168.88.253 - - [24/Aug/2019 20:50:02] code 404, message File not found
192.168.88.253 - - [24/Aug/2019 20:50:02] "GET /favicon.ico HTTP/1.1" 404 -

Streaming server log:

Listening on http://0.0.0.0:8443
Connected to ('127.0.0.1', 47280)
register_peer: REGISTER MEDIA
Registered media server at ('127.0.0.1', 47280)
Connected to ('192.168.88.253', 50673)
register_peer: REGISTER CLIENT
BIND 18 -> media-server: BIND-SESSION-CLIENT 18
Registered client 18 at ('192.168.88.253', 50673)
connection_handler: SESSION 18 BOUND
SESSION MESSAGE: SESSION 18 BOUND
connection_handler: {"command":{"type":"connect-to-mountpoint","data":null},"client_uid":18}
18 -> media-server: {"command":{"type":"connect-to-mountpoint","data":null},"client_uid":18}
connection_handler: {"command":{"type":"connect-to-mountpoint","data":null},"client_uid":18,"success":true}
FWD MESSAGE: {"command":{"type":"connect-to-mountpoint","data":null},"client_uid":18,"success":true}
media-server -> 18: {"command":{"type":"connect-to-mountpoint","data":null},"client_uid":18,"success":true}
connection_handler: {"sdp":{"type":"offer","sdp":"v=0\r\no=- 302095516542658815 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS webrtcbin_18\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:9k5gW5ffmorAd094x9VthRMPvbcR5Y40\r\na=ice-pwd:52Xwy6SLvq9Lpzx9wrSwbs7i1Th6tXrP\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 nack pli\r\na=framerate:30\r\na=fmtp:96 packetization-mode=1;sprop-parameter-sets=Z0JAKJWgUH5A,aM48gA==\r\na=ssrc:577534030 msid:user1319489148@host-51998378 webrtctransceiver0\r\na=ssrc:577534030 cname:user1319489148@host-51998378\r\na=mid:video0\r\na=fingerprint:sha-256 85:B3:BB:6E:2C:31:F6:AB:C3:D9:C4:09:A8:5D:42:96:A2:5C:1B:7F:68:B9:7A:EE:CC:F0:22:8C:15:A8:96:23\r\n"},"client_uid":18}
FWD MESSAGE: {"sdp":{"type":"offer","sdp":"v=0\r\no=- 302095516542658815 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS webrtcbin_18\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:9k5gW5ffmorAd094x9VthRMPvbcR5Y40\r\na=ice-pwd:52Xwy6SLvq9Lpzx9wrSwbs7i1Th6tXrP\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 nack pli\r\na=framerate:30\r\na=fmtp:96 packetization-mode=1;sprop-parameter-sets=Z0JAKJWgUH5A,aM48gA==\r\na=ssrc:577534030 msid:user1319489148@host-51998378 webrtctransceiver0\r\na=ssrc:577534030 cname:user1319489148@host-51998378\r\na=mid:video0\r\na=fingerprint:sha-256 85:B3:BB:6E:2C:31:F6:AB:C3:D9:C4:09:A8:5D:42:96:A2:5C:1B:7F:68:B9:7A:EE:CC:F0:22:8C:15:A8:96:23\r\n"},"client_uid":18}
media-server -> 18: {"sdp":{"type":"offer","sdp":"v=0\r\no=- 302095516542658815 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS webrtcbin_18\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:9k5gW5ffmorAd094x9VthRMPvbcR5Y40\r\na=ice-pwd:52Xwy6SLvq9Lpzx9wrSwbs7i1Th6tXrP\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 H264/90000\r\na=rtcp-fb:96 nack pli\r\na=framerate:30\r\na=fmtp:96 packetization-mode=1;sprop-parameter-sets=Z0JAKJWgUH5A,aM48gA==\r\na=ssrc:577534030 msid:user1319489148@host-51998378 webrtctransceiver0\r\na=ssrc:577534030 cname:user1319489148@host-51998378\r\na=mid:video0\r\na=fingerprint:sha-256 85:B3:BB:6E:2C:31:F6:AB:C3:D9:C4:09:A8:5D:42:96:A2:5C:1B:7F:68:B9:7A:EE:CC:F0:22:8C:15:A8:96:23\r\n"},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:1 1 UDP 2013266431 fe80::ea18:94e:a4d7:6b1b 33090 typ host","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:1 1 UDP 2013266431 fe80::ea18:94e:a4d7:6b1b 33090 typ host","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:1 1 UDP 2013266431 fe80::ea18:94e:a4d7:6b1b 33090 typ host","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:2 1 TCP 1015022847 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:2 1 TCP 1015022847 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:2 1 TCP 1015022847 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:3 1 TCP 1010828543 fe80::ea18:94e:a4d7:6b1b 41319 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:3 1 TCP 1010828543 fe80::ea18:94e:a4d7:6b1b 41319 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:3 1 TCP 1010828543 fe80::ea18:94e:a4d7:6b1b 41319 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:4 1 UDP 2013266430 192.168.88.172 34499 typ host","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:4 1 UDP 2013266430 192.168.88.172 34499 typ host","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:4 1 UDP 2013266430 192.168.88.172 34499 typ host","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:5 1 TCP 1015022079 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:5 1 TCP 1015022079 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:5 1 TCP 1015022079 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:6 1 TCP 1010827775 192.168.88.172 45781 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:6 1 TCP 1010827775 192.168.88.172 45781 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:6 1 TCP 1010827775 192.168.88.172 45781 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:1 2 UDP 2013266430 fe80::ea18:94e:a4d7:6b1b 43716 typ host","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:1 2 UDP 2013266430 fe80::ea18:94e:a4d7:6b1b 43716 typ host","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:1 2 UDP 2013266430 fe80::ea18:94e:a4d7:6b1b 43716 typ host","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:2 2 TCP 1015022846 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:2 2 TCP 1015022846 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:2 2 TCP 1015022846 fe80::ea18:94e:a4d7:6b1b 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:3 2 TCP 1010828542 fe80::ea18:94e:a4d7:6b1b 40375 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:3 2 TCP 1010828542 fe80::ea18:94e:a4d7:6b1b 40375 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:3 2 TCP 1010828542 fe80::ea18:94e:a4d7:6b1b 40375 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:4 2 UDP 2013266429 192.168.88.172 40538 typ host","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:4 2 UDP 2013266429 192.168.88.172 40538 typ host","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:4 2 UDP 2013266429 192.168.88.172 40538 typ host","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:5 2 TCP 1015022078 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:5 2 TCP 1015022078 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:5 2 TCP 1015022078 192.168.88.172 9 typ host tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:6 2 TCP 1010827774 192.168.88.172 56717 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:6 2 TCP 1010827774 192.168.88.172 56717 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:6 2 TCP 1010827774 192.168.88.172 56717 typ host tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:7 2 UDP 1677722110 87.110.122.102 40538 typ srflx raddr 192.168.88.172 rport 40538","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:7 2 UDP 1677722110 87.110.122.102 40538 typ srflx raddr 192.168.88.172 rport 40538","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:7 2 UDP 1677722110 87.110.122.102 40538 typ srflx raddr 192.168.88.172 rport 40538","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:7 1 UDP 1677722111 87.110.122.102 34499 typ srflx raddr 192.168.88.172 rport 34499","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:7 1 UDP 1677722111 87.110.122.102 34499 typ srflx raddr 192.168.88.172 rport 34499","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:7 1 UDP 1677722111 87.110.122.102 34499 typ srflx raddr 192.168.88.172 rport 34499","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:8 1 TCP 847249919 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:8 1 TCP 847249919 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:8 1 TCP 847249919 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:9 1 TCP 843055615 87.110.122.102 45781 typ srflx raddr 192.168.88.172 rport 45781 tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:9 1 TCP 843055615 87.110.122.102 45781 typ srflx raddr 192.168.88.172 rport 45781 tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:9 1 TCP 843055615 87.110.122.102 45781 typ srflx raddr 192.168.88.172 rport 45781 tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:8 2 TCP 847249918 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:8 2 TCP 847249918 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:8 2 TCP 847249918 87.110.122.102 9 typ srflx raddr 192.168.88.172 rport 9 tcptype active","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"ice":{"candidate":"candidate:9 2 TCP 843055614 87.110.122.102 56717 typ srflx raddr 192.168.88.172 rport 56717 tcptype passive","sdpMLineIndex":0},"client_uid":18}
FWD MESSAGE: {"ice":{"candidate":"candidate:9 2 TCP 843055614 87.110.122.102 56717 typ srflx raddr 192.168.88.172 rport 56717 tcptype passive","sdpMLineIndex":0},"client_uid":18}
media-server -> 18: {"ice":{"candidate":"candidate:9 2 TCP 843055614 87.110.122.102 56717 typ srflx raddr 192.168.88.172 rport 56717 tcptype passive","sdpMLineIndex":0},"client_uid":18}
connection_handler: {"sdp":{"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-68.0.2 4704704663412458319 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=sendrecv\r\na=fingerprint:sha-256 F8:0B:E1:08:BF:5D:94:59:D2:39:6E:8E:BB:56:36:CF:A0:74:2F:51:06:CD:77:56:4B:06:47:0B:BA:F7:AC:6E\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=video 0 UDP/TLS/RTP/SAVPF 120\r\nc=IN IP4 0.0.0.0\r\na=inactive\r\na=mid:video0\r\na=rtpmap:120 VP8/90000\r\n"},"client_uid":18}
18 -> media-server: {"sdp":{"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-68.0.2 4704704663412458319 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=sendrecv\r\na=fingerprint:sha-256 F8:0B:E1:08:BF:5D:94:59:D2:39:6E:8E:BB:56:36:CF:A0:74:2F:51:06:CD:77:56:4B:06:47:0B:BA:F7:AC:6E\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=video 0 UDP/TLS/RTP/SAVPF 120\r\nc=IN IP4 0.0.0.0\r\na=inactive\r\na=mid:video0\r\na=rtpmap:120 VP8/90000\r\n"},"client_uid":18}
Sending keepalive ping to ('127.0.0.1', 47280) in recv
Sending keepalive ping to ('192.168.88.253', 50673) in recv

Media server log:

0:00:00.084219625 10444 0x55888acf60 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/zpiic/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Connecting to server...
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:00.107363726 10444 0x55888b8800 WARN omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<video_encoder> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40

GET HTTP/1.1
Soup-Debug-Timestamp: 1566668993
Soup-Debug: SoupSession 1 (0x5588910220), SoupMessage 1 (0x558892c8e0), SoupSocket 1 (0x55886ad6d0)
Host: localhost:8443
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: WnxA/TtsEzRyXx9jJXTJQg==
Sec-WebSocket-Version: 13
Accept-Encoding: gzip, deflate

< HTTP/1.1 101 Switching Protocols
< Soup-Debug-Timestamp: 1566668993
< Soup-Debug: SoupMessage 1 (0x558892c8e0)
< Upgrade: websocket
< Connection: Upgrade
< Sec-WebSocket-Accept: QLRn2uA7TlUa+apiPQ2SDWhJmI4=
< Date: Sat, 24 Aug 2019 17:49:53 GMT
< Server: Python/3.6 websockets/8.0.2

Connected to signalling server
Registering with signalling server
Registered with server
0:00:13.626211912 10444 0x5588d39a80 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc1:src could not send sticky events
0:00:13.626312644 10444 0x5588d39a30 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nicesrc2:src could not send sticky events
Received SDP answer:
v=0
o=mozilla...THIS_IS_SDPARTA-68.0.2 4704704663412458319 0 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 F8:0B:E1:08:BF:5D:94:59:D2:39:6E:8E:BB:56:36:CF:A0:74:2F:51:06:CD:77:56:4B:06:47:0B:BA:F7:AC:6E
a=ice-options:trickle
a=msid-semantic:WMS *
m=video 0 UDP/TLS/RTP/SAVPF 120
c=IN IP4 0.0.0.0
a=inactive
a=mid:video0
a=rtpmap:120 VP8/90000

0:00:13.681411696 10444 0x5588f278f0 ERROR webrtcbin gstwebrtcbin.c:2803:_set_description_task:<webrtcbin_18> media 0 is missing or contains an empty 'ice-ufrag' attribute

I have gst_debug=2 enabled so it does show an error about "media 0 is missing". Running on Jetson Nano so it uses nvenc for encoder. The default in your code is h264 and I don't know if SDP response "a=rtpmap:120 VP8/90000" is correct in this case. All connections between the media, client and signaling seems to work. It just gets stuck waiting on something.

Doesn't work in either Firefox 68 or Chrome 76.