Ask HN: FFmpeg real-time desktop streaming
Hello HN, In the university I'm working in, they have asked me to produce a remote desktop stream for students to connect to and watch the professor coding on his desktop. I would like to use FFMPEG with the latest AVI1 capabilities since we have some gpus (it would be great to use them). 'They' would like a native solution (testing me). I'm looking for information where I can learn to do that elegantly.
Thanks HN, much love What latency are you trying to do? Will the professor being communicating with the students while doing this? Will the students all have the same bandwidth, or will you want multiple renditions (low, med, high quality levels)? If you want AV1 you will not be able to use RTMP. The protocol is orphaned/deprecated, so avoid if possible! If I was building it this is what I would do, and my reasoning. * For capture + encoding I would use OBS. You will want to use something that is easy for users to install configure. Professors will also have lots of custom requirements when it comes to layout etc... it will be tempting to do a ffmpeg command directly, but it will fall apart quick I believe. * To get AV1 out of OBS I would use FFMPEG output. I would have it send RTP. RTP is used to carry video in a sub-second manner. This is the same protocol that WebRTC uses. You know have AV1 + low latency. * Then for users to watch I would use WebRTC. That will allow them to watch in their web browser. Conceptually it will be like this https://github.com/pion/webrtc/tree/master/examples/rtp-to-w... this takes the RTP packets and puts them in the browser. Lots of great projects exist that you could use for 'RTP -> WebRTC' like https://galene.org/ and https://livekit.io/ I would suggest checking them all out! If you have more questions/want to talk to people in the video space always happy to chat on https://pion.ly/slack :) OME [1] is a RTMP server for sub-second WebRTP streaming. Profiles allow for specific encodes if needed. I have a repository hosting an exemplary configuration and a frontend using their player [2]. Another server is SRS [3], but IMO it is more difficult to use, lacks features and the delay is about 1-3s — it requires far less bandwidth though. [1] https://github.com/AirenSoft/OvenMediaEngine
[2] https://github.com/hashworks/simple-ome
[3] https://github.com/ossrs/srs I would also recommend OBS as another commentator has mentioned. I believe in both cases you will need an RTSP server to handle multiple client connections. I looked into a similar ffmpeg solution a couple of months ago and broadcast an libx264 video only stream with ~1 second latency to an rtsp server: https://gist.github.com/andrewmackrodt/88c2233fb9cc4797ada93... However, I found this solution to be too CPU intensive as I was unsure if/how to instruct x11grab to interact with the GPU directly. Perhaps if I used my nvidia card for encoding it would have been more performant. However, OBS makes this entire process much easier. If stuck for a server I've used this in the past. nginx with an RTMP module. Also linking a config file I commented for an example. Unsure if it'll still work and it could definitely be simplified for OP's needs. It was a proof of concept for a streaming service I was thinking up before realising how much money I don't have, haha https://github.com/arut/nginx-rtmp-module https://gist.github.com/cohan/7f676d3f561be62d0550785c015f00... I had been doing Twitch streaming seven years ago (so, RTSP, if I recall correctly) with an ffmpeg script I wrote. https://github.com/tcarrio/ffmpeg-streamcast/blob/master/bas... Also, seven years, and I eventually ended up moving to OBS instead of diving even further, but that was working at the time. However, it won't give you any pointers for using AV1, and this is just the streaming source, you'll still need an RTSP server and clients to access them. I'm not sure if this is what you are looking for, but have you tired https://obsproject.com/ Warning: It has too many options, so it takes a while to get the correct configuration. Also, the difference between "Stopped" and "Recording" is too subtle, so I had to modify the layout style to make it foolproof for me. Check out Galene: I don't think it uses ffmpeg, but it may have the features you are looking for I was able to use fauxstream as a starting point for part of my streaming project. My experiment was streaming via a ipfs transport layer. fauxstream is a very thin wrapper around ffmpeg, it proved invaluable in providing a reference command I could iterate off of to get my stream working. https://github.com/rfht/fauxstream edit: it is hard to tell what op wants but it could be as simple(hah, the ffmpeg command line terrifies me) as stream caster:
ffmpeg -video_size 1280x720 -f x11grab -i :0 -r 30 -c:v libx264 -vb 3500k -minrate 3500k -maxrate 3500k -bufsize 7000k -preset ultrafast -vf format=yuv420p -g 60 -keyint_min 30 -f flv - | nc -l 1234 stream viewer:
nc stream_caster 1234 | ffplay - If you need real time, eg. Teacher interaction in chat with the students I recommend WebRTC. Try to take a look at something like LiveKit, or if you need a little more low level control Pion. Experiment with gstreamer as well. If you're after very low latency, then try Mjpeg. Most modern compressions are temporal, there will always be a several frames lag. For the streaming client I wouldn't reinvent the wheel and go with OBS Studio. Now do you have a streaming server, where the students will connect to? If not you can put something simple together with nginx and RTMP. You should realize that you are describing two separate problems: OBS + nginx-rtmp-module VLC is another option. Do you have streaming server ? ,Or just want encoder suggestions
Which one are you trying to solve? * how to create content for a stream
* how to broadcast that stream to viewers