Steps to reproduce
The webrtc decoded remote H264 video is distorted if the remote stream is attached to two <video> elements.
The sample code is something like this:
remoteVideoElement1.srcObject = remoteVideoStream;
remoteVideoElement2.srcObject = remoteVideoStream;
Local browser: Microsoft Edge 40.15063.0.0
Remote browser: Microsoft Edge 40.15063.0.0, also with Chrome 61
CPU: Pentium N3540, also with i7-4770k
API: ORTC + webrtc-adapter 5.0.1, also with Webrtc 1.0 API
If the remoteVideoStream is assigned only to one <video> element then the video looks good.
If the remoteVideoStream is assigned to two <video> elements then it is distorted.
If the remoteVideoStream is assigned to two <video> elements and the browser is using only one CPU core (set with taskmanager CPU affinity) then video looks good.
If the codec is X-H264UC instead of H264 then the video looks good.
See the attached screenshots.
Comments and activity
- Microsoft Edge Team
Changed Assigned To to “Steven K.”
Can you provide a simplified repro for this? That will speed up my testing and make it easier to make certain we are running the same testing.
Also, was your testing done all locally? i.e. your internet connection bandwidth and latency should not be an issue?
Appreciate the help,
Also, wanted to verify that the video stream appears pixelated and not distorted, E.g. the dimension are different or the aspect ratio is changed.
I will try to create a simplified app that reproduces the problem.
The testing is not done locally. It uses a real internet connection and TURN server.
The bandwidth is not an issue. The video is 720p and it run at 2Mbps.
Dimensions and the aspect ratio are no changed.
Just changing the Edge process the use one CPU core while the connection is active causes the video to become clear. Changing it back to all CPU cores while the connection is active causes the video to become bad again.
Here is a simple app that reproduces the problem. All testing is done locally. Also I have attached a video pc1_edge.mp4. It shows what I see on my monitor.
This samples works fine.
I clonned and modified it at:
- webrtc-adapter is removed. This way the samples uses H264 otherwise it uses X-H264UC
- Added remoteVideo2. This causes the video to become bad.
The test machine is:
CPU: Pentium N3540
Webcam: Logitech HD Pro C910
The problem happens on other machines also, not just this one.
I can confirm that H264 decoder is not working even with a single video element on multicore machine which has hardware H264 decoder.
I imagine that WebRTC 1.0 API is implemented internally on top of ORTC API? Or is that not the caee?
Is it possible to enable X-H264UC codec in the WebRTC 1.0 API? That way at least Edge-to-Edge calls will work fine.
BTW. This also affects video calls between Edge and Safari 11 as Apple only support H.264 in Safari 11.
Essentially the H.264 decoder/encoder implementation for WebRTC in Edge is completely broken.