H.264 streams are missing sane profile and level hints and extradata/avcC
|Assignee:||John Törnblom||% Done:|
|Found in version:||head||Affected Versions:|
I have played with tvheadend by using XBMC on ATV2 as frontend and i just wondered why the HD channels which are h.264 encoded here don't get hardware decoded by the iOS VideoToolBox HW decoder we use on the ATV2. I figured that our HW Decoder implementation refuses the tvheadend stream because the hints delivered by ffmpeg for profile and level are both 0.
I then added a printout in tvheadends pares_h264.c and saw that profile and level are parsed fine from the dvb-c stream (profile is 77 level is 40 here). Its "just" that those 2 hints are not muxed back into the output stream (oh oh half knowledge here on my side haha). I hardcoded profile and level for a short test in XBMC and saw that the ATV2 hw decoder now was complaining about missing avcC data (which i wasn't able to hardcode because i don't know how).
Do you see any possibility to get those H.264 hints + avcC data into the tvheadend output stream? That would be major
Thx for your good work on tvheadend
Here are recordings from the 3 HD Channels i can receive:
Well tbh i saw that the atv2 hw decoder doesn't decode these. But it successfully opens the HW Decoder because the profile, level and extradata stuff is valid for the decoder (decoding fails after that but that might be an issue with our impl.).
#4 Updated by Eric Valette 9 months ago
John Törnblom wrote:
If you could provide me with a ts recording from the channel that is playable with your hardware, I could take a look.
Sounds like an old bug that was preventing to play mkv stream in hardware. Has been fixed long time ago. AVCC code generation has been enhanced also. What tvheadend version is he using? What about a TS dump and a remux with mkvmege to compare?
Here is a tsdump
I'm using 76e785578b4536f8bc586d1402c09fc05c1492bd from github.com/tvheadend (its a rev from aug 17th 2012)tsinfo arteHD.ts
Reading from arteHD.ts
Scanning 1000 TS packets
- TS packet starts 1a, not 47
- Error reading TS packet 1
- tsinfo: Error reporting on stream
There are no hints nor profile and level reported by ffmpeg in XBMC like the complete avcC+extradata is missing. If i record the channel i get an mkv (see the links above which seem to have proper avcC and extradata. Seems to be an issue with streaming only then.
#6 Updated by John Törnblom 9 months ago
I didn't really understand your last post. Are you saying its working now?
the arteHD.ts file you provided is not a ts file, it's a matroska file
$> file arteHD.ts
arteHD.ts: EBML file, creator matroska
both arteHD.ts and h264-1.mkv carry the extra data filed containing profile and level (logged with mplayer).
 [4d]   [ff] [e1]  [1b]  [4d]   [e4]  [a]  [b7]  [2d]    [7c]       [01        [fa]  [c8] 
 [4d]   [ff] [e1]    [4d]   [e4]  [a]  [b7]  [2d]    [7c]       [01  [a2]   [bd] [e0]  [1b]  [eb] [de] [e0]       [fa]  [c8] 
In order for me to verify that these extra data blobs are correct, I need a proper tsdump.
#10 Updated by Dave Chapman 9 months ago
I'm still learning about h264 streams, but found this link enlightening:
I've been working on my own HTSP client (for the Raspberry Pi) and have been having problems with h264 and the Pi's hardware decoder, so have been analysing the H264 HTSP streams.
From what I can tell, everything is fine. HTSP H264 streams are in Annex B format, meaning they use a 00 00 00 01 sync code to separate each NAL unit. The avcC metadata is only required where the stream uses a variable length "NAL length" field before each NAL unit instead of the sync code (it contains the size of that field).
By "extradata", I think you are referring to the SPS and PPS NAL units which in DVB streams (and HTSP streams) are repeated at the start of each I frame. The SPS contains the profile and level. In my tests, every "I" muxpkt data sent in the htsp stream (including the first) started with the SPS, and hence contains the profile and level.
So if I am understanding correctly, XBMC and your hardware decoder should know enough from what is contained in the HTSP stream to play the file - there is no need for tvh to send anything else.
#11 Updated by Scott Davilla 9 months ago
Some hw decoders want AVCC format so XBMC needs to convert Annex-B into AVCC. It needs a proper FFMpeg extradata in oder to do this. Yes the info is contain in the data stream but XBMC codecs do not have access to the data stream during the code open. So we need the same info that FFMpeg provides in extradata for Annex-B format h264.
The purpose of an XBMC decoder hints is to provide enough info to determine if the decoder can handle the stream (hw decoders can have restrictions) and to setup for decoding the stream. Proper Info in the profile, level, extradata fields is a requirement, not an option.
#12 Updated by Dave Chapman 9 months ago
How does XBMC deal with other streaming formats without explicit headers including the extradata, such as MPEG-TS? I would expect these to be the same as HTSP - i.e. Annex B format with the SPS/PPS included before every key frame. Can XBMC play such streams?
But in any case, this appears to be XBMC's problem - are you saying there is definitely no way XBMC can play these HTSP streams unless tvheadend extracts the SPS/PPS NAL units from the beginning of the first packet and sends them separately, presumably as part of the subscriptionStart packet? What should tvheadend do if those change in the stream (which isn't uncommon in DVB broadcasts - e.g. in the UK, some channels switch between 50i and 25p).
#13 Updated by Scott Davilla 9 months ago
MPEG-TS are demuxed by ffmpeg into audio/video packets and these are feed into our decoders. The demuxing via FFMpeg provides the Annex-B extradata. From what I understand about tvheadend is the data is pre-demuxed. I could be wrong and I'll know more in a few days when I setup a tvheadend backend to look closer at the issue.
#14 Updated by Dave Chapman 9 months ago
No, the tvheadend (HTSP) stream isn't "pre-demuxed", it's a single stream of interleaved audio and video packets, like most other container formats. Although the format is designed to be demuxed extremely easily, (it has one complete video frame or audio frame per HTSP packet, with the HTSP packet header including size, frame type (I/B/P), PTS and DTS).
The format is documented here (see "muxpkt"):
(the page isn't completely up to date, but will give you a rough idea).