Add multiple language in EIT
|Assignee:||Adam Sutton||% Done:|
Hello, I would like to see multiple language support for EIT in tvheadend. I attached muxdump from Total TV provider which have at least 4 different languages in EIT.
Now tvheadend mixes all language into EPG and it looks really strange for me. One Program has slovenian language, other serbian and some of them english.
I would also like to mention that in slovenian language some of the character are partly supported (čšž and probablj ćđ also). I wrote down partly because some Programs have them in EPG and some don't.
This should be also included in attached muxdump.
#1 Updated by Alan - almost 3 years ago
Yes i also have the same problem with UPC Direct broadcast with Hungarian, Slovakian, English, Romanian language EPG. Character Encoding is OK though. Maybe a setting for preferred EPG language and a setting with character conversion options would be great. The preference could be for not only one but for two (three) languages. So that One can tell that in case the first preferred language is not there the second preferred language should step in place ( just like in VDR i think)
Thanks for the great work Andoma.
#8 Updated by Riku H over 2 years ago
preferred epg language should be configurable
DVB-DescriptorTag: 77 (0x4d) [= short_event_descriptor]
descriptor_length: 139 (0x8b)
event_name_length: 33 (0x21)
event_name: "House 7. kausi, 2. jakso: Selfish" -- Charset: Latin alphabet
text_length: 101 (0x65)
text_char: "Maailman suosituimman sarjan 7. tuotantokaudella House ja Cuddy yrittävät saada suhteensa toimimaan." -- Charset: Latin alphabet no. 5
DVB-DescriptorTag: 77 (0x4d) [= short_event_descriptor]
descriptor_length: 83 (0x53)
event_name_length: 20 (0x14)
event_name: "House Del 2, Selfish" -- Charset: Latin alphabet
text_length: 58 (0x3a)
text_char: "Amerikansk dramaserie med Hugh Laurie och Lisa Edelstein." -- Charset: Latin alphabet no. 5
#9 Updated by Jarmo Keränen about 2 years ago
This really is annoying feature and very important feature to be corrected. At least for me this is the question to use or not to use TVheadend at the moment, and it seems that this is not suitable for me at the moment... Is it really so that this cannot be patched somehow to work when we're waiting for ver. 3.0? Other option could be (at least for me) to have patch to eliminate EPG data receiving from DVB and use only XMLTV (there is another issue for that, but I mean some temporary solution with patch before we have ver 3.0).
If someone has made patch for this or disbling EPG from DVB signal I would like to have it also...
Anyway, if we do not look at these two problems I mentioned, great work and easy to configure to use. applause ;o)
#10 Updated by Riku H about 2 years ago
#13 Updated by Adam Sutton 9 months ago
Been giving this some thought, it would be quite straightforward to hack this into the EIT handler. However I'm not sure that's the right way to go. I'm beginning to think that it might require a more fundamental update to the EPG code so that this is properly handled throughout the EPG code.
This doesn't mean I'm thinking about storing multiple languages in the EPG database (though that is one way to go, I think its a bit much at this stage). More that I'm thinking it needs to consider the fact we also have inputs from other sources, XMLTV etc.. which might conflict and/or have multi language support.
I need to think this through properly.
#15 Updated by Adam Sutton 9 months ago
Well, someone persuaded me it might be a good idea to actually do things properly and provide full multi-lingual support to the EPG. I.e. I track all available languages in the DB and then filter on request.
Seems a great idea and much of the implementation is no more complicated than basic filtering on input. However since I use title/desc hashing for episode IDs, you get a horrible chicken/egg situation plus a general processing nightmare.
However I'm going to persevere and see what comes of it. But might not be ready today.
#16 Updated by Adam Sutton 9 months ago
- % Done changed from 0 to 80
I have a "working" solution. I'm using the word "working" very loosely here basically I've done all the basic work to include full multi-lingual support to the EPG code. But its VERY experimental, as in I've done almost no testing as I don't have access to a multi-lingual source (at the moment).
I've done basic testing with my own setup and XMLTV input (I'll try and do a bit more tonight). But if anyone wants to give it a got I'd really appreciate that. But do expect some failures/crashes etc...
The EPG DB (as well as related code) is now fully multi-lingual (or at least that's the theory) so info for ALL available languages will be stored in the database. And its then filtered on request.
To make use of it, goto WebUI->Configuration->General and set default languages. This is a comma separated list of ISO-639 (1 and 2T and 2B) codes (infact it'll parse a string as expected by HTTP Accept-Language). This will define the default options, useful for HTSP (to showtime/XBMC). Note: if you don't do this the values you get back could be a random collection of langs (though generally it'll be the first, alphabetically, language available).
From the WebUI itself it will try and pick up the Accept-Language header (as sent by browser) so beware if your browser is configured differently to what you expect.
Anyway let me know how you get on.
Code is on github @ https://github.com/adamsutton/tvheadend and make sure you use the feature/227-multilang-eit branch.
#18 Updated by Adam Sutton 9 months ago
All OTA grabbers are meant to be enabled by default for new installations and on migration from old configuration. However there was a bug in the migration that meant it did not do this, so if you have used several git master builds recently you might have fallen into that trap (i.e. it migrated the config incorrectly, it won't do it a second time). If this was your use of latest code then let me know and I'll check there isn't another mistake in the migration.
I'm glad the code appears to be working, especially since it was a bit of a blind shot as I'd done very little testing.
I'd be quite interested to know how the language selection works, if you're able to change the language your browser requests does the EPG come up in a different language?
Alternatively if you could mail me your EPG database and possibly the contents of the .hts/tvheadend/channels, I might be able to check it myself on my machine.
#21 Updated by doron rabia 9 months ago
For some reason I can't get it to work - I have waited for it for so long (thanks a lot for the development work !!)
I am trying to filter out hebrew out of 4 languages in EIT EPG , but still get all 4 of them randomly.
I am using version 3.1.608.g08434 and have tried to set he language as "heb" "he" and even "hebrew" as per Iso639, but nothing worked.
Any idea ?
Thanks in Advance,
#27 Updated by Adam Sutton 9 months ago
Are you sure you're actually running the right version/binary? That database does not correspond to the format of the latest code.
Can you check that you're definitely running the right version, check what version it says in the UI (About tab). And make sure you refresh incase it's cached something from a previous instance you had running.
#28 Updated by doron rabia 9 months ago
I am pretty sure, as I previous versions were 2.99* and only tonight I have changed to 3.1 tree.
I must say that my config is not fresh ,and I am using channel configs from previous versions.
I did notice that when started with new version (3.1 based) it did create epgdb.v2 which was a new file.
I will refresh my local git and rebuild and see if this helps.
#29 Updated by Adam Sutton 9 months ago
Hold on, my bad. I was looking at someone else's database. I've got WAY too many users config's on this machine I need to put a rule about how to provide stuff in nicely named directories
OK, so having got that out of the way, what exactly is the problem? I can see that it definitely has the hebrew data in (lang code should be "heb") as well as English (eng), Russian (rus) and Arabic (ara).
Although not every entry appears to have every language, probably a limitation of the EIT data. And in many cases although its listed as one of the non-English entries its in English. Again most likely a problem with the upstream data.
#31 Updated by Adam Sutton 9 months ago
Right. It's not actually a filter, its a prioritisation.
If there is no Hebrew entry it will return the next entry in your list. If none of your selections match it will pick whatever it can from what is in the database. This will typically be the first available lang code (alphabetically).
And as mentioned several of the Hebrew entries appear to actually be in English (though they report Hebrew).
The alternative would be to simply return blank fields, where the explicit selections are not available. And I'm not sure that would be right. Certainly most people would not want that.