Automatic cleaning of recordings
Added by Sébastien Aubry about 1 year ago
Hello,
I am using XBMCBuntu Eden + tvheadend 2.12.99~pulse-6.
Quite often, the old recordings are not deleted from my hard disk in order to make some space for the new ones.
This results in a full hard disk, also causing problem with the EPG grabbing and potentially making tvheadend crash.
Is there any way to specify a minimum space amount tvheadend should regularly check and free up? I have seen in the FAQ that this should be automatic but it looks like not enough old recordings are deleted. How to fix this?
From my old Windows setups, I would say that I need about 20 GB of free space before starting to record (sometimes recordings 3 or 4 movies at the same time).
Thanks
/usr/bin$ tvheadend & [1] 9042 [INFO]:dvb: Found adapter /dev/dvb/adapter0 (DiBcom 3000MC/P) via USB (480 Mbit/s) [INFO]:dvb: Found adapter /dev/dvb/adapter1 (DiBcom 3000MC/P) via USB (480 Mbit/s) [INFO]:xmltv: Grabbing "France" using command "/usr/bin/tv_grab_fr" [INFO]:dvr: Creating new configuration '' [INFO]:CSA: Using SSE2 128bit parallel descrambling [NOTICE]:START: HTS Tvheadend version 2.12.99~pulse-6 started, running as PID:9042 UID:1000 GID:1000, settings located in '/home/spontex/.hts/tvheadend' [INFO]:AVAHI: Service 'Tvheadend' successfully established. [INFO]:dvr: "Irène" on "W9" recorder starting [INFO]:subscription: "DVR: Irène" subscribing on "W9", weight: 300, adapter: "DiBcom 3000MC/P", network: "F", mux: "F: 522,000 kHz", provider: "MULTI4", service: "W9", quality: 100 [INFO]:dvr: "Pirates" on "France 4" recorder starting [INFO]:subscription: "DVR: Pirates" subscribing on "France 4", weight: 300, adapter: "DiBcom 3000MC/P", network: "F", mux: "F: 610,000 kHz", provider: "NTN", service: "France 4", quality: 100 [ERROR]:MKV: /pchc_f/Recorded TV/Pirates/France-4-Pirates.2012-05-13.22-30-1.mkv: Unable to write -- No space left on device [INFO]:dvr: France 4-Pirates.2012-05-13.22-30 from adapter: "DiBcom 3000MC/P", network: "F", mux: "F: 610,000 kHz", provider: "NTN", service: "France 4" [INFO]:dvr: # type lang resolution samplerate channels [INFO]:dvr: 1 MPEG2VIDEO 720 x 576 [INFO]:dvr: 2 MPEG2AUDIO fra 48000 2 [INFO]:dvr: 3 MPEG2AUDIO eng 48000 2 [INFO]:dvr: 4 DVBSUB fra [INFO]:dvr: 5 DVBSUB fra [ERROR]:MKV: /pchc_f/Recorded TV/Irène/W9-Irène.2012-05-13.22-20-1.mkv: Unable to write -- No space left on device [INFO]:dvr: W9-Irène.2012-05-13.22-20 from adapter: "DiBcom 3000MC/P", network: "F", mux: "F: 522,000 kHz", provider: "MULTI4", service: "W9" [INFO]:dvr: # type lang resolution samplerate channels [INFO]:dvr: 3 MPEG2VIDEO 720 x 576 [INFO]:dvr: 1 MPEG2AUDIO eng 48000 2 [INFO]:dvr: 2 MPEG2AUDIO qad 48000 2 [INFO]:dvr: 4 MPEG2AUDIO fra 48000 2 [INFO]:dvr: 5 DVBSUB fra [INFO]:dvr: 6 DVBSUB fra
Replies (9)
RE: Automatic cleaning of recordings
-
Added by Hein Rigolo about 1 year ago
As far as I can tell there is no entry in the FAQ that states that old recordings are automatically deleted.
There is a project proposal to implement such a feature, but that is not yet implemented.
So for now you manually need to monitor the available disc space and manually delete recordings that are no longer needed.
Hein
RE: Automatic cleaning of recordings
-
Added by Sébastien Aubry about 1 year ago
Hi,
You are right! I didn't see that in the FAQ but in the basic guidelines: Tvheadend will clear enough room to store the expected recording size at the start of the recording. It will take into account already running recordings.
Do you guys always delete your recordigs manually? I happen to record more than 10 programs per day so I would like an automated behaviour. If it does not exist at this time, I will now write a script which can be put inside the crontab in order to automatically delete the oldest .mkv files recursively under a given directory, provided that the free space is below a given threshold.
Regards
RE: Automatic cleaning of recordings
-
Added by Hein Rigolo about 1 year ago
like you said in an other thread: With 1TByte of storage you have enough room for 1 month of recordings. Why keep a programs after you have seen it? How often do you watch it again? And if you record more then you can watch, then why record it at all.
So yes ... I delete my recordings manually. And actually, I move them away from the server to a central location so that my recording directory always has enough room.
Hein
RE: Automatic cleaning of recordings
-
Added by Sébastien Aubry about 1 year ago
Hello,
The fact is that I don't see all of them! I only want to be able to choose between 20 movies when I want to see one. Thee oldest programs should be deleted when space is needed.
I record all these programs to have the choice (in order to never have to go through Live TV!), not to see all of them. This kind of feature is available in the other PVRs.
I will now write an autodelete script and share it with the community.
RE: Automatic cleaning of recordings
-
Added by Sébastien Aubry about 1 year ago
The attached script will do the job.
Usage:
- Save the script inside any folder (e.g. $HOME/scripts/autodelete)
- Give it the executable access rights:
chmod a+x <path to autodelete>
- By default, the script runs in simulation mode and will not delete any file. It allows you to check its behaviour before you can use it in production.
Try a test run using:/path/to/autodelete --space 20 <recordings_path>
where 20 is the amount (in GB) of space you want to free and recordings_path is the path where you keep your recordings. - Sample output:
2012-05-18 22:51:00,275 INFO Cleaning 20.0 GB of .mkv files in folder /pchc_f/Recorded TV/ 2012-05-18 22:51:00,348 INFO Current free space in /pchc_f/Recorded TV/ is 12.45 GB 2012-05-18 22:51:00,348 INFO At least 7.55 GB need to be cleaned 2012-05-18 22:51:00,348 INFO The following files need to be deleted: 2012-05-18 22:51:00,348 INFO * 01.33 GB - Sun Apr 1 10:05:00 2012 - /pchc_f/Recorded TV/Karambolage/ARTE-HD-Karambolage.2012-04-01.09-45.mkv 2012-05-18 22:51:00,349 INFO * 02.91 GB - Sun Apr 1 13:05:00 2012 - /pchc_f/Recorded TV/Philosophie/ARTE-HD-Philosophie.2012-04-01.12-30.mkv 2012-05-18 22:51:00,349 INFO * 02.37 GB - Sun Apr 1 15:10:00 2012 - /pchc_f/Recorded TV/Sherlock/France-4-Sherlock.2012-04-01.13-30.mkv 2012-05-18 22:51:00,350 INFO * 00.91 GB - Sun Apr 1 20:15:00 2012 - /pchc_f/Recorded TV/Karambolage/ARTE-HD-KARAMBOLAGE.2012-04-01.20-01.mkv 2012-05-18 22:51:00,350 INFO * 01.01 GB - Mon Apr 2 00:35:00 2012 - /pchc_f/Recorded TV/L'Øcho-des-lois/LCP-L'écho-des-lois.2012-04-02.00-00.mkv
- When you are sure that you have found the right settings, you can use the script in production by adding the --production switch. The files which were found in simulation mode will REALLY be deleted!
/path/to/autodelete --space 20 <recordings_path> --production
- You can then automate the execution of this script after each recording. Since I could not configure Tvheadend to run a post-processing script with arguments, you will have to create a script (e.g.
my_autodelete) by copying and pasting the line you have come up with:#!/usr/bin/env sh /path/to/autodelete --space 20 <recordings_path> --production
- Give it the executable rights:
chmod a+x /path/to/my_autodelete
- Ask Tvheadend to run it after each recording by using the web interface, going to Configuration / DVR and set "Post-processor command" as /path/to/my_autodelete
You can then regularly check the work performed by the script by looking at the /tmp/autodelete.log file.
autodelete (3.95 KB)
Useful script, but watch out for empty dir deletion
-
Added by Richard Lloyd about 1 year ago
A handy script, but beware about tvheadend directory permissions if you decide to delete an empty directory. If someone is recording in, say, /data/recordings, and records all programmes "flat" (no date or channel sub-directories), then if all the recordings are deleted in that dir (e.g. manually or by the autodelete script), then the next time autodelete runs, it will actually delete the /data/recordings empty dir itself.
Now, it may be that /data is writable by the tvheadhead user and it can just create /data/recordings again and carry on, but there's a decent chance it's a root-owned mount point and therefore future recordings will fail to start because /data/recordings isn't creatable.
What the script probably needs is to test that the parent directory of the empty dir to be deleted is writable by the tvheadend user (if it isn't, don't delete the empty dir).
RE: Automatic cleaning of recordings
-
Added by Sébastien Aubry about 1 year ago
Hi,
This new version (1.1) checks, before deleting an empty directory:
- that it has the write access rights on this directory
- that it is not the root directory
Moreover, the logs (including the exceptions) are now written to ~/.config/autodelete, using automatic rolling logs (like syslog).
I have been using this script in production for 3 weeks now, and I am extremely happy: I now record about 20 programs per day using Tvheadend, which are automatically deleted after about one month. I never run out of free space, which was a major cause of bugs when I did not have this script.
autodelete (5.34 KB)
RE: Automatic cleaning of recordings
-
Added by Richard Lloyd about 1 year ago
I use XBMC as a front-end with tvheadend as the back-end and XBMC does offer the option to manually delete recordings (one at a time, ho hum), though there is an option in XBMC's System -> Settings -> Appearance -> File Lists to toggle this on or off. However, if I write click on a finished recording (Live TV -> Recordings), I'm presented with a "Delete" option even when I've turned off the rename/delete option in the XBMC Settings (duh! Bug?!).
What's a double whammy, though, is that if I attempt to delete a TV recording via XBMC as detailed above (and with the XBMC rename/delete option enabled), it will indeed remove it from the recordings list on XBMC, but does not delete it from the tvheadend backend, leaving potentially large files hanging around on the tvheadend server that - as people here seem to do - have to be deleted "again" from outside of XBMC.
As I've said in another thread, tvheadend leaves a lot to be desired when it comes to space management - there needs to be some status page with filestore stats and a warning appearing if it's getting now. As my above experience shows, deletion requests from the front-end should be passed back to the backend and acted upon (assuming permissions are OK), rather than giving the impression a programme has been deleted on the tvheadend filestore when it clearly hasn't.
RE: Automatic cleaning of recordings
-
Added by Sébastien Aubry 4 months ago
Hi,
I just improved the "autodelete" script in order to allow to only keep the last 3 episodes of some shows (e.g. TV series, daily or frequent shows).
In combination with #1083 we will now get a much better space management, until #1398 is implemented.
Here is how I use it: I added a post-processor command to a script containing:
autodelete /pchc_f/Recorded\ TV --space 30 --production --limit_folders "Philosophie;Le-dessous-des-cartes;Yourope;Metropolis;Ludo;Le-petit-journal;Entrée-libre;Envoyé-spécial;Suspect-n°1;Silence,-ça-pousse;Personne-ne-bouge;Mystères-d'archives;Des-mots-de-minuit;Un-livre,-un-jour;Karambolage;La-grande-librairie;Ce-soir-(ou-jamais-\!);Le-blogueur;Les-aventuriers-d'Explô;Archipels;28-minutes"
the --limit_folders argument will only keep the last 3 recordings of the shows passed as an argument.
autodelete (6.35 KB)
(1-9/9)