Archive:MythicalLibrarian: Difference between revisions
Jump to navigation
Jump to search
>Outleradam (New page: THIS IS BETA --~~~~ #!/bin/bash #mythicalLibrarian by Adam Outler #email: [email protected] #Software the way it should be: Free and Open Source #Please contact me with any b...) |
>Outleradam No edit summary |
||
Line 695: | Line 695: | ||
echo CREATING SYMLINK IN MOVE MODE | echo CREATING SYMLINK IN MOVE MODE | ||
ln -s "$MoveDir/$ShowFileName.$originalext" "$3" | ln -s "$MoveDir/$ShowFileName.$originalext" "$3" | ||
fi | fi |
Revision as of 21:45, 26 December 2009
THIS IS BETA --Outleradam 07:02, 25 December 2009 (UTC)
#!/bin/bash #mythicalLibrarian by Adam Outler #email: [email protected] #Software the way it should be: Free and Open Source #Please contact me with any bug reports #Intention: # This program was designed to be a user job in MythTV. It can be called by creating a user job. # The user job can be called as follows: # /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh "%TITLE%" "%SUBTITLE%" "%DIR%/%FILE%" # #Usage: # mythicalLibrarian.sh -v "show name" "episode name" "Target Folder" # eg. mythicalLibrarian.sh "South Park" "Here Comes the Neighborhood" "/home/mythrecordings/2308320472023429837.mpg" # #Output-target # If an error occurs and the file cannot be moved, then no change will occur to the original file. If the Movedir # is full or not available, such as when running a NAS and the computer is disconnected from the network, the # AlternateMoveDir will be used. If both of these dirs fail, the show will be SymLinked in the FailSafeDir. # You may elect to run the user job at a later time when the issue has been resolved. Output dir and link type # will depend on user settings. The file name however, is preset to the most acceptable standard: # Show Title - SxxExx (Episode Title).ext # #Symlinking: # When Symlinking is enabled, mythicalLibrarian will follow its normal mode of operation. In MOVE mode, # mythicalLibrarian will Xreate a symlink from the new file in the same name and location of the old file. In # LINK mode, mythicalLibrarian will not move the file, LINK mode creates a new symlink to the original file. # #Output-Files # mythicalLibrarian will create several files in it's working folder. This is a list of the files and their functions. # -comskiplog.tracking-keeps track of created comskip.txt files so they can be deleted in the future if their video file # is deleted. # -doover.sh is designed to keep track of failed jobs. It is designed to be executable. #Commented commands are # those which are determined to be questionable. This file can be made executable and ran after a problem is corrected # which caused the problem. Questionable commands are those which will require you to add a episode title and set the # mythicalLibrarian.sh Database=Disabled setting. Questionable files do not have sufficient guide data. # -markupstart.txt and markupstop.txt are files which contain information from the last comskip generation. # Deletion will cause no adverse effects. # -output.log keeps track of operations and can be used to determine problems. # -shn.txt, sid.txt, and working.xml are used each time to determine the name and show id of the last show identified. # -The DailyReport folder is used to log the files which were moved that day. It can be used as a "program guide" of sorts # to keep track of what has been added to your library. # #Logging: # Log file will show information for troubleshooting. You can find the log file in the working folder # Log file default location: /home/mythtv/mythicalLibrarian/output.log # #Database-external: # This program will make 3 calls to TheTvDb for every episode. The first one is to obtain the series ID and verify the show # name is correct. The seccond is to check if the internally managed database is up-to-date. The third call will only be # made if the internal database is not up-to-date. The third call will download a larger file which contains all information # about the show which is known on TheTvDb. # #Database-internal: # While mythicalLibrarian maintains and requires it's own external file/folder database in the working directory, there is # also support for integration with MythTV's internal database. MythTV Database is required for movies to be recognized and # handled by mythicalLibrarian. Also, in the event that the integrated fuzzy logic cannot make a determination of the # correct show name, mythicalLibrarian will pull the original air date from the MythTV database and attempt to make an # exact match to theTvDb.com supplied data. In addition, the type of program is extracted from the mythtv database and a # determination is made weather or not there is sufficient information available to identify the show based upon guide data # In order to make mythicalLibrarian work to it's full potential, all settings must be filled out correctly under the # database section of the user settings. Currently, the only guide data supported is schedulesdirect through mythtv. # #Dependencies: depends on "Curl", "agrep", "libnotify-bin" and Mythtv Backend for database access. # install curl with "apt-get install curl" # install agrep with "apt-get install agrep" # optional: install libnotify-bin with "apt-get install libnotify-bin" # #Notifications: # In order for mythicalLibrarian to send notifications to the GNOME desktop, it must have no-password sudo access. It uses # this access strictly to send complete, moving and failure status notifications. Because this program is launched by the # user mythtv under normal circumstances, mythtv must temporarily become your user name in order to send a notification to # your desktop. This requires the use of a separate script, and for mythtv to have a sudoers group with no password option. # Notifications are an optional feature # #Show Name Translation # The user may elect to create a file in the mythicalLibrarian/ working folder which will then translate any recorded # show name into the desired show name. This is useful for adding a year to distinguish between a new series # and an older series and/or typos in your guide data. By default it should be called "showtranslations" and # it will be in your home/username/mythicalLibrarian folder. showtranslations is not needed by most users and the file # should only be created if it is needed. Under most circumstances, the integrated fuzzy logic will be # sufficient to translate the guide name to the TvDb name, however showtranslations is available to improve # accuracy to 100%. The format of showtranslations is as follows: #Filename: /$mythicalLibrarian/showtranslations ############################################################## #My Guide Show Title = www.TheTvDb.com Show Title # #Battlestar Gallactica = Battlestar Gallactica (2003) # #Millionaire = Who Wants To Be A Millionaire # #Aqua teen Hungerforce = Aqua Teen Hunger Force # ############################################################## #######################USER SETTINGS########################## #MoveDir is the folder which mythicalLibrarian will move the file. No trailing / is accepted eg. "~/videos" MoveDir=/home/mythtv/NAS/Video/shows #AlternateMoveDir will act as a seccondary MoveDir if the primary MoveDir fails. AlternateMoveDir=/home/mythtv/Shared #Disable moving of files. If UseOriginalDir is Enabled, original dir will override MoveDir. Useful for multiple recording dirs. Enabled|Disabled UseOriginalDir=Disabled #When Enabled, mythicalLibrarian will move the file to a folder of the same name as the show. This is not affected by UseOriginalDir. Enabled|Disabled UseShowNameAsDir=Enabled #SYMLINK has 3 modes. MOVE|LINK|Disabled #Create symlink in original dir from file after MOVE. Do not move just LINK original in MoveDir(overrides moving), Simlinking Disabled SYMLINK=MOVE #DailyReport provides a log of shows added to your library per day. DailyReport=Enabled #Internet access Timeout in seconds: Default Timeout=50 (seconds) Timeout=50 #Update database time in secconds, Longer duration means faster processing time and less strain on TheTvDb. Default='84000' (1 day) UpdateDatabase=84000 #mythicalLibrarian working file dir: Default is "~/mythicalLibrarian" mythicalLibrarian=~/mythicalLibrarian #FailSafe mode will enable symlinks to be formed in FailSafeDir if the move or symlink operation fails. FailSafeMode=Enabled #FailSafeDir is used when the file cannot be moved to the MoveDir. FailSafe will not create folders. FailSafeDir=/home/adam #The following line tells mythicalLibrarian to send a notification upon completion. Enabled|Disabled Notify=Enabled #If notifications are enabled, NotifyUserName should be the same as the user logged into the GNOME Session. NotifyUserName=adam #the following line contains the API key from www.TheTvDb.Com. Default: 6DF511BB2A64E0E9 APIkey=6DF511BB2A64E0E9 #Enables debug mode. This is a verbose mode of logging which should be used for troubleshooting. Enabled|Disabled DEBUGMODE=Enabled ###Database settings### #Database access Enabled|Disabled Database=Enabled #MySQL User name: Default="mythtv", MySQL access is required to obtain year for movies. MySQLuser=mythtv #MySQL Password: Default="mythtv" MySQLpass=mythtv #MySQL Myth Database: Default="mythconverg" MySQLMythDb=mythconverg #Primary Movie Dir. mythicalLibrarian will attempt to move to this dir first. No trailing / is accepted eg. "~/videos" PrimaryMovieDir=/home/mythtv/NAS/Video/Movies #AlternateMoveDir will act as a Seccondary move dir if the primary moive dir fails AlternateMovieDir=/home/mythtv/Videos #CommercialMarkup will generate comskip files for recordings when they are moved. #Comskips are deleted when mythtv deletes the recording Enabled|Disabled CommercialMarkup=Enabled #########################USER SETTINGS########################## ################################################################ ################Do not modify below this line################### ################################################################ echo "@@@@@@@@@@@NEW SEARCH INITIATED AT `date`@@@@@@@@@@@@@">>"$mythicalLibrarian"/output.log #####DEFINE ENVIRONMENT AND VARIABLES##### #make our working dir if it does not exist if [ ! -d "$mythicalLibrarian" ]; then mkdir $mythicalLibrarian echo "creating home/mythicalLibrarian and log file">>"$mythicalLibrarian"/output.log fi #Set episode name, dir, extension, and showname from the input parameters. ShowName=$1 epn=`echo $2|sed 's/;.*//'|tr -d [:punct:]` originalext=`echo "${3#*.}"` originaldirname=`dirname "$3"` FileBaseName=${3##*/} #Check for show translations relating to the show in question. if [ -f $mythicalLibrarian/showtranslations ]; then showtranslation=`grep "$ShowName = " "$mythicalLibrarian/showtranslations"|replace "$ShowName = " ""|replace "$mythicalLibrarian/showtranslations" ""` if [ "$showtranslation" != "$null" ];then ShowName=$showtranslation echo "USER TRANSLATION: $1 = $ShowName">>"$mythicalLibrarian"/output.log elif [ "$showtranslation" = "$null" ];then showtranslation="Inactive" fi fi #Check for Use Original Dir Parameter test "$UseOriginalDir" = "Enabled" && MoveDir="$originaldirname" #Check and make doover.sh if it does not exist if [ ! -f "$mythicalLibrarian/doover.sh" ]; then echo 'rm "'$mythicalLibrarian'"/doover.sh'>$mythicalLibrarian/doover.sh fi #Function daily report data dailyreport() { if [ $DailyReport = Enabled ]; then if [ ! -d "$mythicalLibrarian/DailyReport" ]; then mkdir "$mythicalLibrarian/DailyReport" fi reportfilename=`date +%Y-%m-%d` reporttime=`date +%T` echo "$reporttime "$1>>"$mythicalLibrarian/DailyReport/$reportfilename" fi } #CheckPermissions by writing a small file then deleting it, checking along the way. #CheckPermissions takes file size, free space on dir, and the dir, then it performs #tests. the result will be $TMoveDirWritable as a 1 or a 0 for writable or not. checkpermissions () { if [ "$1" -lt "$2" ]; then echo "Testing write permission on $3">$3/arbitraryfile.ext if [ -f "$3/arbitraryfile.ext" -a -s "$3/arbitraryfile.ext" ]; then rm "$3/arbitraryfile.ext" if [ ! -f "$3/arbitraryfile.ext" ]; then TMoveDirWritable=1 elif [ -f "$3/arbitraryfile.ext" ]; then TMoveDirWritable=0 fi else TMoveDirWritable=0 echo "CHECK PERMISSIONS ON $MoveDir" fi elif [ "$1" -ge "$2" ]; then TMoveDirWritable=0 echo "NO SPACE AVAILBLE ON $3" fi } #####MAINTENANCE##### #Loop through the list of created comskip files and remove orphans. if [ -f "$mythicalLibrarian/comskiplog.tracking" ]; then while read line do mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1` FileToCheck=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/comskiplog.tracking"` FileToCheckOriginalDirName=`dirname "$FileToCheck"` FileToCheckBaseName=$(basename $FileToCheck) FileToCheckBaseName=${FileToCheckBaseName%.*} if [ "$FileToCheck" != "" ]; then FileToCheckLS=`ls "$FileToCheckOriginalDirName"|grep "$FileToCheckFileBaseName\."|sed -n 2p` if [ "$FileToCheckLS" = "" ]; then echo "MAINTENANCE FOUND NO MOVIE: REMOVING COMSKIP $FileToCheck">>"$mythicalLibrarian"/output.log rm $FileToCheck elif [ "$FileToCheckLS" != ""]; then echo $FileToCheck>>"$mythicalLibrarian/comskiplog.tracking2" fi fi done <"$mythicalLibrarian/comskiplog.tracking" rm "$mythicalLibrarian/comskiplog.tracking" mv "$mythicalLibrarian/comskiplog.tracking2" "$mythicalLibrarian/comskiplog.tracking" fi #####GATHER INFORMATION FROM MYTHTV DATABASE##### #Get information if database is enabled if [ "$Database" = Enabled ]; then #get chanid for recordings to identify program table ChanID=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select chanid from recorded where basename like '$FileBaseName';"|sed -n 2p|replace "chanid" ""|replace " " ""` #get ProgramID from recorded to identify program ProgramID=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select programid from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "starttime" ""` mythicalLibrarianProgramIDCheck=${ProgramID:0:2} #Extrapolate data from Programid echo Check: $mythicalLibrarianProgramIDCheck test "$mythicalLibrarianProgramIDCheck" = "SH" && ProgramIDType="Generic Episode With No Data" test "$mythicalLibrarianProgramIDCheck" = "MV" && ProgramIDType="Movie" test "$mythicalLibrarianProgramIDCheck" = "EP" && ProgramIDType="Series With Episode Data" #if the ProgramID does not meet criteria, then end the program if [ "$ProgramIDType" = "Generic Episode With No Data" ]; then echo "PROGRAM GUIDE DATA IS NOT COMPLETE. TYPE: $ProgramIDType">>"$mythicalLibrarian"/output.log echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%">>"$mythicalLibrarian"/output.log echo "%%%%%%%%%%GENERIC GUIDE DATA WAS SUPPLIED $1, $2">>"$mythicalLibrarian"/output.log echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%">>"$mythicalLibrarian"/output.log if [ $Notify = "Enabled" ]; then sudo -u "$NotifyUserName" /usr/local/bin/alt-notify-send "mythicalLibrarian Guide error" "Could not obtain enough information for library: $ProgramIDType" error fi echo $mythicalLibrarian'/mythicalLibrarian.sh "'$1'" "'$2'" "'$3'"'>>$mythicalLibrarian/doover.sh exit 1 fi #get show start time to identify program ----future development ShowStartTime=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select starttime from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "starttime" ""` #get category from recorded to identify program table -----future development ShowCategory=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select category from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "category" ""` #get original air date for tv shows OriginalAirDate=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select originalairdate from recorded where basename like '$FileBaseName' ; "|sed -n "2p"|replace "originalairdate" ""` test "$OriginalAirDate" = "0000-00-00" && OriginalAirDate="$null" #get year for movies MovieAirDate=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select airdate from recordedprogram where programid like '$ProgramID' and chanid like '$ChanID' ; "|replace "airdate" ""|sed -n "2p"|replace " " ""` #Blank year if it is invalid if [ "$MovieAirDate" != "" ]; then if [ "$MovieAirDate" -lt "1900" ]; then MovieAirDate="" fi fi #Report found data echo RECSTART:$ShowStartTime MOVIEYEAR:$MovieAirDate SERIESDATE:$OriginalAirDate echo PROGRAMID:$ProgramID ShowCategory:$ShowCategory #####COMSKIP DATA##### #Remove old and generate a comskip Start list rm $mythicalLibrarian/markupstart.txt mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "4" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstart.txt #Remove old and generate comskip Stop list rm $mythicalLibrarian/markupstop.txt mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "5" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstop.txt #Set up counter, remove old markup data and generate new markup file from markupstart and stop if [ "$CommercialMarkup" = "Enabled" ]; then mythicalLibrarianCounter=1 echo "FILE PROCESSING COMPLETE">"$mythicalLibrarian"/markupframes.txt echo "------------------------">>"$mythicalLibrarian"/markupframes.txt while read line do mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1`; StartData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstart.txt"` StopData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstop.txt"` if [ "$StopData" != "" ]; then echo "$StartData $StopData">>"$mythicalLibrarian"/markupframes.txt CommercialMarkup=Created echo "COMMERCIAL DATA START:$StartData STOP:$StopData" fi done <"$mythicalLibrarian/markupstop.txt" fi fi #####SEARCH FOR SHOW NAME##### if [ "$MovieAirDate" = "$null" ]; then echo "SEARCHING: www.TheTvDb.com SHOW NAME: $ShowName EPISODE: $epn">>"$mythicalLibrarian"/output.log echo "FILE NAME: $3">>"$mythicalLibrarian"/output.log #Format Show name for Sending to www.TheTvDb.com tvdbshowname=`echo $ShowName|replace " " "%20"` #download series info for show, parse into temporary text db- sid.txt shn.txt curl -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>$mythicalLibrarian/working.xml cat $mythicalLibrarian/working.xml | grep "<seriesid>"|replace "<seriesid>" ""|replace "</seriesid>" "">$mythicalLibrarian/sid.txt cat $mythicalLibrarian/working.xml | grep "<SeriesName>"|replace "<SeriesName>" ""|replace "</SeriesName>" "">$mythicalLibrarian/shn.txt #Use fuzzy logic to make the best match of the show name serieslinenumber=`agrep -Byn "${showname:0:27}" $mythicalLibrarian/shn.txt|sed 's/:.*//'|grep -m1 ^` #Get the seriesid based on the showname seriesid=`sed -n $serieslinenumber'p' $mythicalLibrarian/sid.txt|grep -m1 ^` NewShowName=`sed -n $serieslinenumber'p' $mythicalLibrarian/shn.txt|grep -m1 ^` elif [ "$MovieAirDate" != "$null" ]; then NewShowName=$1 fi #Create folder for database if it does not exist if [ ! -d "$mythicalLibrarian/$NewShowName" ]; then mkdir $mythicalLibrarian/"$NewShowName" echo "creating home mythicalLibrarian and log file">>mythicalLibrarian/output.log fi echo "SEARCH FOUND:""$NewShowName" "ID#:" $seriesid >>"$mythicalLibrarian"/output.log #If series ID is obtained, then get show information. if [ "$seriesid" != "" ]; then #####GET SERIES INFORMATION##### #Get current server time curl -s -m"$Timeout" "http://www.thetvdb.com/api/Updates.php?type=none">"$mythicalLibrarian/$NewShowName/current.time" #Parse file into usable data only cat "$mythicalLibrarian/$NewShowName/current.time"|grep "