Archive:MythicalLibrarian

From Official Kodi Wiki
Revision as of 07:02, 25 December 2009 by >Outleradam (New page: THIS IS BETA --~~~~ #!/bin/bash #mythicalLibrarian by Adam Outler #email: [email protected] #Software the way it should be: Free and Open Source #Please contact me with any b...)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

THIS IS BETA --Outleradam 07:02, 25 December 2009 (UTC) 


#!/bin/bash

#mythicalLibrarian by Adam Outler
#email: [email protected]
#Software the way it should be: Free and Open Source
#Please contact me with any bug reports 

#Intention:
# This program was designed to be a user job in MythTV.  It can be called by creating a user job.
# The user job can be called as follows:
# /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh  "%TITLE%" "%SUBTITLE%" "%DIR%/%FILE%"
#
#Usage:
#  mythicalLibrarian.sh -v "show name" "episode name" "Target Folder"
#  eg. mythicalLibrarian.sh "South Park" "Here Comes the Neighborhood" "/home/mythrecordings/2308320472023429837.mpg"
#  
#Output-target
# If an error occurs and the file cannot be moved, then no change will occur to the original file. If the Movedir
# is full or not available, such as when running a NAS and the computer is disconnected from the network, the 
# AlternateMoveDir will be used. If both of these dirs fail, the show will be SymLinked in the FailSafeDir.
# You may elect to run the user job at a later time when the issue has been resolved.  Output dir and link type 
# will depend  on user settings. The file name however, is preset to the most acceptable standard:
#   Show Title - SxxExx (Episode Title).ext
#
#Symlinking:
# When Symlinking is enabled, mythicalLibrarian will follow its normal mode of operation.  In MOVE mode, 
# mythicalLibrarian will Xreate a symlink from the new file in the same name and location of the old file.  In 
# LINK mode, mythicalLibrarian will not move the file, LINK mode creates a new symlink to the original file. 
# 
#Output-Files
# mythicalLibrarian will create several files in it's working folder.  This is a list of the files and their functions.
# -comskiplog.tracking-keeps track of created comskip.txt files so they can be deleted in the future if their video file
# is deleted.
# -doover.sh is designed to keep track of failed jobs.  It is designed to be executable.  #Commented commands are 
# those which are determined to be questionable.  This file can be made executable and ran after a problem is corrected
# which caused the problem. Questionable commands are those which will require you to add a episode title and set the
# mythicalLibrarian.sh Database=Disabled setting.  Questionable files do not have sufficient guide data.
# -markupstart.txt and markupstop.txt are files which contain information from the last comskip generation.
# Deletion will cause no adverse effects.
# -output.log keeps track of operations and can be used to determine problems.
# -shn.txt, sid.txt, and working.xml are used each time to determine the name and show id of the last show identified.
# -The DailyReport folder is used to log the files which were moved that day.  It can be used as a "program guide" of sorts
# to keep track of what has been added to your library.
#
#Logging:
# Log file will show information for troubleshooting. You can find the log file in the working folder
# Log file default location: /home/mythtv/mythicalLibrarian/output.log
#
#Database-external:
# This program will make 3 calls to TheTvDb for every episode.  The first one is to obtain the series ID and verify the show
# name is correct.  The seccond is to check if the internally managed database is up-to-date.  The third call will only be
# made if the internal database is not up-to-date.  The third call will download a larger file which contains all information
# about the show which is known on TheTvDb.
#
#Database-internal:
# While mythicalLibrarian maintains and requires it's own external file/folder database in the working directory, there is 
# also support for integration with MythTV's internal database.  MythTV Database is required for movies to be recognized and 
# handled by mythicalLibrarian. Also, in the event that the integrated fuzzy logic cannot make a determination of the 
# correct show name, mythicalLibrarian will pull the original air date from the MythTV database and attempt to make an 
# exact match to theTvDb.com supplied data. In addition, the type of program is extracted from the mythtv database and a 
# determination is made weather or not there is sufficient information available to identify the show based upon guide data
# In order to make mythicalLibrarian work to it's full potential, all settings must be filled out correctly under the 
# database section of the user settings. Currently, the only guide data supported is schedulesdirect through mythtv.
#
#Dependencies: depends on "Curl", "agrep", "libnotify-bin" and Mythtv Backend for database access.
# install curl with "apt-get install curl"  
# install agrep with "apt-get install agrep"
# optional: install libnotify-bin with "apt-get install libnotify-bin" 
#
#Notifications:
# In order for mythicalLibrarian to send notifications to the GNOME desktop, it must have no-password sudo access.  It uses
# this access strictly to send complete, moving and failure status notifications.  Because this program is launched by the
# user mythtv under normal circumstances, mythtv must temporarily become your user name in order to send a notification to
# your desktop. This requires the use of a separate script, and for mythtv to have a sudoers group with no password option.
# Notifications are an optional feature
#
#Show Name Translation
# The user may elect to create a file in the mythicalLibrarian/ working folder which will then translate any recorded
# show name into the desired show name.  This is useful for adding a year to distinguish between a new series
# and an older series and/or typos in your guide data.  By default it should be called "showtranslations" and
# it will be in your home/username/mythicalLibrarian folder.  showtranslations is not needed by most users and the file 
# should only be created if it is needed. Under most circumstances, the integrated fuzzy logic will be 
# sufficient to translate the guide name to the TvDb name, however showtranslations is available to improve 
# accuracy to 100%. The format of showtranslations is as follows:
#Filename: /$mythicalLibrarian/showtranslations
##############################################################
#My Guide Show Title = www.TheTvDb.com Show Title            #
#Battlestar Gallactica = Battlestar Gallactica (2003)        # 
#Millionaire = Who Wants To Be A Millionaire                 #
#Aqua teen Hungerforce = Aqua Teen Hunger Force              #
##############################################################


#######################USER SETTINGS##########################
#MoveDir is the folder which mythicalLibrarian will move the file.  No trailing / is accepted eg. "~/videos"
MoveDir=/home/mythtv/NAS/Video/shows
#AlternateMoveDir will act as a seccondary MoveDir if the primary MoveDir fails.
AlternateMoveDir=/home/mythtv/Shared
#Disable moving of files.  If UseOriginalDir is Enabled, original dir will override MoveDir.  Useful for multiple recording dirs. Enabled|Disabled
UseOriginalDir=Disabled
#When Enabled, mythicalLibrarian will move the file to a folder of the same name as the show. This is not affected by UseOriginalDir. Enabled|Disabled
UseShowNameAsDir=Enabled
#SYMLINK has 3 modes.  MOVE|LINK|Disabled
#Create symlink in original dir from file after MOVE. Do not move just LINK original in MoveDir(overrides moving), Simlinking Disabled 
SYMLINK=MOVE
#DailyReport provides a log of shows added to your library per day.
DailyReport=Enabled
#Internet access Timeout in seconds: Default Timeout=50 (seconds)
Timeout=50 
#Update database time in secconds, Longer duration means faster processing time and less strain on TheTvDb. Default='84000' (1 day)
UpdateDatabase=84000
#mythicalLibrarian working file dir: Default is "~/mythicalLibrarian" 
mythicalLibrarian=~/mythicalLibrarian
#FailSafe mode will enable symlinks to be formed in FailSafeDir if the move or symlink operation fails.
FailSafeMode=Enabled
#FailSafeDir is used when the file cannot be moved to the MoveDir. FailSafe will not create folders.
FailSafeDir=/home/adam
#The following line tells mythicalLibrarian to send a notification upon completion. Enabled|Disabled
Notify=Enabled
#If notifications are enabled, NotifyUserName should be the same as the user logged into the GNOME Session.
NotifyUserName=adam
#the following line contains the API key from www.TheTvDb.Com. Default: 6DF511BB2A64E0E9
APIkey=6DF511BB2A64E0E9
#Enables debug mode.  This is a verbose mode of logging which should be used for troubleshooting.  Enabled|Disabled
DEBUGMODE=Enabled

###Database settings### 
#Database access Enabled|Disabled
Database=Enabled
#MySQL User name: Default="mythtv", MySQL access is required to obtain year for movies.
MySQLuser=mythtv
#MySQL Password: Default="mythtv"
MySQLpass=mythtv
#MySQL Myth Database: Default="mythconverg"
MySQLMythDb=mythconverg
#Primary Movie Dir. mythicalLibrarian will attempt to move to this dir first. No trailing / is accepted eg. "~/videos"
PrimaryMovieDir=/home/mythtv/NAS/Video/Movies
#AlternateMoveDir will act as a Seccondary move dir if the primary moive dir fails
AlternateMovieDir=/home/mythtv/Videos
#CommercialMarkup will generate comskip files for recordings when they are moved. 
#Comskips are deleted when mythtv deletes the recording Enabled|Disabled
CommercialMarkup=Enabled

#########################USER SETTINGS########################## 

################################################################
################Do not modify below this line###################
################################################################
echo "@@@@@@@@@@@NEW SEARCH INITIATED AT `date`@@@@@@@@@@@@@">>"$mythicalLibrarian"/output.log 

#####DEFINE ENVIRONMENT AND VARIABLES#####
#make our working dir if it does not exist
if [ ! -d "$mythicalLibrarian" ]; then 
	mkdir $mythicalLibrarian
	echo "creating home/mythicalLibrarian and log file">>"$mythicalLibrarian"/output.log
fi

#Set episode name, dir, extension, and showname from the input parameters.
ShowName=$1
epn=`echo $2|sed 's/;.*//'|tr -d [:punct:]`
originalext=`echo "${3#*.}"`
originaldirname=`dirname "$3"`
FileBaseName=${3##*/}  

#Check for show translations relating to the show in question.
if [ -f $mythicalLibrarian/showtranslations ]; then 
	showtranslation=`grep "$ShowName = " "$mythicalLibrarian/showtranslations"|replace "$ShowName = " ""|replace "$mythicalLibrarian/showtranslations" ""`		 
	if [ "$showtranslation" != "$null" ];then 
		ShowName=$showtranslation
		echo "USER TRANSLATION: $1 = $ShowName">>"$mythicalLibrarian"/output.log
	elif [ "$showtranslation" = "$null" ];then
		showtranslation="Inactive"
	fi
fi

#Check for Use Original Dir Parameter
test "$UseOriginalDir" = "Enabled" && MoveDir="$originaldirname"

#Check and make doover.sh if it does not exist
if [ ! -f "$mythicalLibrarian/doover.sh" ]; then
	echo 'rm "'$mythicalLibrarian'"/doover.sh'>$mythicalLibrarian/doover.sh
fi


#Function daily report data
dailyreport() {
	if [ $DailyReport = Enabled ]; then
		if [ ! -d "$mythicalLibrarian/DailyReport" ]; then
			mkdir "$mythicalLibrarian/DailyReport" 
		fi
		reportfilename=`date +%Y-%m-%d`
		reporttime=`date +%T`
    		echo "$reporttime "$1>>"$mythicalLibrarian/DailyReport/$reportfilename"
	fi
}

#CheckPermissions by writing a small file then deleting it, checking along the way.
#CheckPermissions takes file size, free space on dir, and the dir, then it performs
#tests.  the result will be $TMoveDirWritable as a 1 or a 0 for writable or not.
checkpermissions () { 
if [ "$1" -lt "$2" ]; then
	echo "Testing write permission on $3">$3/arbitraryfile.ext 
	if [ -f "$3/arbitraryfile.ext" -a -s "$3/arbitraryfile.ext" ]; then
		rm "$3/arbitraryfile.ext"
		if [ ! -f "$3/arbitraryfile.ext" ]; then
			TMoveDirWritable=1
		elif [ -f "$3/arbitraryfile.ext" ]; then
			TMoveDirWritable=0
		fi
	else
		TMoveDirWritable=0
		echo "CHECK PERMISSIONS ON $MoveDir"
	fi
elif [ "$1" -ge "$2" ]; then
	TMoveDirWritable=0
	echo "NO SPACE AVAILBLE ON $3"
fi
}
  
#####MAINTENANCE#####

#Loop through the list of created comskip files and remove orphans.
if [ -f "$mythicalLibrarian/comskiplog.tracking" ]; then
	while read line
	do
		mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1`
		FileToCheck=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/comskiplog.tracking"`
		FileToCheckOriginalDirName=`dirname "$FileToCheck"`
		FileToCheckBaseName=$(basename $FileToCheck)
		FileToCheckBaseName=${FileToCheckBaseName%.*}
		if [ "$FileToCheck" != "" ]; then
			FileToCheckLS=`ls "$FileToCheckOriginalDirName"|grep "$FileToCheckFileBaseName\."|sed -n 2p`
	 		if [ "$FileToCheckLS" = "" ]; then
				echo "MAINTENANCE FOUND NO MOVIE: REMOVING COMSKIP $FileToCheck">>"$mythicalLibrarian"/output.log
				rm $FileToCheck
			elif [ "$FileToCheckLS" != ""]; then
				echo $FileToCheck>>"$mythicalLibrarian/comskiplog.tracking2"
			fi
		fi
	done <"$mythicalLibrarian/comskiplog.tracking"
 	rm "$mythicalLibrarian/comskiplog.tracking"
 	mv "$mythicalLibrarian/comskiplog.tracking2" "$mythicalLibrarian/comskiplog.tracking"
 fi




#####GATHER INFORMATION FROM MYTHTV DATABASE#####

#Get information if database is enabled
if [ "$Database" = Enabled ]; then 

#get chanid for recordings to identify program table
	ChanID=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select chanid from recorded where basename like '$FileBaseName';"|sed -n 2p|replace "chanid" ""|replace " " ""`
 
#get ProgramID from recorded to identify program
	ProgramID=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select programid from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "starttime" ""`
	mythicalLibrarianProgramIDCheck=${ProgramID:0:2}

#Extrapolate data from Programid
	echo Check: $mythicalLibrarianProgramIDCheck
	test "$mythicalLibrarianProgramIDCheck" = "SH" && ProgramIDType="Generic Episode With No Data"
	test "$mythicalLibrarianProgramIDCheck" = "MV" && ProgramIDType="Movie"
	test "$mythicalLibrarianProgramIDCheck" = "EP" && ProgramIDType="Series With Episode Data"

#if the ProgramID does not meet criteria, then end the program
	if [ "$ProgramIDType" = "Generic Episode With No Data" ]; then
		echo "PROGRAM GUIDE DATA IS NOT COMPLETE.  TYPE: $ProgramIDType">>"$mythicalLibrarian"/output.log
		echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%">>"$mythicalLibrarian"/output.log
		echo "%%%%%%%%%%GENERIC GUIDE DATA WAS SUPPLIED $1, $2">>"$mythicalLibrarian"/output.log
		echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%">>"$mythicalLibrarian"/output.log
 		if [ $Notify = "Enabled" ]; then
			sudo -u "$NotifyUserName" /usr/local/bin/alt-notify-send "mythicalLibrarian Guide error" "Could not obtain enough information for library: $ProgramIDType" error
		fi
 		echo $mythicalLibrarian'/mythicalLibrarian.sh "'$1'" "'$2'" "'$3'"'>>$mythicalLibrarian/doover.sh
		exit 1
	fi



#get show start time to identify program ----future development
	ShowStartTime=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select starttime from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "starttime" ""`

#get category from recorded to identify program table -----future development
	ShowCategory=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select category from recorded where basename like '$FileBaseName' ; " |sed -n "2p"|replace "category" ""`

#get original air date for tv shows
	OriginalAirDate=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select originalairdate from recorded where basename like '$FileBaseName' ; "|sed -n "2p"|replace "originalairdate" ""`
	test "$OriginalAirDate" = "0000-00-00" && OriginalAirDate="$null"

#get year for movies 
	MovieAirDate=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select airdate from recordedprogram where programid like '$ProgramID' and chanid like '$ChanID' ; "|replace "airdate" ""|sed -n "2p"|replace " " ""`

#Blank year if it is invalid
	if [ "$MovieAirDate" != "" ]; then
		if [ "$MovieAirDate" -lt "1900" ]; then
			MovieAirDate=""
		fi
	fi

#Report found data
	echo RECSTART:$ShowStartTime MOVIEYEAR:$MovieAirDate SERIESDATE:$OriginalAirDate 
	echo PROGRAMID:$ProgramID ShowCategory:$ShowCategory
	


#####COMSKIP DATA#####
#Remove old and generate a comskip Start list
	rm $mythicalLibrarian/markupstart.txt
	mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "4" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstart.txt

#Remove old and generate comskip Stop list
	rm $mythicalLibrarian/markupstop.txt
	mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "5" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstop.txt

#Set up counter, remove old markup data and generate new markup file from markupstart and stop
	if [ "$CommercialMarkup" = "Enabled" ]; then	
		mythicalLibrarianCounter=1
		echo "FILE PROCESSING COMPLETE">"$mythicalLibrarian"/markupframes.txt
		echo "------------------------">>"$mythicalLibrarian"/markupframes.txt	
		while read line
			do
			mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1`;
			StartData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstart.txt"`
			StopData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstop.txt"`
			if [ "$StopData" != "" ]; then
				echo "$StartData $StopData">>"$mythicalLibrarian"/markupframes.txt
				CommercialMarkup=Created
				echo "COMMERCIAL DATA START:$StartData STOP:$StopData"
			fi
		done <"$mythicalLibrarian/markupstop.txt"
	fi


fi 





#####SEARCH FOR SHOW NAME#####
if [ "$MovieAirDate" = "$null" ]; then
	 echo "SEARCHING: www.TheTvDb.com SHOW NAME: $ShowName EPISODE: $epn">>"$mythicalLibrarian"/output.log
	 echo "FILE NAME: $3">>"$mythicalLibrarian"/output.log

#Format Show name for Sending to www.TheTvDb.com
	 tvdbshowname=`echo $ShowName|replace " " "%20"`

#download series info for show, parse into temporary text db- sid.txt shn.txt
	 curl -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>$mythicalLibrarian/working.xml
	 cat $mythicalLibrarian/working.xml | grep "<seriesid>"|replace "<seriesid>" ""|replace "</seriesid>" "">$mythicalLibrarian/sid.txt
	 cat $mythicalLibrarian/working.xml | grep "<SeriesName>"|replace "<SeriesName>" ""|replace "</SeriesName>" "">$mythicalLibrarian/shn.txt

#Use fuzzy logic to make the best match of the show name
	 serieslinenumber=`agrep -Byn "${showname:0:27}" $mythicalLibrarian/shn.txt|sed 's/:.*//'|grep -m1 ^`

#Get the seriesid based on the showname
	 seriesid=`sed -n $serieslinenumber'p' $mythicalLibrarian/sid.txt|grep -m1 ^`
	 NewShowName=`sed -n $serieslinenumber'p' $mythicalLibrarian/shn.txt|grep -m1 ^`
elif [ "$MovieAirDate" != "$null" ]; then
	NewShowName=$1
fi

#Create folder for database if it does not exist
if [ ! -d "$mythicalLibrarian/$NewShowName" ]; then
	mkdir $mythicalLibrarian/"$NewShowName"
	echo "creating home mythicalLibrarian and log file">>mythicalLibrarian/output.log
fi
	echo "SEARCH FOUND:""$NewShowName" "ID#:" $seriesid >>"$mythicalLibrarian"/output.log

#If series ID is obtained, then get show information.
if [ "$seriesid" != "" ]; then

#####GET SERIES INFORMATION#####
 
#Get current server time
	curl -s -m"$Timeout" "http://www.thetvdb.com/api/Updates.php?type=none">"$mythicalLibrarian/$NewShowName/current.time"

#Parse file into usable data only
	cat "$mythicalLibrarian/$NewShowName/current.time"|grep "
Retrieved from "https://kodi.wiki/index.php?title=Archive:MythicalLibrarian&oldid=21259"