SFS-TCZ_Linker-2.2.pet

Miscellaneous tools
Message
Author
seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#41 Post by seaside »

jrb wrote: It creates a list of the files in /root/.packages/XXX.files, sadly it does not distinguish between symlinks and files. When you uninstall the SFS it deletes these files and symlinks regardless of whether it created them or not.
It could - by adding code to the unlinking script that would test if the file were a symlink (if -L file). If it wasn't then don't remove. This could save the removal of a critical file.

Copied files would remain - but there are usually only a couple of these and they wouldn't usually cause anything to fail.

Regards,
s

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#42 Post by WarMocK »

Okay, bumping this thread up a bit because I'm trying to find a solution for the uninstall problem.
I wrote a tiny script that compares the index file of an installed SFS with all the other package index files, sorting out what is overwritten by an sfs or dotpet installed after the sfs you want to remove.
Technically, it moves the *.files file of the sfs from .packages to $HOME, dumps the content of each *.files file into a temporary file under /tmp, and then compares that file with the index file of the sfs, leaving only what would be added from the sfs index file into the temporary dump list.
I think that pretty much sorts out any files which might have been overwritten by either dotpets or other sfs files, so your unlinker would only have to remove the symlinks listed in the difference file generated by my script.
The script works damn fast, so it wouldn't add too much to the uninstallation time.
So, if you might want to give it a try:

Code: Select all

#!/bin/sh

mv $HOME/.packages/$1 $HOME
cat $HOME/.packages/*.files > /tmp/installedfiles
diff $HOME/$1 /tmp/installedfiles | grep / | grep -v ">" | cut -b 3- > /tmp/difference

I still need to figure out how to implement it into your unlinker script, among with the test for if the file to be removed is actually a symlink or not (as a safety measure). Do you have any documentation about how your script works in detail? That would help me a lot. ;-)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#43 Post by technosaurus »

if you use ln -s instead of ln -s -f it will not overwrite existing files - if you set up a variable called
OVERWRITE="-f" #set to "" to avoid overwriting
It could even use an Xdialog to do on original linking
.... then something like
ln -s $OVERWRITE $SFSMNTPATH$ONEFILE $ONEFILE
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#44 Post by jrb »

WarMock, I believe you've done it! :D I have incorporated your technique into /usr/local/bin/sfs_linker and done a few preliminary tests and it works!

I placed:

Code: Select all

cat /root/.packages/*.files > /tmp/installedpackagefiles #suggested by WarMock 22mar10
at the beginning (line 6) to generate a list of previously installed files.

I modified the .file generating code (line 96) to read:

Code: Select all

find ./$SFSNAME -mount -mindepth 1 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' > /tmp/$SFSNAME.files
and (line 98]

Code: Select all

find ./$SFSNAME -type l -mount -mindepth 1 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' >> /tmp/$SFSNAME.files
then used (line 100):

Code: Select all

diff /tmp/$SFSNAME.files /tmp/installedpackagefiles | grep / | grep -v ">" | cut -b 3- > /tmp/$SFSNAME.difference
to list only those new files installed by this package, and finally(line 101)

Code: Select all

cp /tmp/$SFSNAME.difference /root/.packages/$SFSNAME.files
to create the SFSNAME.files file in /root/.packages.

I created a .pet of the icons from one of my SFS's and installed it. Then used SFS_Linker to install the SFS. The icons were not listed in /root/.packages/SFS.files, although all the other files were, and were not uninstalled along with the SFS, perfect!

Using this technique /usr/local/bin/sfs_unlinker does not need to be changed and files copied from the SFS package can be removed safely at uninstall as opposed to just removing links.

I am attaching the modified /usr/local/bin/sfs_linker for you to test. Will repackage the .pet and post after further testing.

Thanks, J
Attachments
sfs_linker.gz
test file. Extract, set executable and place in /usr/local/bin
(1.77 KiB) Downloaded 690 times

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#45 Post by jrb »

technosaurus,
This is the code that I used to create the symlinks from /mnt/SFSpackage to /

Code: Select all

cp -rs /mnt/"$SFSNAME"/* /
It doesn't overwrite other existing files or symlinks.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#46 Post by WarMocK »

You're welcome, man. ;-)
This means that now I can fully implement your script into K-9 Linux and embed my SFS management framework into the ISO.
Oh, and I just realized there's still one little issue with the unlinking process: if there's a petget running while you try to unlink a module, it might occur that the unlinker takes a snapshot of the required libs BEFORE petget managed to add the installing dotpet's *.files to the package directory. Needless to say, that would be pretty bad if this dotpet would contain files that overwrite some of the symlinks.
A simple solution would be an Xdialog window telling the user that they need to make sure there are no other installations running while the SFS is unlinked. Add a yes-no option to the mix ("yes" starts unlinking, "no" cancels the process), and the prob would be solved as well. Of course, you could also use ps to check the system for running installation processes and either pause your own script or tell linux to pause the petgets and block any new instances of petget right away.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#47 Post by technosaurus »

jrb wrote:technosaurus,
This is the code that I used to create the symlinks from /mnt/SFSpackage to /

Code: Select all

cp -rs /mnt/"$SFSNAME"/* /
It doesn't overwrite other existing files or symlinks.
Wouldn't that generate an error message for files that exist that you could capture?
Something like

Code: Select all

cp -rs /mnt/"$SFSNAME"/* 2>>/tmp/DONOTUNLINK
then add a grep of /tmp/DONOTUNLINK in the unlink script (you may also need to use cut or something to remove the rest of the error message)

If the other way works though - that's great, just that sometimes find can be really slow
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#48 Post by WarMocK »

Damn, I knew there was something fishy about the whole thing with the modification.
@jrb: I think that diff is not the best way to check out what files are already there. I made a new attempt using "sort" and "uniq", which removes duplicate lines from the filedump before comparing it to a second dump with the sfs list included. These two dumps are then compared with sort AND uniq, sending only the unique files of the SFS to the standard output. Several checks with my .packages folder were successful, and it worked DAMN fast!:D

The code is:

Code: Select all

cat $HOME/.packages/*.files | sort -u > /tmp/dump0
mv $HOME/.packages/$SFSFILE $HOME
cat $HOME/.packages/*.files | sort -u > /tmp/dump1
sort /tmp/dump0 /tmp/dump1 | uniq -u > /tmp/difference
mv $HOME/$SFSFILE $HOME/.packages
Add this one to both the linker AND the unlinker, and use the generated difference list for adding symlinks or removing them (after you verified that it is a symlink at all). Please keep in mind that this check only works with the FULL list of the SFS content, otherwise you might accidently skip files that were added or removed after the sfs file was linked.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#49 Post by technosaurus »

@Warmock - sort |uniq is a nice useful combo especially with the latest busybox (use #! /bin/ash vs. #! /bin/sh)

this should give you the duplicates list in case you want an option for user override

Code: Select all

mv $HOME/.packages/$SFSFILE $HOME/$SFSFILE
cat $HOME/.packages/*.files | sort -u > /tmp/dump0
mv $HOME/$SFSFILE $HOME/.packages/$SFSFILE
cat $HOME/.packages/$SFSFILE /tmp/dump0 |sort |uniq -d > /tmp/duplicates #don't unlink automatically
#next line is optional depending on how you implement unlinking
cat $HOME/.packages/$SFSFILE /tmp/duplicates |sort |uniq -u > /tmp/difference #safe to unlink
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#50 Post by seaside »

I have alot of files that are not registered and probably others do as well.

This method of technosaurus's -

Code: Select all

cp -rs /mnt/"$SFSNAME"/* / 2>>DONOTUNLINK
would protect against removing files that should be kept and are not registered.

Another approach afterwards, might be something like this -

Copy SFSPACKAGE file to /tmp/SAFE-TO-UNLINK and then:

Code: Select all

for NAME in $(cat DONOTUNLINK)
do
  sed -ie "\|^$NAME\$|d" /tmp/SAFE-TO-UNLINK
done
This would match the linking errors (those files that couldn't be linked because they already existed) to the presumed installed SFSPACKAGE list and output a SAFE-TO-UNLINK list.

(Might even be faster - not sure, though.... :D :D )

s

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#51 Post by jrb »

OK guys,
Here it is. I used Technosaurus idea to generate an error list of files that already exist and then added that to the list of files contained in the sfs. Then I used WarMocks idea to sort and uniq the file list resulting in a list of files unique to the sfs and not already existing in the files system. This becomes the list of files that will be uninstalled. Seems to work very well.

I also took the precaution of putting in an abort if someone tries to install the same .sfs twice, without uninstalling. Not a problem before but now it would have resulted in an empty file list and no removals.

I used .pets and sfs's which contained some but not all of the same files. The only way I see to get in trouble is to install a .pet after installing a .sfs. I'm sure somebody will but what can you do?

Here's the new sfs_linker. Extract it, set it executable and place it in /usr/local/bin. If it works okay I"ll repackage the .pet and post it.

Bye for now, J
Attachments
sfs_linker-1.4.gz
For testing - extract, set executable and place in /usr/local/bin
(1.93 KiB) Downloaded 520 times

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

SFS-TCZ_Linker-1.4

#52 Post by jrb »

I have just uploaded SFS-TCZ_Linker-1.4.pet and sfs_linker_default_apps431-0.1.pet

SFS-TCZ_Linker-1.4 has some new safeguards against removing common files that were installed before the SFS that you are removing. I have also modified it to be Upup compatible.

Rather than have two versions of SFS-TCZ_Linker, one with modified /usr/local/bin/defaults and one without, I have made those into a seperate .pet for Puppy431. If you like them use this .pet. If there is a demand I will make .pets for other pups as well.

Enjoy, J

http://puppylinux.ca/members/choicepup/ ... et-md5.txt
http://puppylinux.ca/members/choicepup/ ... et-md5.txt

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#53 Post by WarMocK »

Hey jrb,
a dotpet installed after the sfs was linked shouldn't pose a problem if you both compare the original filelist of the sfs to the list of installed files (which also should have the *.files of the installed dotpet in it) and make sure the file you're about to delete is a dead symlink. All you need to do is to make sure there are no alterations to the filelist after you took a snapshpt of the .packages directory and sorted out the files that were not listed in the other *.files

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#54 Post by jrb »

WarMock wrote:a dotpet installed after the sfs was linked shouldn't pose a problem
I think I have workable solution. I"ve built a little script for sfs_unlinker:

Code: Select all

	#jrb - compare SFS filelist with other installed filelists.  
	#Duplicates should not be uninstalled
	mv /root/.packages/$SFSNAME.files /tmp/$SFSNAME.files 
	cat /root/.packages/*.files > /tmp/installedpackages.files
	cat /tmp/$SFSNAME.files >> /tmp/installedpackages.files
	sort /tmp/installedpackages.files | uniq -d >> /tmp/$SFSNAME.files
	#jrb - send twice to make them duplicates
	sort /tmp/installedpackages.files | uniq -d >> /tmp/$SFSNAME.files  
	sort /tmp/$SFSNAME.files | uniq -u > /root/.packages/$SFSNAME.files
As you can see it moves the SFSNAME.files to /tmp from /root/.packages then creates a new list of all the other installed files and then compares the SFSNAME.files to that, creating a new /root/.packages/SFSNAME.files with only files unique to the SFS listed.

Haven't much time to test right now but will work on it this weekend.

Thanks for the inspiration, J

OldYogi
Posts: 27
Joined: Sun 13 Jan 2008, 01:08

#55 Post by OldYogi »

Jrb,

this looks excellent. I have two dumb questions:
first -- I'm running the alpha version of Puppy 4.4 (i.e. Pupply 4.31-2) which has SFS Linker installed. Somehow I deleted the sfs_mnt_home folder in ~/my_links on the desktop. How do I restore it?

second: will this handle tcz files without modification? There is a link in the menu to the Tiny Core repository, but it isn't clear whether those are sfs3 or sfs4. I know they can be converted -- but will that happen automatically?

thanks.

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#56 Post by jrb »

OldYogi wrote:Somehow I deleted the sfs_mnt_home folder in ~/my_links on the desktop. How do I restore it?
I'm a little puzzled. If the SFS_TCZ_LInker script in Puppy4.4 is the same as mine it should make a new /root/my_links/sfs_mnt_home folder automatically after rebooting.

Have you tried opening that folder and making one manually? Rt-click in the folder, choose "New" - "Directory" and name it "sfs_mnt_home". Try that and let me know if it disappears.
will this handle tcz files without modification?
Yes. The .tcz files listed in that website are all SFS4's. Older tinycore .tcz's are SFS3's but they are in a different directory at ibiblio.

Keep me posted, J

reckrhodes
Posts: 116
Joined: Wed 30 May 2007, 08:15

#57 Post by reckrhodes »

Hello jrb!

This linker to sfs is a very useful one. Thank you so much for this. The improvements are so good. Hope that this contribution of yours will be included in all official or community pupplets.

I tried using the lucid puppy and it works so fine. Just converting the pet to tar.gz(pet2tgz) and copying tar.gz contents to a directory and converting directory to sfs(dir2sfs) is enough to load using your sfs linker.

Thanks also to sir BK and the Puppy Linux community.


Have a great day! :D :D

Eric
Cebu, Philippines

OldYogi
Posts: 27
Joined: Sun 13 Jan 2008, 01:08

#58 Post by OldYogi »

Jrb,

Yes, I did recreate the sfs_mnt_home directory -- but on reboot it doesn't pick up the Opera.tcz file.

I'm booting off of a usb, and as I have two versions of Puppy there, each in its own subdirectory, I put the .tcz file both in the top layer of the usb (mnt/home) and in the subdirectory (mnt/home/Puppy4312). But either way it doesn't display in sfs_mnt_home, which I assume it should do. When I click on it directly to install with sfs linker it turns up in the next folder in /my_links, suggesting it is loaded, an icon appears in the System menu, but it won't run.

I'm puzzled.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#59 Post by WarMocK »

@OldYogi,
did you run it from a terminal (like rxvt) as well? If you haven't done this yet, you should give it a try and see what error messages it gives you. In case it DOES give you some error messages, redirect the error output into a text file and post it here so we can see what's going on:

Code: Select all

$nameofprogram > /tmp/erroroutput

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#60 Post by jrb »

OldYogi wrote:I put the .tcz file both in the top layer of the usb (mnt/home) and in the subdirectory (mnt/home/Puppy4312). But either way it doesn't display in sfs_mnt_home, which I assume it should do.
It should, but it doesn't. The version of SFS Linker (1.2) in Puppy4.4 doesn't link .tcz's from /mnt/home to /sfs_mnt_home. An oversight on my part. Install the latest version (1.4) and it wil.
but it won't run
TinyCore breaks everything down into the smallest possible .tcz units. This is the list of dependencies from opera10.tcz.dep:

Code: Select all

qt-4.5-base.tcz
expat2.tcz
fontconfig.tcz
In actual fact puppy already has the last two. You only neet to install qt-4.5-base.tcz.
WarMock wrote:did you run it from a terminal (like rxvt) as well?
WarMock is exactly right. If you open a terminal and type

Code: Select all

opera
it will tell you any deps that are missing. You then have to track them down and install.

Post Reply