Defrag FAT and NTFS from your puppy

Requests go here. If you fill a request, give it a new thread in the appropriate category and then link to it in the request thread.
Post Reply
Message
Author
User avatar
neerajkolte
Posts: 516
Joined: Mon 10 Feb 2014, 07:05
Location: Pune, India.

Defrag FAT and NTFS from your puppy

#1 Post by neerajkolte »

Before you all say, what, why, no need, hear me out.

I came across few post in which savefile got currupt and didn't load after all humms.. and dumms.. the problem got solved after booting in windows and defraging the drive.

That got me thinking is there an app in Linux that can defrag FAT or NTFS.

I went to search.....

In all the Ubuntu, Mint, Arch and many other forums I searched last three days.

There are questions like this, with similar answers,
Linux is so great it doesn't need defrag...
Ditch Windows altogether...
Use EXT2,3,4....
Take a backup, format drive, copy back data....
And so on, never actually answering the post in the first place.

Puppy being more friendly (both OS and Users), I am posting this request for a defrag tool as I couldn't find one here.

In my search though I came across ShAke and Pyfragtools.

I am using Fatdog64-700a primarily.
But I have other pups as well just haven't used them extensively.

Can a script be made so it can be done on all Puppies.

Thanks.

- Neeraj.
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson

“We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.â€￾
- Amara’s Law.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#2 Post by Burn_IT »

I too have spent ages looking for Linux based defrag' tools only to be told that Linux doesn't need it.

That is a load of codswallop. Not only is the fragmentation nothing to do with the OS, but more to do with the file format, but ALL file formats cause fragmentation.


By all means continue looking.

Ext2,3,4 and so on are more tolerant, but all file systems could do with the same background space gathering that takes place on SSDs.
"Just think of it as leaving early to avoid the rush" - T Pratchett

User avatar
neerajkolte
Posts: 516
Joined: Mon 10 Feb 2014, 07:05
Location: Pune, India.

#3 Post by neerajkolte »

I also found this script fromHere. Can any one take a look at it and tell me what it does..

Code: Select all

#!/bin/bash
# defrag v0.08 by Con Kolivas <kernel@kolivas.org
# Braindead fs-agnostic defrag to rewrite files in order largest to smallest
# Run this in the directory you want all the files and subdirectories to be
# reordered. It will only affect one partition. It works best when run twice.
# Are you really crazy enough to be using this? It might blow your data
# into tiny little useless chunks.


trap 'abort' 1 2 15

renice 19 $$ > /dev/null

abort()
{
	echo -e "\nAborting"
	rm -f tmpfile dirlist
	exit 1
}

fail()
{
	echo -e "\nFailed"
	abort
}

declare -i filesize=0
declare -i numfiles=0

#The maximum size of a file we can easily cache in ram
declare -i maxsize=$((`awk '/MemTotal/ {print $2}' /proc/meminfo`*1024))
(( maxsize-= `awk '/Mapped/ {print $2}' /proc/meminfo` ))
(( maxsize/= 2))

if [[ -a tmpfile || -a dirlist  ]] ; then
	echo dirlist or tmpfile exists
	exit 1
fi

# Sort in the following order:
# 1) Depth of directory
# 2) Size of directory descending
# 3) Filesize descending
# I made this crap up. It's completely unvalidated.

echo "Creating list of files..."

#stupid script to find max directory depth
find -xdev -type d -printf "%d\n" | sort -n | uniq > dirlist

#sort directories in descending size order
cat dirlist | while read d;
do
	find -xdev -type d -mindepth $d -maxdepth $d -printf "\"%p\"\n" | \
		xargs du -bS --max-depth=0 | \
		sort -k 1,1nr -k 2 |\
		cut -f2 >> tmpfile
	if (( $? )) ; then
		fail
	fi

done

rm -f dirlist

#sort files in descending size order
cat tmpfile | while read d;
do
	find "$d" -xdev -type f -maxdepth 1 -printf "%s\t%p\n" | \
		sort -k 1,1nr | \
		cut -f2 >> dirlist
	if (( $? )) ; then
		fail
	fi
done

rm -f tmpfile

numfiles=`wc -l dirlist | awk '{print $1}'`

echo -e "$numfiles files will be reordered\n"

#copy to temp file, check the file hasn't changed and then overwrite original
cat dirlist | while read i;
do
	(( --numfiles ))
	if [[ ! -f $i ]]; then
		continue
	fi

	#We could be this paranoid but it would slow it down 1000 times
	#if [[ `lsof -f -- "$i"` ]]; then
	#	echo -e "\n File $i open! Skipping"
	#	continue
	#fi

	filesize=`find "$i" -printf "%s"`
	# read the file first to cache it in ram if possible
	if (( filesize < maxsize ))
	then
		echo -e "\r $numfiles files left                                                            \c"
		cat "$i" > /dev/null
	else
		echo -e "\r $numfiles files left - Reordering large file sized $filesize ...                \c"
	fi

	datestamp=`find "$i" -printf "%s"`
	cp -a -f "$i" tmpfile
	if (( $? )) ; then
		fail
	fi
	# check the file hasn't been altered since we copied it
	if [[ `find "$i" -printf "%s"` != $datestamp ]] ; then
		continue
	fi

	mv -f tmpfile "$i"
	if (( $? )) ; then
		fail
	fi
done

echo -e "\nSucceeded"

rm -f dirlist
Thanks.

- Neeraj.
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson

“We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.â€￾
- Amara’s Law.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

Re: Defrag FAT and NTFS from your puppy

#4 Post by Sylvander »

neerajkolte wrote:Take a backup, format drive, copy back data....
Don't know if it's still the same with the latest versions of Windows, but...
That was the method that made the best and quickest defrag of the contents of my Win2000Pro FAT32 partition.
Actually, I used Xfe to delete the contents rather than re-formatting the partition.
Everything was totally contiguous; no empty regions or odd files left at the end of the partition, except...
The only imperfection was a small region at the beginning of the partition, used for and during the process and left empty.
Even the best method I'd used within Windows [O&O Defrag 2000] wasn't that good.

Here's my original 2009 record of the procedure.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#5 Post by Burn_IT »

That method misses de-fragging the file allocation tables.
"Just think of it as leaving early to avoid the rush" - T Pratchett

User avatar
neerajkolte
Posts: 516
Joined: Mon 10 Feb 2014, 07:05
Location: Pune, India.

#6 Post by neerajkolte »

That will need empty partition or backup drive too.

Which some puppy user's might not have...
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson

“We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.â€￾
- Amara’s Law.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#7 Post by Flash »

Reformatting a partition doesn't delete the file allocation tables?

User avatar
neerajkolte
Posts: 516
Joined: Mon 10 Feb 2014, 07:05
Location: Pune, India.

Re: Defrag FAT and NTFS from your puppy

#8 Post by neerajkolte »

Sylvander wrote: Actually, I used Xfe to delete the contents rather than re-formatting the partition.
@Flash, he didn't reformat the partition.


Anybody took a look at ShAke link I posted It looks rather new and in continuous devlopment.
Pyfragtools looks old but people say it works.

Thanks.
- Neeraj
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson

“We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.â€￾
- Amara’s Law.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#9 Post by Sylvander »

Burn_IT wrote:That method misses de-fragging the file allocation tables.
Didn't know that. :oops:
So I've just learned something. :D
Hope I can remember that.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#10 Post by Burn_IT »

The FAT gets fragmented as well.

In NTFS Microsoft call it the MFT, but it is the same thing.

Fragmentation of the FAT is more significant than fragmentation of the data since the FAT is accessed for EVERY file accessed, whereas fragmentation of the file only? affects that file.

Again in NTFS many small files are actually stored in the MFT.
"Just think of it as leaving early to avoid the rush" - T Pratchett

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#11 Post by amigo »

Con Kolivas solution does basically the same as shake. Re-formatting a partition overwrites the FAT/MFT but does not remove/alter existing data blocks. Removing them with any file manager also does not actually remove the data -only the metadata.

One could actually sort the files before copying them in order to optimize the read-ahead speed of the drive...

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#12 Post by p310don »

compress all files on disk into one file, uncompress.

Post Reply