Defragment Linux file systems
Defragment Linux file systems
On a hard drive, I had occasion to move three frugally installed puppies
from one ext2 partition to another (empty) partition. I was surprised to
experience a significant performance improvement when using the
moved pups. I thought "hmm, just like defragging Windoze".
It turns out that I had indeed defragged, and stumbled on the
recommended method of doing so (in a way). Googling "linux
defragment" turned up some good articles and explanations. Simply
moving the files out of a partition and back again will do the trick.
To check that everything is cool on sda2 (for example), in a terminal type:
fsck /dev/sda2 -f
(sda2 cannot be mounted for this check)
This will provide you with info on "non-contiguous inodes" which is linux-
speak for "fragmentation" I don't know how much fragmentation there
was before I moved pups to a different partition. I don't know how much
will likely lead to significant performance increase after defrag.
Art
from one ext2 partition to another (empty) partition. I was surprised to
experience a significant performance improvement when using the
moved pups. I thought "hmm, just like defragging Windoze".
It turns out that I had indeed defragged, and stumbled on the
recommended method of doing so (in a way). Googling "linux
defragment" turned up some good articles and explanations. Simply
moving the files out of a partition and back again will do the trick.
To check that everything is cool on sda2 (for example), in a terminal type:
fsck /dev/sda2 -f
(sda2 cannot be mounted for this check)
This will provide you with info on "non-contiguous inodes" which is linux-
speak for "fragmentation" I don't know how much fragmentation there
was before I moved pups to a different partition. I don't know how much
will likely lead to significant performance increase after defrag.
Art
I'm glad you brought the subject back, artsown!
Fragmentation is something that occurs in magnetized hard drives, whatever
the OS. It is a physical thing, as in "physics".
On Linux, 10 % hard disk fragmentation is acceptable, although ever seen,
because the Linux file systems ("ext" or others) have incorporated ways NOT
to let fragmentation happen.
You will find my script "automverif" for hard drives, here. ("Automverif"
stands for automatic verification.) Please use the latest version if you do.
If I may... When you say:
were, and the fragmentation rate will not change. You have to copy a rather
BIG FILE to the original disk before you recopy your big files for some
defragmentation to happen. This BIG FILE can be full of zeroes and you may
even erase it later, but the disk table used by the file system has to change.
The absolute defragger remains, as you guessed in your 1st paragraph:
-- copy all files from partition A to partition B.
-- reformat partition A
-- recopy all files from partition B to partition A.
BFN.
Fragmentation is something that occurs in magnetized hard drives, whatever
the OS. It is a physical thing, as in "physics".
On Linux, 10 % hard disk fragmentation is acceptable, although ever seen,
because the Linux file systems ("ext" or others) have incorporated ways NOT
to let fragmentation happen.
You will find my script "automverif" for hard drives, here. ("Automverif"
stands for automatic verification.) Please use the latest version if you do.
If I may... When you say:
I beg to differ! If you only do that, the files will go back where theySimply moving the files out of a partition and back again will do the trick.
were, and the fragmentation rate will not change. You have to copy a rather
BIG FILE to the original disk before you recopy your big files for some
defragmentation to happen. This BIG FILE can be full of zeroes and you may
even erase it later, but the disk table used by the file system has to change.
The absolute defragger remains, as you guessed in your 1st paragraph:
-- copy all files from partition A to partition B.
-- reformat partition A
-- recopy all files from partition B to partition A.
BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
Hi musher0,
As I understand from the articles I've read, the Linux ext2, ext3 and ext4
(at least) distribute files over the entire partition, keeping them as
separated as possible, so when they grow they can grow without
fragmenting them. If this is true, then it makes sense to me that using
very large partitions having only a few small pups in them should
minimize fragmentation.
Now, it so happens that I have no use for 250 gig hard drive space. So
I created three "small" partitions for various purposes, plus one partition
that's 179 gig. My three little pups drown in that huge space.
So my current experiment with this old pentium D class machine is to
see if any fragmentation of significance of the pup files eventually
occurs. Nutty, huh?
Do you have reason to think it will? If so, why?
Art
As I understand from the articles I've read, the Linux ext2, ext3 and ext4
(at least) distribute files over the entire partition, keeping them as
separated as possible, so when they grow they can grow without
fragmenting them. If this is true, then it makes sense to me that using
very large partitions having only a few small pups in them should
minimize fragmentation.
Now, it so happens that I have no use for 250 gig hard drive space. So
I created three "small" partitions for various purposes, plus one partition
that's 179 gig. My three little pups drown in that huge space.
So my current experiment with this old pentium D class machine is to
see if any fragmentation of significance of the pup files eventually
occurs. Nutty, huh?
Do you have reason to think it will? If so, why?
Art
Hello, artsown.
I'm not saying you need my script! Use it or not -- as you wish.
What I'm saying is --
As hard as we try to limit defragmentation, it is a law of nature concerning
magnetized stuff: any large file stored as ones (1) and zeroes (0) on a hard
drive AND used often will eventually become fragmented.
I think you have a logical approach to it, though.
For the record, Linux does have a FILE defrag utility, written in Python:
https://github.com/ThomasCX/defragfs
One advantage is that the partition the file is on can stay mounted.
But be warned, you will have to repeat the process many times before your
big file finally congeals into one single fragment.
Finally, my preferred one-liner for doing an fsck of partitions iswhere $2 is the ext type and $1 is the partition.
The "-p" parameter sets automatic correction on.
Best of luck with your approach.
BFN.
I'm not saying you need my script! Use it or not -- as you wish.
What I'm saying is --
As hard as we try to limit defragmentation, it is a law of nature concerning
magnetized stuff: any large file stored as ones (1) and zeroes (0) on a hard
drive AND used often will eventually become fragmented.
I think you have a logical approach to it, though.
For the record, Linux does have a FILE defrag utility, written in Python:
https://github.com/ThomasCX/defragfs
One advantage is that the partition the file is on can stay mounted.
But be warned, you will have to repeat the process many times before your
big file finally congeals into one single fragment.
Finally, my preferred one-liner for doing an fsck of partitions is
Code: Select all
fsck."$2" -p -f -v /dev/"$1"
The "-p" parameter sets automatic correction on.
Best of luck with your approach.
BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
This is only about hard drives.
I tried to do something about fragmentation in Linux formats and never found any program that worked that good.
As you say. Seems you had to run the program many times to really see any improvement.
To eliminate the possibility of fragmenting. (ext3, ext4 formats)
Make sure you always have some free space in a Linux formatted partition.
That gives it room to self adjust the location of files, as needed, to stop fragmentation.
I just checked a partition formatted ext 4 that I have used for a long time and has a lot of stuff on it that has been deleted, added, moved around, etc.........
The check results:
Total/best extents 37004/35921
Average size per extent 990 KB
Fragmentation score 0
[0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
This device (/dev/sdc7) does not need defragmentation.
This is already in most Puppies, but it only works on ext4 formats.
The partition has to be mounted to check.
Example:
to check partition sdc3.
e4defrag (-v) option will run the DE-fraging process.
1
I tried to do something about fragmentation in Linux formats and never found any program that worked that good.
As you say. Seems you had to run the program many times to really see any improvement.
To eliminate the possibility of fragmenting. (ext3, ext4 formats)
Make sure you always have some free space in a Linux formatted partition.
That gives it room to self adjust the location of files, as needed, to stop fragmentation.
I just checked a partition formatted ext 4 that I have used for a long time and has a lot of stuff on it that has been deleted, added, moved around, etc.........
The check results:
Total/best extents 37004/35921
Average size per extent 990 KB
Fragmentation score 0
[0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
This device (/dev/sdc7) does not need defragmentation.
This is already in most Puppies, but it only works on ext4 formats.
The partition has to be mounted to check.
Code: Select all
e4defrag -c /dev/partition ID.
Code: Select all
e4defrag -c /dev/sdc3
e4defrag (-v) option will run the DE-fraging process.
1
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
@bigpup
There's a issue with ext4 when using Fatdog's humungus initrd to get
fast booting. Ext4 does the opposite, causing bootup to be much
slower than when using ext2 or ext3. Jamesbond and Kirk have
talked about this.Kirk prefers ext3.
To me, this is unfortunate since e4defrag looks attractive, and
fatdog 721 is one of my favorite pups. Right now, I've split my little
kennel so I can use ext4 for xenialpup64 and tahrpup64, and ext3 (now)
for fatdog. Just a bit awkward, but no big deal
Art
There's a issue with ext4 when using Fatdog's humungus initrd to get
fast booting. Ext4 does the opposite, causing bootup to be much
slower than when using ext2 or ext3. Jamesbond and Kirk have
talked about this.Kirk prefers ext3.
To me, this is unfortunate since e4defrag looks attractive, and
fatdog 721 is one of my favorite pups. Right now, I've split my little
kennel so I can use ext4 for xenialpup64 and tahrpup64, and ext3 (now)
for fatdog. Just a bit awkward, but no big deal
Art
Wonder what would show for that ext4 partition, if Fatdog was installed to it.
Code: Select all
e4defrag -c
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
Sorry to sidetrack things but what was the reason for using the large initrd rather than using a "base sfs" file like a typically puppy?artsown wrote:@bigpup
There's a issue with ext4 when using Fatdog's humungus initrd to get
fast booting. Ext4 does the opposite, causing bootup to be much
slower than when using ext2 or ext3. Jamesbond and Kirk have
talked about this.Kirk prefers ext3.
To me, this is unfortunate since e4defrag looks attractive, and
fatdog 721 is one of my favorite pups. Right now, I've split my little
kennel so I can use ext4 for xenialpup64 and tahrpup64, and ext3 (now)
for fatdog. Just a bit awkward, but no big deal
Art
P.S. Why not put the fatdog64 save file on the ext4 partition and put the initrd file on a different partition? Heck, you could put the initrd on a fat partition if you wanted (I think).
@ s43a
The humungous initrd can speed up bootup times, as I had said.
In my experiences, the speedups have been quite impressive.
Your idea of placing the fatdog save file only on a ext4 partition
makes sense. I implemented this and so far so good I'm considering
the idea of placing only save files/folders in a partition that's set aside
for fragmentation checks and possible defrags.
Thanks for the idea!
Art
The humungous initrd can speed up bootup times, as I had said.
In my experiences, the speedups have been quite impressive.
Your idea of placing the fatdog save file only on a ext4 partition
makes sense. I implemented this and so far so good I'm considering
the idea of placing only save files/folders in a partition that's set aside
for fragmentation checks and possible defrags.
Thanks for the idea!
Art