Check all your ext* and vfat partitions automatically

How to do things, solutions, recipes, tutorials
Message
Author
musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

Check all your ext* and vfat partitions automatically

#1 Post by musher0 »

Now up to version 11b. See changes log below for added &
modified features.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hello, all.

Here is a little script that will help you maintain your partitions / disks in tip-top
shape. The script is well remarked, so I won't repeat myself!

Main features:
- a cinch to install and use; almost no intervention required on your part once the pet is installed;
- keeps your partitions running at optimal speed;
- runs four times a month, unobtrusive on other days; can also be run "standalone";
- easy browsing of fsck reports in the CLI window to spot any issues;
- NEW, March 25 2014:
-----> check partition types separately or one after the other;
-----> ANSI colors used;
-----> complete check each time.
- NEW, March 27 2014:
-----> CD/DVD types totally excluded, as well as add'l non-Puppy file systems.
-----> new longer, easier to read, window.


~~~~~~~~~~~
I am sharing this script in the hope of making everybody's computing life easier.
Even if the ext* partition systems are among the most stable, partition checking
must still be done from time to time, to maintain the health and speed of your
system
. Now you can run this script to "defrag" your drives and properly align
their nodes, check superblocks, etc., and forget about the repetitive typing part.

That said, there are two reasons this script uses the "verbose" mode:
a) you can consult the log that shows up for each drive. If a problem show up, you
will know;
b) each user has his/her personalized partition or drive configuration, which is why
I didn't favor a batch check for partitions.

Please note that this script is not a cure-all for partitions. You may still have to use
the more arcane fsck parameters on partition x or y, if it seems to have problems
like bizarre superblocks, etc.

~~~~~~~~~~~
A little how-to note is in order.

1) On the first boot of your Puppy after the script is installed, the script will run automa-
tically. It takes a few minutes, unfortunately, 'cause there is no hurrying the partition
correction process. Once it has finished, though, you should notice some speed
improvement in the general operation of your Puppy if you had never done a fsck of
your partitions before, or in a long time.

After that first time, though, the script will run automatically once a week. Six days
out of seven, then, a rectangle shows up at launch, towards the top-left of your
screen, saying that the script is active, but that it does not need to run.

If you need to run the script anyway between that time interval, click on the
"automverif" entry in your File System sub-menu, and follow the prompts.

2) The script checks ext2, ext3, ext4 and vfat partitions ONLY. The reason for
this constraint is as follows:

* ntfs partitions can only be checked reliably from a WhineDose OS. The ntfs utilities
on Linux allow the user to read-write on ntfs partitions, but it can only set the check
flag on the ntfs drive, not actually check it.

* there are reports on the Internet that the fsck.reiser utility is the "closest thing to
file system Lotto" http://linuxmafia.com/faq/Filesystems/reiserfs.html. I'm not
all that familiar with reiser fs (except on a KNOPPIX full install), so I decided to play
it safe and NOT offer the choice to use it, even if the utility is included in some
Puppies. Maybe a Puppy-ist with wide reiser fs experience will come up and offer
good advice on this subject, and then the reiserfs choice could be added, but until
then, I'm playing it safe.

* Puppies present a limited choice of file systems (which is OK for our needs, IMO).
If on your computer you have other Linuxes using other file systems, those distros
no doubt have proper partition checking utilities, so use those. The present script
leaves those partitions un-touched until you boot into that other distro to check
the partition(s) it resides on.

3) Don't overuse this script or fsck generally. Using fsck every day -- or worse,
every hour... -- doesn't improve the disk speed.

4) Other scripts are included in this pet, that you may find of use:

* Createfstab -- Populates the default Puppy fstab with whatever partitions blkid
shows up with. For meaningful results, it is important that you mount all your
partitions BEFORE you use this script.
Creating a static fstab comes in handy when you wish to mount or unmount
all your inactive partitions at once. With a static fstab, you can simply enter a terminal and type

Code: Select all

mount -a 
OR

Code: Select all

umount -a
and the job is done. To do the same in a dynamic environment, you have to issue
as many mount or unmount commands as you have partitions.

* rep-to-mnt -- Again based on blkid, this script creates a sub-directory in /mnt
for every partition you have on your system, if the proper corresponding sub-folders
are not already in /mnt.

* tunefs25.sh -- This script allows the user to set the mount count (how many times
a partition has been mounted) before fsck does anything. Can be set between 25
and 35 mounts. The "automverif" script ignores it and does a forced fsck anyway,
but it's good to have the option of setting the frequency of fs checks independently.

5) Following a picture hint by some1, starting with version 11, I have excluded
checking all CD / DVD file systems (audiocd, iso9660, udf). In previous versions,
only the iso9660 system was being excluded from the check. (Forgive me, some1!)

Enjoy! Any constructive comment welcome.

musher0

~~~~~~~~~~~~~
NB's. 1) Long list of version changes now condensed in attached zip.
____ 2) The free Monaco TTF font used in this script was taken at
http://www.eaglefonts.com/monaco-ttf-15 ... tompreview
It's a TTF Mono font, that is, a very nice non-proportional font usable in a
terminal.
Attachments
automverif11.txt.zip
Changes log.
(4.08 KiB) Downloaded 299 times
automverif-11b.pet
Latest version.
(71.86 KiB) Downloaded 458 times
automverif11.jpg
(31.42 KiB) Downloaded 398 times
automverif-10.pet
Kept for reference.
(71.82 KiB) Downloaded 399 times
Last edited by musher0 on Tue 08 Apr 2014, 16:11, edited 23 times in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#2 Post by sunburnt »

Hi musher; A really useful script.

I`m sure awk will do the grep and cut all in the one awk command.
I`m not an awk expert, best to ask technosaurus about it.

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

Re: Check all your ext* and vfat partitions automatically

#3 Post by L18L »

:oops:

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#4 Post by musher0 »

sunburnt wrote:Hi musher; A really useful script.

I`m sure awk will do the grep and cut all in the one awk command.
I`m not an awk expert, best to ask technosaurus about it.
Hi, sunburnt.

Thanks for the compliment. As to awk, you're right. I just started, I need to study it more.

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

Re: Check all your ext* and vfat partitions automatically

#5 Post by musher0 »

L18L wrote::oops:
My most sincere condolences for whatever is afflicting you. ;)
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#6 Post by musher0 »

Updated. See first post.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#7 Post by musher0 »

Hello, all.

I've updated the script to v. 3. Please see first post for details.

Enjoy!

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#8 Post by musher0 »

Hello, all.

Here's a handy interactive companion for automverif3.sh.

I ran < tune2fs -l > on my ext? drives and the intervals indicated were a mess.
One drive in particular had been mounted 698 times without having been checked.

Linux file systems are reknowned for their stability, and I wasn't worried about the
data access being at risk. Still, that number was no good. An uncleaned partition
gets slower and slower without you noticing it, and affects the reactiveness of your
entire system. I decided to do something about it.

This companion script sets the interval for checking your ext? partitions at a
reasonable interval of between 25 and 35 mounts. It requires your input.

Obviously, you don't access all your partitions at every session, but this way,
fsck will check all your ext? partitions at the same interval, based on the use
of the partition, when you run the automverif?.sh script.

Enjoy!

musher0

~~~~~~~~~~~~~
P.S. Unfortunately, tune2fs doesn't handle vfat partitions. Ring in if you know of
^^^^^ such an utility running in Linux. Thanks in advance.
Attachments
tuneDisks25.zip
(593 Bytes) Downloaded 361 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#9 Post by sunburnt »

I`m thinking there`s no facility for counting FAT mounts for such a utility.
Like Puppy should check /mnt/home before mounting and check+mounting the Save file.
Simple enough to setup a mount counter to read at boot and do the partition check.

musher; Is this partition checker what you wanted gui help with.?
.
Last edited by sunburnt on Wed 11 Dec 2013, 22:26, edited 1 time in total.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#10 Post by musher0 »

sunburnt wrote:I`m thinking there`s no facility for counting FAT mounts for such a utility.
Like Puppy should check /mnt/home before mounting and check+mounting the Save file.
Simple enough to setup a mount counter to read a boot and do the partition check.

musher; Is this partition checker what you wanted gui help with.?
.
Hi, sunburnt and all!

Answering in reverse order:
-- No, that was for the pupsave creation script.
I discovered that ASRI, from the French side of the forum, had already created
such an utility with GUI, at the end of October or thereabouts. It had escaped me.
He's good at what he does, so the pressure is off, there's no rush anymore.

-- IMO, since the partition checker is a rather low-level utility, there's no need for
anything fancy. Besides as it is, the user can move up and down the CLI window
to read the info.

-- As to the first problem: actually, if we use a time stamp, it's just as good, I think.
Drives should not be checked too frequently anyway, it's intensive use of them.
Maybe once every two weeks? This way, we sort of bypass the sad fact that in
Linux, we cannot know the last time a vfat drive was checked.

See my latest version, v.4. (I spent the night debugging it, but I think it was worth it!)
I promise it will be the last version for awhile. (Something like BK's retirement...) :lol:

BFN.

musher0
Last edited by musher0 on Wed 11 Dec 2013, 16:26, edited 1 time in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#11 Post by musher0 »

Hello, all.

Now up to version to v. 4. Please see first post for details. Thanks.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#12 Post by sunburnt »

Yep, time is almost as good as a counter, the counter is exact of course...

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#13 Post by musher0 »

sunburnt wrote:Yep, time is almost as good as a counter, the counter is exact of course...
True, but this is the best we can do if vfat drives. The ext? drives have the counter stored
on them somehow, so they say they're clean when it's not really the time to check them.
But begs the question: how to put a counter on a vfat drive? Maybe there's some doc
from the DOS era about this... BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
RetroTechGuy
Posts: 2947
Joined: Tue 15 Dec 2009, 17:20
Location: USA

#14 Post by RetroTechGuy »

musher0 wrote:
sunburnt wrote:Hi musher; A really useful script.

I`m sure awk will do the grep and cut all in the one awk command.
I`m not an awk expert, best to ask technosaurus about it.
Hi, sunburnt.

Thanks for the compliment. As to awk, you're right. I just started, I need to study it more.
Thanks for the script, Musher. I'll have to give it a try.

I found that AWK was even more obscure than most Linux commands... I had an old script I wrote because I didn't like the way "ls" displayed data (IIRC, using awk and sed to rearrange things).

And it is obscure right down to its name... What does A W K mean?

Aho, Weinberger, and Kernighan...

Boooooo!... ;-)
[url=http://murga-linux.com/puppy/viewtopic.php?t=58615]Add swapfile[/url]
[url=http://wellminded.net63.net/]WellMinded Search[/url]
[url=http://puppylinux.us/psearch.html]PuppyLinux.US Search[/url]

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#15 Post by musher0 »

Hi, R.T.G.

I hope you like my little partitions checker.

Yeah, AWK is a bit obscure from the outside. Slow but sure is the way to learn it. But it's
worth it, if only for the ease with which it can juggle items on a line (for example).

[Edited + URL added]
I got really interested when I read an article about a booking DB written in awk in only a few lines.
http://www.freesoftwaremagazine.com/art ... nks_to_awk

Besides, learning something new keeps my brain cells growing! :)

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#16 Post by sunburnt »

Once Puppy has booted to the point that PUPSTATE is sourced then:

Code: Select all

if [ "$DEV1FS" = vfat ];then
 PF=/etc/rc.d/vfat-boot-cnt
 [ -f $PF ]|| echo 0 > $PF
 CNT=$(<$PF)
 if [ $CNT -gt 32 ];then
 fsck.vfat
 echo 0 > $PF
 else
 echo $((CNT+1)) > $PF
 fi
fi
disktype could be used in place of PUPSTATE, and other ways to do it also.
# I haven`t tested this code... So it`s not proofed yet.
.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#17 Post by musher0 »

Hi, sunburnt.

I confess my failings. I had to comment your script to fully understand it! (See attached.
My IQ is only of...) :twisted: That said...

The logic of it is great! I know a little bit what PUPSTATE is and does, but please
explain where to find it, and most importantly, how we insert / hook up your routine
in the Puppy boot process.

I suspect we'd have to add in front of it something like

Code: Select all

(code in words)
if [ PUPSTATE number means Puppy is running frugal or full -a we have a vfat drive somewhere on this rig ]
   activate sunburnt's vfat-boot-counter
fi
Then all we need to do is use the counter setter routine alongside tunefs, ONCE, to set
the counter trigger for both ext? and vfat drives at, say, 32, as you suggest. We incorporate your routine in the partition checker, and we're all set!

(The above logic is still a bit fuzzy now, but I'm sure you get the idea.)

As a refinement, we could link the partition checker in ~/Startup, and voilà, disks will be
checked automatically every 32 boots.

BFN.

musher0
Attachments
vfat-boot-cnt.sh.zip
(670 Bytes) Downloaded 239 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#18 Post by sunburnt »

Hi musher; We`re talking about the Save file partition, so it must be checked early in init script.
init is where most of the Puppy setup occurs, the first image file mounted is the Save file.
Otherwise I put mounting code like this ( like my mnt-all.puppy script ) into: /etc/rc.d/rc/local
PUPSTATE is also in: /etc/rc.d/rc/local

Your addition isn`t needed I think, $DEV1FS tells what the Save partition is, that`s all we need.
In my PUPSTATE, $DEV1FS='ext3' I`m assuming vfat is the only fat designation used.
If it uses: vfat, fat32, fat16, msdos, etc., then my script would be need additional code added.
Perhaps this inplace of the first line: if [ "`echo $DEV1FS |grep fat`" ];then
This will use any $DEV1FS that has "fat" in it, but doesn`t account for "msdos" ( very old ).
.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#19 Post by musher0 »

Hello, everyone!

I've uploaded a new version, v. 5.

For the user, the main innovations of this version are:
- The archive is in pet format, easy to install.

- Automatic partition checks will be done once every fortnight. The wrapper queries the
day of the month and launches the script only on the 1st and 15th day of the month.

- This is done through a simple link in ~/Startup, so this disk-checking script will run at
every boot. A quick message appears if the time to check the partitions has not come.
In other words, the script reassures the user that it's there, but it checks the drives
only twice a month. I believe that this checking interval should be enough to keep our
partitions running at optimal speed.

Please see updated first post for other, more internal type, details.

Enjoy!

musher0
Last edited by musher0 on Thu 12 Dec 2013, 21:36, edited 1 time in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#20 Post by musher0 »

sunburnt wrote:Hi musher; We`re talking about the Save file partition, so it must be checked early in init script.
init is where most of the Puppy setup occurs, the first image file mounted is the Save file.
Otherwise I put mounting code like this ( like my mnt-all.puppy script ) into: /etc/rc.d/rc/local
PUPSTATE is also in: /etc/rc.d/rc/local

Your addition isn`t needed I think, $DEV1FS tells what the Save partition is, that`s all we need.
In my PUPSTATE, $DEV1FS='ext3' I`m assuming vfat is the only fat designation used.
If it uses: vfat, fat32, fat16, msdos, etc., then my script would be need additional code added.
Perhaps this inplace of the first line: if [ "`echo $DEV1FS |grep fat`" ];then
This will use any $DEV1FS that has "fat" in it, but doesn`t account for "msdos" ( very old ).
.
Hi, sunburnt.

Sorry for the late reply. I've been busy all day... :) (See post above this one.)

Thanks for the add'l info. Ok, so the vfat checking counter would go in /et/rc.d/rc.local.
Shucks, there'll have to be v. 5a! :)

But isn't the pupsave file checked automatically if you add pfix=fsck in the boot line parameter?

Or do you mean that the drive on which the pupsave resides should also be checked at each boot?

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Post Reply