Addressing LOGS in Puppy removes some inconsistencies

[gcmartin]

I agonized on whether to post this here, for Puppy Distro development, or to Barry's Blog.

Hope the intent for the community is understood.

Logs in Puppy

Sometimes a suggestion to a development system is not met well unless you are considered an "insider".

But, also, sometimes the real insiders can see merit in someone making a suggestion that may improve the overall system.

Here's something I noticed and it appears that the insiders had planned to address this in the past, but, never got around to it.

System Logs
In many systems the rule is to place system logs in a single folder. It has does make a world of difference when trying to support the system as the system and application logs are all in the same folder.
For example, in couple of systems all logs are written to /var/logs

Currently, Puppy "kinda" does this. Many of the logs are written to /tmp. This is a good central location even though some logs do end in other places.
For those logs which do end in /tmp its where the system developers would look to analyze what's occurring. And, I envision that the Planner(s) for Puppy evaluated this initially as a temporary solution until they got back to it.

Here's why this MAY be time to consider what should best be done! Live media! Frugal! We all know what happens to /tmp when the system is shutdown/rebooted when save-session is selected! This is why I have felt that the planners had envision getting back to address this.
If anyone needed to review the log files from a Puppy previous run, they do NOT exist for review. Thus important/critical log information was lost due to reboot; an un-intentional occurrence.

Just as Linux has overall consistent structure, in 2012, as Puppy matures, it might be one of the several items that could be considered and addressed. Because I am not the system planner (or insider), I would not be in a good position to suggest how best to address this. But, I'm sure, we all see the benefit of a simplistic log system which could be available for off-line review with some controls on managing its size.

Hope this helps

[technosaurus]

Honestly, I would prefer them all go to one log, with appripriate prefix text...

MSG="......" Uuid=... Etc...

Then they could be parsed much easier.

[gcmartin]

Honestly, I would prefer them all go to one log, with appripriate prefix text...

MSG="......" Uuid=... Etc...

Then they could be parsed much easier.

Thanks @Techno. That's a database, you are referring to. I'm not so sure who would be in favor of adding a DB to the Puppy product set. Also, syslog does this, but, it's not available until further along in the boot process. And, last, but certainly not least, every developer of an application would need to be trained on proper logging and registrationing their log data.

But, it could have some wide-spread appeal if presented attractively. Not being a distro or product developer, i wouldn't know where to begin.

As a starter; If the Woof system was setup to start all of us off in a "logging" folder (a folder that would, somehow, survive reboots even temporarily), could set the mindset to get the community on-board for system/application logging. I understand that logging is ONLY applicable when trying to review what's occurring, and I understand that many of the system and apps we use produce this very useful data. If its save in a common lib, I'm sure all see the benefit to this. Its not high priority, but, its still very important as Puppy continues to mature.

In the meantime, it could serve Puppy with a "log" folder and at least those things Puppy in that log folder.

[amigo]

Flat-text-files for configuration and logging.

[Karl Godt]

you could look in /etc/rc.d/rc.shutdown for something like these :

Code: Select all

#when the working files run in tmpfs in ram, they are saved above and /tmp and /var
#are screened out. however, some PUPMODES mount ${DISTRO_FILE_PREFIX}save.2fs directly on /initrd/pup_rw,
#the top unionfs layer, meaning that there is no intermediary tmpfs in ram for working
#files, hence everything is saved directly, ditto for PUPMODE=2 a full h.d. install.
#hence need to do some explicit wiping here...
#echo -n "" > /var/log/messages #delete, as it keeps growing.(note choosepartfunc uses this)
rm -f /var/log/X*
#rm -rf /tmp/*  ##+--2011-11-10 to be done in /sbin/init
rm -f /var/lock/LCK*
rm -f /var/run/*.pid
rm -rf /root/tmp 2> /dev/null #...note, not screening this out in any of above save modes.

if test "$PUPMODE" != "2" ; then  ###KRG
FP=`df | grep '/pup_rw' | tr -s ' ' | cut -f5 -d ' ' | tr -d '%'`  ###KRG
if [ "$FP" -gt 89 ]; then  ###KRG
echo "Frugal Install ,free space less than 11%; removing thumbnails" > /dev/console  ###DBG
rm -rf /root/.thumbnails/* 2> /dev/null
else  ###KRG
echo "Frugal Install ,free space more than 10%; leaving thumbnails ok "
fi  ###KRG
else  ###KRG
FP=`df | grep -w '/' | tr -s ' ' | cut -f5 -d ' ' | tr -d '%'`  ###KRG
if test "$FP" -lt 5; then  ###KRG
echo "Full install , free space lesser than 5%; removing thumbnails"  ###KRG
rm -rf /root/.thumbnails/* 2> /dev/null  ###KRG
else  ###KRG
echo "Full install , free space more than 5%; leaving thumbnails ok"  ###KRG
fi  ###KRG
fi  ###KRG
sleep 3
sync
echo "RM UNEEDED FILES END"  ###KRG

which includes adjustments that i made for full installation .
My rc.shutdows look different everywhere until i had found the best suiting ones .

My /sbin/init andor rc.sysinit contains a line

Code: Select all

mv /tmp/* /var/tmp/

to have these logs rescued next boot - just in case .

For the moment i am not willing to decipher the rc.shutdown code if it would copy the /tmp folder into the savefile in odd pupmodes like 13 .

The actual puppy 5 codes are logging to /tmp but don't look or grep in /tmp directory anymore at boot
like Puppy 4 series did .

It was intended not to load modules twice , but everything gets forked at boot and the commands queue up in the cache of the cpu and get processed in a not ordered mode .
modprobe a module a second time does not harm , as i can see it .
modprobe --first-time could have done it too .
modprobe seems to recognize already loaded modules faster than lsmod for example and simply does not load a module twice by default .
But grep MODULE in /tmp/pup_event* IF not mv OR rm due to a hard poweroff/reset had /sbin/pup_event_backend_modprobe exited in Puppy 4 series .
This behavior had lead to not loading kernel modules at next boot having had a hard poweroff at Puppy 4 series .

The /sbin/pup_event_backend_modprobe_protect in Lupu is also a way to prevent module loading twice . Interesting code , but unfortunately not needed .
Meant Good != Done Good in this case .

Also the new Puppy 5 series /sbin/init script does a wiping of /var and /tmp files on full installations
if /fsckme.err or /fsckme.flg file existent at boot .

HTH a little

[Karl Godt]

looking into rc.shutdown if using a save file

/usr/sbin/snapmergepuppy

gets called to copy the files from tmpfs into the save file .

/usr/sbin/snapmergepuppy includes several

#Copy Files... v409 remove '^var'. w003 screen out some /dev files.
find . -mount -not \( -regex '.*/\.wh\.[^/]*' -type f \) -not -type d | sed -e 's/\.\///' | grep -v -E '^mnt|^initrd|^proc|^sys|^tmp|^pup_|^zdrv_|^root/tmp|_zdrv_|^dev/\.|^dev/fd|^dev/pts|^dev/shm' | grep -v -E -i '\.thumbnails|trash|\.part$'

with several grep -v to avoid copying them into the save file .

Using an entire partition at first shutdown

Code: Select all

#unionfs layers:                RW (top)      RO1             RO2              PUPMODE
#pup_save is a partition:   PDEV1                             pup_xxx.sfs      6
#ditto, but flash drive:         tmpfs         PDEV1           pup_xxx.sfs      7

The PUPMODE gets temporarily set to 32 and

RDIRS="`find /initrd/pup_rw/ -maxdepth 1 -mount -type d | grep -v "/$" |grep -v "/mnt"|grep -v "/tmp"|grep -v "/proc"|grep -v "/sys"|grep -v "/var"|grep -v "/dev" | grep -v "/lost" |tr "\n" " "`"
for ONEDIR in $RDIRS
do
cp -a $ONEDIR ${DEV1MNT}/
done

does the work to copy the files to the save-partition .

It is similar to the save to save-file . In this case PUPMODE 5 temporarily set to 128 .

In either cases a

Code: Select all

cp -a /tmp /path/to/dir/tmp"`date`"
cp -a /initrd/tmp /path/to/dir/initrdtmp"`date`"
cp -a /var/log /path/to/dir/varlog"`date`"

could do the trick to save log files .