(old)Puppy Linux Discussion Forum

I was writing a volume monitor plugin for GIO, this thought came to me.

I explored the /sbin/pup_event_backend_* files, and as far as I can understand I think it gets almost entire hardware information from udev.

If we rewrite the event manager backend in C, we get the following advantages:

• Faster execution: Programs written in compiled languages obviously run faster.
• No synchronization problems: I also observe synchronization explicitly forced through pup_event_backend_modprobe_protect. Such thing is already achieved in C without synchronization primitives.
• pup_event_backend and volume monitor backend can be combined together resulting in a single process efficiently managing module and firmware loading and at the same time managing drive hot-plugging events and broadcasting them to puppy's programs as well as GIO applications.

My design strategy is to move the drive hot-plugging management to the backend so that frontend doesn't need to carry out its own probing and at the same time probe results can be shared among all GIO applications. The result: We can see drive icons on the desktop as well as in the sidebar of the file chooser, just like that in Ubuntu (GTK uses GIO to get drive information).

For the frontend, I think the following design changes:

• Icons can be drawn on a separate window. This means drive icons will work with any desktop manager or pinboard, and even with no pinboard at all.
• Use of GIO API instead of polling sysfs. This means reusing polling results of the backend as mentioned before and also much better response to drive plugging and removing. This also means we can readily start off with development of frontend without waiting for backend's GIO modules to be complete. (In that case the frontend has to be tested on Ubuntu, obviously)

Note pup-volume-monitor has been moved to http://www.murga-linux.com/puppy/viewtopic.php?t=78881

Dear awash_rawal,

So far it looks very nice. Haven't studied the source yet but I think you are on to something good....

With kind regards,
vovchik

I personally think firmware loading is working pretty well, and there would be little benefit to rewriting it.

The frontend is worth a discussion.

Ideally, it would be more modular. The bits that handle the icons should be separate modules, so the developer can swap them out easily depending on the what the desktop environment is.

Keep the icon handling as scripts. You don't want that part in C as it is not flexible or transparent enough.

I seem to remember Barry or someone talking about something similar fairly recently.

Personally I think the world could do with a good lightweight frontend to udisks...

jemimah wrote: The bits that handle the icons should be separate modules, so the developer can swap them out easily depending on the what the desktop environment is.

Drive icons could be rendered in its own (pseudo)transparent window. So even that swapping is not required. Same code would work everywhere.(We have the convenience of using X11 when we are using C.)

Yes, there wouldn't be much gain in rewriting the backend, other than a small performance gain. The only main highlight is a design change: moving the drive hot-plugging management to the backend. (Basically it means moving the entire event handling to the backend.) Also, sharing of events with other programs is another plus point.

That could be one option, but don't force that on the user either.

The desktop provides a lot more functionality than just drawing the icons.

Check out how dhcpcd handles its scripts. The program itself is in C but every time an event occurs it calls every script in the dhcpcd-hooks directory with an argument describing the event.

That way you can easily add or remove scripts, and the scripts themselves decide what to do with various events. If a user wants to install a new desktop environment, the hooks needed to properly respond to drive events can be installed as part of the package.

I have already implemented a somewhat equivalent thing in my incomplete volume monitor, for shell scripts.

While the pup-volume-monitor is running, one can run pup-volume-monitor-admin --listen-drv. The command then waits listening to events from pup-volume-monitor and directs them to standard output.

Your suggestion seems to be better, as many programmers (if not, then at least me) don't like writing daemons as they have to be started from somewhere. I'm going to implement it.

Anyways the user will not be forced, my plans are to maintain the frontend as a separate package (and hence a separate process) which itself uses GIO to get its information. In turn, GIO will use backend supplied modules to communicate with the backend. So if user doesn't want the frontend, it simply won't be started and Puppy can still function normally. This will also make it possible to run the frontend without backend, but in that case GIO has to be implemented using GVFS. (GIO doesn't bundle any implementations)

As of the last time I built it, gvfs needs hal or gnome-disk-util to do anything useful.

If you pull in hal, I don't think there's any point to this exercise at all is there?

Ideally we need a solution with no hal, and no dbus.

Dear guys and gals,

Totally agree with Jemimah about hal and dbus...let us avoid unnecessary dependencies.

With kind regards,
vovchik

jemimah wrote: If you pull in hal, I don't think there's any point to this exercise at all is there?

Ideally we need a solution with no hal, and no dbus.

Exactly.

I am ensuring to use minimum of dependencies, which have already been in puppy. Currently the volume monitor uses only GLib, libblkid and libudev. The modules have only GIO as dependency.

So finally it is confirmed that the frontend is to be rewritten.

But status of the backend is still unconfirmed. What is the final verdict? Should backend be written in C?

If yes, the volume monitor will be converted into the pup-event-backend. Then some of us could implement module and firmware loading (see a note about monitor.c in first post) while I finish writing GIO modules and then write the universal pup-event-frontend using GIO. I will also implement jemimah's suggestion in the backend (see sixth post), so users with special needs can easily write their own frontends without hassles of probing drives.

If no, then volume monitor remains volume monitor.

Writing something in C does not in itself guarantee that it will be faster, though this is generally the case, since typically programmers who write in C are decent programmers by comparison. Compare fixmenus (which uses tools written in C to generate a basic menu from a template) to jwm_menu_create (all standard shell script and generates either a basic menu or with submenus as well as a drive icons tray) _If_ I knew of tools in C that could do the same stuff built into shell, a C version would be even faster.

Sometimes good programming involves knowing the best language for writing the tool (for example if you have a slow shell script that is parsing one really long file and doing a lot of sed, grep and non-integer math, then well-written awk _could_ give a huge performance boost using its builtins in a short period of time with a small amount of code, comparable to a larger compiled C program that would also take much longer to write/debug)

My point here is just that once you know the full problem it is easier to choose the right solution. Unfortunately I really don't understand the "problem" ... is it the daemon process to monitor events? ... while read LINE; do case "$LINE" in ....*);; esac; done </var/log/messages ... pretty easy/fast to have a case statement for any and all kernel events that way (not much more difficult in C btw - probably just needs a series of strstr)

...or is it the generation of devices etc... jwm_menu_create and a couple other of my examples in sit (simple icon tray) uses /sys/block/... (this part _is_ AFAICT much more difficult in C - would need a filetree walk like ftw or nftw and a lot of dirent comparisons for the likes of "sd" to filename[0] filename[1] etc... though I do have a method for doing this part with a switch case and bit shift operation, which are much simpler/faster than a series of if strcmp()... )

Are you just wanting to patch gio to not require gvfs (I looked into this before with regard to abiword falsely assuming gvfs on gtk>2.14 which leaves gtk with a broke uri handler - the code seemed to be pretty deeply nested wrt the uri handling and I never got to the root problem) or just rewrite the backend for the event handler (in which case I could at least improve the script speed and/or help with the C rewrite - just link me to the backend source and I'll take a look)?

The problem with shell is not that it is slow. Shell scripts, when running without using external programs, have acceptable speed. The performance issues start once we start extending it.

If we want a new functionality in a C program, we link a library and work using functions in the library. To do same thing in shell we execute external programs like grep, sed, etc. This involves loading programs to memory which is the actual overhead in a shell program, which usually makes it thousand times slower than equivalent C program. In contrast to this, a function call in C merely involves a few assembly instructions.

Also for each such executable you can always find an equivalent library or library function that does just the same, or better. We aren't limited to standard C library.

The problem discussed here is not just performance issues. The problem is also to 'unlock' drive management features present in many programs which were unused because of absence of backends in GIO. Till now, the ROX desktop was the only way to access the drives. If user uses another DE, (s)he is helpless unless (s)he can hack /sbin/pup_event_frontend_d .

I suggest handling all events from udev in one program, and then after performing usual module and firmware loading, probe for newly connected or disconnected drives and then relay this information to all needed programs using or without using GIO. There are already many programs using GIO for drive management (including GTK itself for its file chooser). Also there are many scripts in puppy who probe the system themselves for drives.

Note that I am not suggesting an alternative to udev. (In fact my volume monitor needs udev to be running to detect drive hot-plugging events.)

As far as GIO is concerned, I am implementing only volume monitoring portion by writing GIO modules, as probably that's all we need. We don't need a full VFS backend like gvfs. (From your feedback about abiword it seems abiword needs a full VFS, but this usually shouldn't happen as usually GIO silently resolves the issues like absence of backends.)

The original source code for pup_event_backend is at /sbin/pup_event_backend_* scripts. Also see /etc/udev/rules.d/50-udev-puppy-basic.rules . I am unable to understand it fully, so I need help in this context.

The fix for uri handling is here: http://murga-linux.com/puppy/viewtopic.php?t=77295

I checked the latest version of gvfs and it now has a disk monitor that uses udisks instead of HAL or GDU. But udisks depends on PolicyKit (which needs PAM) which seems extremely inappropriate for Puppy.

I'm still not seeing a good way to get rid of pup_event_frontend_d for non-ROX environments.

Apparently, I'm not the only one complaining about this:
http://forum.xfce.org/viewtopic.php?id= ... 9NKbaR5kjQ

The problem discussed here is not just performance issues. The problem is also to 'unlock' drive management features present in many programs which were unused because of absence of backends in GIO. Till now, the ROX desktop was the only way to access the drives. If user uses another DE, (s)he is helpless unless (s)he can hack /sbin/pup_event_frontend_d .

I agree with this, but I think this has been solved in other ways before. I remember seeing floating drive tray icons in the days of puppy4. Not sure where what's the state of that now, and what was used to implement it.

But doing it in C, especially tying to GTK/GIO - hmmm ... upgrade GTK and stuff breaks. Well not always, but sometimes they do. I would rather we have a non-X daemon that communicates with a GUI tool. The GUI tool can change time to time (GTK/Qt/Xlib/whatever), but the daemon stays the same, so I prefer that daemon to be in script instead of C.

As jemimah said before, though (welcome back Jemimah, glad to see you back from hibernation), pup_event_frontend_d is doing much more than just icons. I'm also considering to replace it - but before I can do that I must understand every aspects of it Needless to say, there are many hooks beyond that file everywhere in the system (to name a few: save2flash, mount, umount ...)

If you still insist in C, like technosaurus said in another thread, there is always tcc and you can make a C script (yes, a C script - the real thing, not an oxymoron).

@Jemimah - I was considering udisk, but now you tell me it depends on a lot of other stuff that I don't want, I think I'd stay away. I thought freedesktop.org would implement lightweight stuff and let the DE takes care of the full functionality? Hmmm...

jamesbond wrote:
@Jemimah - I was considering udisk, but now you tell me it depends on a lot of other stuff that I don't want, I think I'd stay away. I thought freedesktop.org would implement lightweight stuff and let the DE takes care of the full functionality? Hmmm...

PolicyKit looks like a hard dependency. Maybe it can be hacked. I haven't tried that yet.

@jamesbond
yes, I made a script for jwm that is part of my jwm_tools package and later adapted it as an example in "Sit" my simple icon tray here:
http://www.murga-linux.com/puppy/viewtopic.php?t=76431
but the code could just as easily be adapted to work with wbar or whatever (I don't know for sure on the gnome-centric file managers though, specifically because I avoided them after discovering they all had this issue ... eventually pushing me back to gtk1.2) - I could probably make a hacky version of gvfs based on it - enough to work with gio, but every time I have sat down to look for it, I have lost interest before finding the exact code - maybe jemimah's link to the patch will help point me in the right direction. ... I'd really like to get glib/gtk "fixed" before the next major abiword release though - last time I had to build it on Puppy-4.1 with a lot of rebuilt libs just to get it working.

btw scripting with tcc is a good tool for _developing_, but never as fast as precompiled ... but it may also save some size on smaller code <~4kb (elf garbage)

jemimah wrote:

jamesbond wrote:
@Jemimah - I was considering udisk, but now you tell me it depends on a lot of other stuff that I don't want, I think I'd stay away. I thought freedesktop.org would implement lightweight stuff and let the DE takes care of the full functionality? Hmmm...

PolicyKit looks like a hard dependency. Maybe it can be hacked. I haven't tried that yet.

I just had a look at the code. I was able to get it to compile and run without policykit. I'm not sure it's working, but I will try to get it to work for Saluki-2. It'd would be nice to have native disk handling in thunar.

jamesbond wrote: But doing it in C, especially tying to GTK/GIO - hmmm ... upgrade GTK and stuff breaks. Well not always, but sometimes they do. I would rather we have a non-X daemon that communicates with a GUI tool. The GUI tool can change time to time (GTK/Qt/Xlib/whatever), but the daemon stays the same, so I prefer that daemon to be in script instead of C.

I am not trying to change the source code of GIO or GTK. Nor gvfs does.

I am writing a GIO module, so that we don't need to install gvfs just for volume monitoring features. Nothing breaks in case of any upgrades unless GIO introduces incompatible API changes (which they must mention in documentation). This is not a hack.

I think there is a lot of confusion regarding this. Let me clarify.

Here's my design of the event management system:


The GIO module that I am writing is a part of this implementation. It will make the programs communicate with the backend. So far I am having a good progress, without those heavy dependencies like gnome-disk-utility, udisks, gudev, PolicyKit, or even dbus.

The reason that you mentioned actually favours the backend to be written in C. The design you mentioned is already present in my volume monitor. It is in fact very easy to implement in C.

The frontend will be completely different process, outside of the event management. It will communicate with the backend just like all other programs.