Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 24 Oct 2014, 11:54
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Suggestions
Rewriting pup_event in C
Moderators: Flash, Ian, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 7 Posts_count   Goto page: 1, 2, 3, 4, 5, 6, 7 Next
Author Message
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Sun 08 Apr 2012, 02:40    Post_subject:  Rewriting pup_event in C
Sub_title: Can lead to a faster puppy and many more advantages
 

I was writing a volume monitor plugin for GIO, this thought came to me.

I explored the /sbin/pup_event_backend_* files, and as far as I can understand I think it gets almost entire hardware information from udev.

If we rewrite the event manager backend in C, we get the following advantages:

  • Faster execution: Programs written in compiled languages obviously run faster.
  • No synchronization problems: I also observe synchronization explicitly forced through pup_event_backend_modprobe_protect. Such thing is already achieved in C without synchronization primitives.
  • pup_event_backend and volume monitor backend can be combined together resulting in a single process efficiently managing module and firmware loading and at the same time managing drive hot-plugging events and broadcasting them to puppy's programs as well as GIO applications.

My design strategy is to move the drive hot-plugging management to the backend so that frontend doesn't need to carry out its own probing and at the same time probe results can be shared among all GIO applications. The result: We can see drive icons on the desktop as well as in the sidebar of the file chooser, just like that in Ubuntu (GTK uses GIO to get drive information).

For the frontend, I think the following design changes:

  • Icons can be drawn on a separate window. This means drive icons will work with any desktop manager or pinboard, and even with no pinboard at all.
  • Use of GIO API instead of polling sysfs. This means reusing polling results of the backend as mentioned before and also much better response to drive plugging and removing. This also means we can readily start off with development of frontend without waiting for backend's GIO modules to be complete. (In that case the frontend has to be tested on Ubuntu, obviously)


Note pup-volume-monitor has been moved to http://www.murga-linux.com/puppy/viewtopic.php?t=78881

Edited_times_total
Back to top
View user's profile Send_private_message 
vovchik


Joined: 23 Oct 2006
Posts: 1285
Location: Ukraine

PostPosted: Sun 08 Apr 2012, 09:17    Post_subject:  

Dear awash_rawal,

So far it looks very nice. Haven't studied the source yet but I think you are on to something good....

With kind regards,
vovchik
Back to top
View user's profile Send_private_message 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL

PostPosted: Sun 08 Apr 2012, 13:12    Post_subject:  

I personally think firmware loading is working pretty well, and there would be little benefit to rewriting it.

The frontend is worth a discussion.

Ideally, it would be more modular. The bits that handle the icons should be separate modules, so the developer can swap them out easily depending on the what the desktop environment is.

Keep the icon handling as scripts. You don't want that part in C as it is not flexible or transparent enough.
Back to top
View user's profile Send_private_message Visit_website 
disciple

Joined: 20 May 2006
Posts: 6449
Location: Auckland, New Zealand

PostPosted: Sun 08 Apr 2012, 18:53    Post_subject:  

I seem to remember Barry or someone talking about something similar fairly recently.

Personally I think the world could do with a good lightweight frontend to udisks...

_________________
DEATH TO SPREADSHEETS
- - -
Classic Puppy quotes
- - -
Beware the demented serfers!
Back to top
View user's profile Send_private_message 
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Mon 09 Apr 2012, 06:25    Post_subject:  

jemimah wrote:

The bits that handle the icons should be separate modules, so the developer can swap them out easily depending on the what the desktop environment is.

Drive icons could be rendered in its own (pseudo)transparent window. So even that swapping is not required. Same code would work everywhere.(We have the convenience of using X11 when we are using C.)

Yes, there wouldn't be much gain in rewriting the backend, other than a small performance gain. The only main highlight is a design change: moving the drive hot-plugging management to the backend. (Basically it means moving the entire event handling to the backend.) Also, sharing of events with other programs is another plus point.
Back to top
View user's profile Send_private_message 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL

PostPosted: Mon 09 Apr 2012, 09:01    Post_subject:  

That could be one option, but don't force that on the user either.

The desktop provides a lot more functionality than just drawing the icons.

Check out how dhcpcd handles its scripts. The program itself is in C but every time an event occurs it calls every script in the dhcpcd-hooks directory with an argument describing the event.

That way you can easily add or remove scripts, and the scripts themselves decide what to do with various events. If a user wants to install a new desktop environment, the hooks needed to properly respond to drive events can be installed as part of the package.
Back to top
View user's profile Send_private_message Visit_website 
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Mon 09 Apr 2012, 12:33    Post_subject:  

I have already implemented a somewhat equivalent thing in my incomplete volume monitor, for shell scripts.

While the pup-volume-monitor is running, one can run pup-volume-monitor-admin --listen-drv. The command then waits listening to events from pup-volume-monitor and directs them to standard output.

Your suggestion seems to be better, as many programmers (if not, then at least me) don't like writing daemons as they have to be started from somewhere. I'm going to implement it.

Anyways the user will not be forced, my plans are to maintain the frontend as a separate package (and hence a separate process) which itself uses GIO to get its information. In turn, GIO will use backend supplied modules to communicate with the backend. So if user doesn't want the frontend, it simply won't be started and Puppy can still function normally. This will also make it possible to run the frontend without backend, but in that case GIO has to be implemented using GVFS. (GIO doesn't bundle any implementations)
Back to top
View user's profile Send_private_message 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL

PostPosted: Mon 09 Apr 2012, 12:56    Post_subject:  

As of the last time I built it, gvfs needs hal or gnome-disk-util to do anything useful.

If you pull in hal, I don't think there's any point to this exercise at all is there?

Ideally we need a solution with no hal, and no dbus.
Back to top
View user's profile Send_private_message Visit_website 
vovchik


Joined: 23 Oct 2006
Posts: 1285
Location: Ukraine

PostPosted: Mon 09 Apr 2012, 13:13    Post_subject:  

Dear guys and gals,

Totally agree with Jemimah about hal and dbus...let us avoid unnecessary dependencies.

With kind regards,
vovchik
Back to top
View user's profile Send_private_message 
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Mon 09 Apr 2012, 13:21    Post_subject:  

jemimah wrote:

If you pull in hal, I don't think there's any point to this exercise at all is there?

Ideally we need a solution with no hal, and no dbus.

Exactly.

I am ensuring to use minimum of dependencies, which have already been in puppy. Currently the volume monitor uses only GLib, libblkid and libudev. The modules have only GIO as dependency.
Back to top
View user's profile Send_private_message 
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Wed 18 Apr 2012, 11:22    Post_subject:  

So finally it is confirmed that the frontend is to be rewritten.

But status of the backend is still unconfirmed. What is the final verdict? Should backend be written in C?

If yes, the volume monitor will be converted into the pup-event-backend. Then some of us could implement module and firmware loading (see a note about monitor.c in first post) while I finish writing GIO modules and then write the universal pup-event-frontend using GIO. I will also implement jemimah's suggestion in the backend (see sixth post), so users with special needs can easily write their own frontends without hassles of probing drives.

If no, then volume monitor remains volume monitor.
Back to top
View user's profile Send_private_message 
technosaurus


Joined: 18 May 2008
Posts: 4353

PostPosted: Thu 19 Apr 2012, 15:25    Post_subject:  

Writing something in C does not in itself guarantee that it will be faster, though this is generally the case, since typically programmers who write in C are decent programmers by comparison. Compare fixmenus (which uses tools written in C to generate a basic menu from a template) to jwm_menu_create (all standard shell script and generates either a basic menu or with submenus as well as a drive icons tray) _If_ I knew of tools in C that could do the same stuff built into shell, a C version would be even faster.

Sometimes good programming involves knowing the best language for writing the tool (for example if you have a slow shell script that is parsing one really long file and doing a lot of sed, grep and non-integer math, then well-written awk _could_ give a huge performance boost using its builtins in a short period of time with a small amount of code, comparable to a larger compiled C program that would also take much longer to write/debug)

My point here is just that once you know the full problem it is easier to choose the right solution. Unfortunately I really don't understand the "problem" ... is it the daemon process to monitor events? ... while read LINE; do case "$LINE" in ....*);; esac; done </var/log/messages ... pretty easy/fast to have a case statement for any and all kernel events that way (not much more difficult in C btw - probably just needs a series of strstr)

...or is it the generation of devices etc... jwm_menu_create and a couple other of my examples in sit (simple icon tray) uses /sys/block/... (this part _is_ AFAICT much more difficult in C - would need a filetree walk like ftw or nftw and a lot of dirent comparisons for the likes of "sd" to filename[0] filename[1] etc... though I do have a method for doing this part with a switch case and bit shift operation, which are much simpler/faster than a series of if strcmp()... )

Are you just wanting to patch gio to not require gvfs (I looked into this before with regard to abiword falsely assuming gvfs on gtk>2.14 which leaves gtk with a broke uri handler - the code seemed to be pretty deeply nested wrt the uri handling and I never got to the root problem) or just rewrite the backend for the event handler (in which case I could at least improve the script speed and/or help with the C rewrite - just link me to the backend source and I'll take a look)?

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send_private_message 
akash_rawal

Joined: 25 Aug 2010
Posts: 232
Location: ISM Dhanbad, Jharkhand, India

PostPosted: Fri 20 Apr 2012, 13:23    Post_subject:  

The problem with shell is not that it is slow. Shell scripts, when running without using external programs, have acceptable speed. The performance issues start once we start extending it.

If we want a new functionality in a C program, we link a library and work using functions in the library. To do same thing in shell we execute external programs like grep, sed, etc. This involves loading programs to memory which is the actual overhead in a shell program, which usually makes it thousand times slower than equivalent C program. In contrast to this, a function call in C merely involves a few assembly instructions.

Also for each such executable you can always find an equivalent library or library function that does just the same, or better. We aren't limited to standard C library.

The problem discussed here is not just performance issues. The problem is also to 'unlock' drive management features present in many programs which were unused because of absence of backends in GIO. Till now, the ROX desktop was the only way to access the drives. If user uses another DE, (s)he is helpless unless (s)he can hack /sbin/pup_event_frontend_d .

I suggest handling all events from udev in one program, and then after performing usual module and firmware loading, probe for newly connected or disconnected drives and then relay this information to all needed programs using or without using GIO. There are already many programs using GIO for drive management (including GTK itself for its file chooser). Also there are many scripts in puppy who probe the system themselves for drives.

Note that I am not suggesting an alternative to udev. (In fact my volume monitor needs udev to be running to detect drive hot-plugging events.)

As far as GIO is concerned, I am implementing only volume monitoring portion by writing GIO modules, as probably that's all we need. We don't need a full VFS backend like gvfs. (From your feedback about abiword it seems abiword needs a full VFS, but this usually shouldn't happen as usually GIO silently resolves the issues like absence of backends.)

The original source code for pup_event_backend is at /sbin/pup_event_backend_* scripts. Also see /etc/udev/rules.d/50-udev-puppy-basic.rules . I am unable to understand it fully, so I need help in this context.
Back to top
View user's profile Send_private_message 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL

PostPosted: Fri 20 Apr 2012, 15:15    Post_subject:  

The fix for uri handling is here: http://murga-linux.com/puppy/viewtopic.php?t=77295

I checked the latest version of gvfs and it now has a disk monitor that uses udisks instead of HAL or GDU. But udisks depends on PolicyKit (which needs PAM) which seems extremely inappropriate for Puppy.

I'm still not seeing a good way to get rid of pup_event_frontend_d for non-ROX environments.
Back to top
View user's profile Send_private_message Visit_website 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL

PostPosted: Fri 20 Apr 2012, 15:33    Post_subject:  

Apparently, I'm not the only one complaining about this:
http://forum.xfce.org/viewtopic.php?id=6782&ei=CrqRT8LlJa_46QHu5eSPBA&usg=AFQjCNFotAgndFjrPnfzNeDf9NKbaR5kjQ
Back to top
View user's profile Send_private_message Visit_website 
Display_posts:   Sort by:   
Page 1 of 7 Posts_count   Goto page: 1, 2, 3, 4, 5, 6, 7 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Taking the Puppy out for a walk » Suggestions
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0929s ][ Queries: 13 (0.0043s) ][ GZIP on ]