Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 23 Nov 2014, 12:25
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Package categories help wanted
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [29 Posts]   Goto page: 1, 2 Next
Author Message
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7082
Location: Perth, Western Australia

PostPosted: Tue 13 Nov 2012, 11:05    Post subject:  Package categories help wanted  

My blog post says it all:

http://bkhome.org/blog2/?viewDetailed=00029

If anyone is interested, post here so that we don't have two people doing the same thing -- well, I am assuming that more than one person will be interested in doing this! Or even one for that matter.

I was thinking that this is something that someone can do who wants to contribute to Puppy development, but is not into shell/bacon/c or whatever programming, or anything system-level.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Tue 13 Nov 2012, 11:44    Post subject:  

all of this _could_ be accomplished by downloading the package, extracting the .desktop file and parsing it ... if it doesn't have a desktop file it would be pretty safe to assume building block. I wrote a similar script for pet packages that scottman used in akita last year.

it was along the lines of:
Code:
for listing in `tar -tf package |grep desktop`; do tar -xf package listing; parse_func listing; done


I don't have that kind of bandwidth though

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 2577
Location: www.eussenheim.de/

PostPosted: Tue 13 Nov 2012, 12:17    Post subject:  

technosaurus wrote:
I don't have that kind of bandwidth though

I have some bandwidth, just give me parse_func and a little help Cool

------
edit 20:40
generating list of packages that have .desktop

0ad is 1st in HASdesktopFILE and is 1st in PKGS_HOMEPAGES
.
.
bouncy is 175th in HASdesktopFILE and is 843rd in PKGS_HOMEPAGES
.
and so on (wait some hours)
packages which have .desktop file: less than 20%

--
edited (110 minutes later)

Quote:
--2012-11-13 21:17:54-- http://packages.ubuntu.com/precise/all/zziplib/filelist
Auflösen des Hostnamen »packages.ubuntu.com (packages.ubuntu.com)«.... 91.189.94.203
Verbindungsaufbau zu packages.ubuntu.com (packages.ubuntu.com)|91.189.94.203|:80... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: nicht spezifiziert [text/html]
Saving to: `filelist'

[ <=> ] 2.697 --.-K/s in 0s

2012-11-13 21:17:54 (7,63 MB/s) - `filelist' saved [2697]



2080 packages have desktop files
HASdesktopFILE.gz
Description  list of packages with desktop files
remove .gz
gz

 Download 
Filename  HASdesktopFILE.gz 
Filesize  20.82 KB 
Downloaded  500 Time(s) 

Last edited by L18L on Tue 13 Nov 2012, 17:23; edited 9 times in total
Back to top
View user's profile Send private message 
aragon

Joined: 15 Oct 2007
Posts: 1698
Location: Germany

PostPosted: Tue 13 Nov 2012, 14:39    Post subject:  

This shouldn't sound harsh and maybe i don't understand the problem but to to me building such a db sounds like a graet waste of lifetime.

If a puppy is based on other distros packages it needs to reflect the distros menu-categories or it needs a script that converts the offending categories to puppy-categories on installation.

Aragon

_________________
PUPPY SEARCH: http://wellminded.com/puppy/pupsearch.html
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Tue 13 Nov 2012, 16:25    Post subject:  

I almost forgot that debian/ubuntu keep a list of files in each package, which could help avoid unnecessary downloads.

@L18L - most of the code for parsing the desktop files is already written as part of jwm_menu_create, just have to remove the jwm xml bits and unneeded lines, but I don't have a good feel of what barry is checking against regarding package names from distro to distro. In actuality it would be easy enough to create a whole database template that only requires filling in the full package name and size for each distro and appending any that are missing from the template

@aragon, once the database is created, it should be applicable to most distros

@barry, I'm not sure how much the different distros vary in their package names and how you account for the different naming such as abiword-x.x.txz vs abiword-bin-x.x.deb, etc... if they all somehow listed the executable, that would be the easiest since I could just use the Exec= and Categories= lines from the desktop files, ... otherwise it would require some manipulation of the package name (I am assuming this is coded somewhere already)

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 2577
Location: www.eussenheim.de/

PostPosted: Tue 13 Nov 2012, 17:02    Post subject:  

technosaurus wrote:
... debian/ubuntu keep a list of files in each package, which could help avoid unnecessary downloads.

@L18L - most of the code for parsing the desktop files is already written as part of jwm_menu_create, just have to remove the jwm xml bits and unneeded lines, but I don't have a good feel of what barry is checking against regarding package names from distro to distro. In actuality it would be easy enough to create a whole database template that only requires filling in the full package name and size for each distro and appending any that are missing from the template


I have grepped that filelist, results in 2080 packages (only Very Happy ), see above.

Now I do not know how to "look" inside a .deb (extract the .desktop file) without installing
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7082
Location: Perth, Western Australia

PostPosted: Tue 13 Nov 2012, 20:17    Post subject:  

Quote:
all of this _could_ be accomplished by downloading the package, extracting the .desktop file and parsing it ... if it doesn't have a desktop file it would be pretty safe to assume building block. I wrote a similar script for pet packages that scottman used in akita last year.


technosaurus,
This already happens on a per-package basis in /usr/local/petget/installpkg.sh when a package is installed. The script tries to make sure the .desktop file "Categories" field is appropriate for the Puppy menu structure.
Doesn't always get it right though, which is why a 'categories.dat' would be nice to check against -- perhaps that file could be initially created with a script, then each entry manually checked for correctness -- but then it might be better to do it all manually, which was my original thinking, and why I suggested a non-developer would like this.

Quote:
2080 packages have desktop files


L18L,
That's great, then probably a huge number of these are very exotic and could be ignored -- not put into categories.dat, meaning that the automatic category determination performed in debdb2pupdb will be deemed good enough.

Quote:
If a puppy is based on other distros packages it needs to reflect the distros menu-categories or it needs a script that converts the offending categories to puppy-categories on installation.


aragon,
Puppy's menu structure is unique. .desktop files are modified at installation, as mentioned above. However, the automatic mechanism doesn't always get it right.
Also, the PPM needs to know the categories of all packages prior to installation, as they are displayed in categories, and icon thumbnails are assigned on a per-sub-category basis.
The categories that the PPM displays must fit the Puppy menu structure.
The categories are in every package database entry -- which is done by the -to-pup-db converters such as 'debdb2pupdb'.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7082
Location: Perth, Western Australia

PostPosted: Tue 13 Nov 2012, 20:27    Post subject:  

technosaurus wrote:
@aragon, once the database is created, it should be applicable to most distros


Yes. Well, applicable to all distros supported by Woof.

technosaurus wrote:
@barry, I'm not sure how much the different distros vary in their package names and how you account for the different naming such as abiword-x.x.txz vs abiword-bin-x.x.deb, etc... if they all somehow listed the executable, that would be the easiest since I could just use the Exec= and Categories= lines from the desktop files, ... otherwise it would require some manipulation of the package name (I am assuming this is coded somewhere already)


PKGS_HOMEPAGES has generic names for packages. Ubuntu and Debian do use some odd names, and also split original packages up into many smaller ones -- however I have got around this problem as the Ubuntu/Debian database does also contain generic names.

Ubuntu/Debian keep packages in their online repos in folders named by the generic name, that's how I am able to obtain the generic name and check against lists such as categories.dat and PKGS_HOMEPAGES.

Though, the folder names in the Ubuntu/Debian online repos do not always correspond to the original package name, so it is not a perfect method.

Most other distros keep the original generic naming of packages, Slackware for example, so no problem.

Mageia is one example that splits up packages a lot and renames them. I don't yet know if the generic names can be determined for Mageia, probably some way to do it.

_________________
http://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Tue 13 Nov 2012, 21:29    Post subject:  

L18L wrote:
Now I do not know how to "look" inside a .deb (extract the .desktop file) without installing
undeb <pkg>,... but I forgot to send my undeb patch in to busybox for xz and I seem to have lost it, so you will need the full (and recent) package since they seem to be using xz in at least some if not all deb packages now.
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
seaside

Joined: 11 Apr 2007
Posts: 887

PostPosted: Wed 14 Nov 2012, 00:40    Post subject:  

technosaurus wrote:
L18L wrote:
Now I do not know how to "look" inside a .deb (extract the .desktop file) without installing
undeb <pkg>,... but I forgot to send my undeb patch in to busybox for xz and I seem to have lost it, so you will need the full (and recent) package since they seem to be using xz in at least some if not all deb packages now.


Could just the desktop files be extracted by something like this
Code:
tar -xfJ PACKAGE.tar --wildcards --no-anchored '*.desktop'


Cheers,
s
Back to top
View user's profile Send private message 
aragon

Joined: 15 Oct 2007
Posts: 1698
Location: Germany

PostPosted: Wed 14 Nov 2012, 02:33    Post subject:  

Ok, sorry ... Now i see the points where the db is needed.

One more point about the menu-structure: it would be nice if the menu-points help and submenu shutdown would be implemented as desktop-files and desktop-catetegory. This would ease internationalization and the need to have static parts in templates would be superflous.

Aragon

_________________
PUPPY SEARCH: http://wellminded.com/puppy/pupsearch.html
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Wed 14 Nov 2012, 02:59    Post subject:  

aragon wrote:
Ok, sorry ... Now i see the points where the db is needed.

One more point about the menu-structure: it would be nice if the menu-points help and submenu shutdown would be implemented as desktop-files and desktop-catetegory. This would ease internationalization and the need to have static parts in templates would be superflous.

Aragon
It may be too big for puppy but it isn't _that_ much to copy all of the desktop files to a separate directory like /usr/share/applications.uninstalled
and replace Exec=app with Exec=installer package_name (where "installer" is a wrapper around petget and removes the .uninstalled/*.desktop file after install) such that the whole thing can be parsed into an install menu ... we could also go through them and check the Icon= field to see if the icon exists and if not, make a symlink for it based on the Categories= field (jwm_tools already has a similar installer menu to this)

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 2577
Location: www.eussenheim.de/

PostPosted: Wed 14 Nov 2012, 06:19    Post subject:
Subject description: desktop files to a separate directory
 

technosaurus wrote:
...it isn't _that_ much to copy all of the desktop files to a separate directory...

Yes, and thank you for undeb that does work

BTW, there are more packages in that list that do NOT have .desktop
example:
389-admin
389-admin-console
389-adminutil
389-console
389-ds-base
389-ds-console
389-dsgw
accountsservice
aewm++
afterstep


wait some minutes...hours
script is running....

edit
the list NOTfound continues:

amarok
amiwm
antiword
apparmor
apper
arbtt
ark
at-spi2-core
audex
awesome
bangarang
bash-completion
basket
biomaj-watcher
blackbox
blinken
bluedevil

anyhow, seems to be all exotic

--edit
here is a very bad (buntu Laughing ) example
gamazons.desktop wrote:
[Desktop Entry]
Encoding=UTF-8
Name=Gamazons
Name[ca]=Gamazons
Name[cs]=Gamazons
Name[da]=Gamazons
Name[de]=Gamazons
Name[el]=Gamazons
Name[es]=Gamazons
Name[et]=Gamazons
Name[fi]=Gamazons
Name[ga]=Gamazons
Name[gl]=Gamazons
Name[ja]=
Name[ko]=
Name[lt]=Gamazons
Name[ms]=Gamazons
Name[nl]=Gamazons
Name[no]=Gamazons
Name[pl]=Gamazons
Name[pt]=Gamazons
Name[pt_BR]=Gamazons
Name[ru]=
Name[sk]=Gamazons
Name[sl]=Gamazons
Name[sv]=
Name[vi]=Gamazons
Name[wa]=Gamazons
Name[zh_TW]=Gamazons
Comment=Gamazons game
Exec=gamazons
Icon=gnome-gamazons.png
Terminal=false
Type=Application
Categories=GNOME;Application;Game;BoardGame;
X-Desktop-File-Install-Version=0.3
OnlyShowIn=GNOME;

I will have to remove all translations in all .desktop files

----
...done

99 minutes
remove of translations (2 minutes) saved 785408 Bytes
directory containing 1902 desktop files is size 1983k
in attached archive 174k
applications.tar.gz
Description 
gz

 Download 
Filename  applications.tar.gz 
Filesize  173.82 KB 
Downloaded  531 Time(s) 
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 2577
Location: www.eussenheim.de/

PostPosted: Wed 14 Nov 2012, 15:25    Post subject: Package categories help wanted
Subject description: 1765 decisions
 

filtered packages that
- are already in categories.dat
- do not have an entry CATEGORIES= in .desktop (they exist!)
and sorted by category

remaining 1765 packages

The first one:
Quote:
Accessibility;Utility;Settings;X-GNOME-Settings-Panel;X-GNOME-PersonalSettings; myunity


In /etc/xdg/menu/hierarchy we find Accessibility
Quote:
Desktop :settings X-DesktopCountry,Accessibility

Thus this package myunity belongs to
Category Desktop
Subcategory Settings

Now myunity has to be inserted into the line containing Desktop settings in file categories.dat:
Quote:
CONST PKG_CAT_Desktop_settings$=" jwmconfig2 myunity pupx xclock xkbconfigmanager "


Any volunteers?
Or split the work?
-multimedia
-games
-...

--
edit
... and the last one is:
Quote:
X-XFCE;Utility; xfprint-manager
Very Happy
Categories_package.sorted.gz
Description  remove extension .gz
gz

 Download 
Filename  Categories_package.sorted.gz 
Filesize  55.88 KB 
Downloaded  501 Time(s) 
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Wed 14 Nov 2012, 17:47    Post subject:  

@L18L does it look like we need to do something like:
echo Package=$PACKAGENAME >> package/.../*.desktop
so we have a basis for determining the package name, or are most fairly straight forward

I am stuck in windows mode until I finish my programming languages project, so I can only give guidance right now, but if you open up jwm_menu_create (from jwm_tools) all of the language is there to generate all subcategories, all that needs to be done is:
set the subcategories variable to true (i forget the exact var name)
change the directory path from /usr/share/applications to the appropriate dir. remove the unneeded jwm markup
change the output to just give the format that Barry needs, which is ???
(p.s. my sub-category names may slightly differ but that is easily fixed by humans pretty quickly)
I assume that output could be:
category|subcategory|packagea packageb

(note to self: may need tolower so we can use strstr instead of needing strcasestr which is slower)

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [29 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1153s ][ Queries: 13 (0.0070s) ][ GZIP on ]