PPKG Puppy Linux Package Handler

A home for all kinds of Puppy related projects
Message
Author
User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

PPKG Puppy Linux Package Handler

#1 Post by Wosh »

Preface
I started this topic originally because of performance problems in the Puppy Linux package management of Woof and the Pet Package Manager.

But it mixed two topics:
1. PPKG Puppy Linux Package Handler
2. PPKG C-Programming Contest

So I have decided to split them. If you are a skilled C programmer you might want to participate at the PPKG C-Programming Contest. There you also can read why and how the project was started.

This discussion will continue with the Puppy Linux Package Handler

PPKG Puppy Linux Package Handler
The package management in Puppy Linux is largely done by shell scipts. This concept causes performance problems for some extensive tasks. For these tasks I have designed ppkg.

At the moment ppkg is able to transform Debian/Ubuntu-Package-Info files to Puppy Linux Package-Database files with an enourmous speed win.
E.g.: The original shell script needs about one hour for the conversion of the Debian Lenny Package Info ppkg less than 400 ms.

Concept
The basic operation is to convert Debian/Ubuntu-Package-Infos like:

Package: abiword
Priority: optional
Section: editors
Installed-Size: 7352
Maintainer: Masayuki Hatta (mhatta) <mhatta@debian.org>
Architecture: i386
Version: 2.6.4-5
Replaces: abiword-gnome
Provides: abiword-gnome
Depends: libaiksaurus-1.2-0c2a (>= 1.2.1+dev-0.12), libaiksaurusgtk-1.2-0c2a (>= 1.2.1+dev-0.12)
Recommends: abiword-plugin-grammar, abiword-plugin-mathview, abiword-help
Suggests: abiword-plugin-goffice
Conflicts: abiword-gnome
Filename: pool/main/a/abiword/abiword_2.6.4-5_i386.deb
Size: 2882324
MD5sum: 7fabfdf5ea014d67541441b930674ff0
SHA1: 792d8d83177ef23cc802b7c249b47b12fa797031
SHA256: c642cd84e17d9e0e88c539f10e812ea187d8a2861acc2566ff1ca21a555ead3d
Description: efficient, featureful word processor with collaboration

to Puppy-Package-Database rows:
abiword_2.6.4|abiword|2.6.4|5|Document|7352K|pool/main/a/abiword|
abiword_2.6.4-5_i386.deb|
+libaiksaurus-1.2-0c2a,+libaiksaurusgtk-1.2-0c2a,+libart-2.0-2|efficient featureful word processor with collaboration|

Additionally these packages must be assigned to the categories for the package management in Puppy Linux. This is done by puppy-category-files (extension ctg), that assign the packages to their category. They look like:

[Category-Name]
package-1
package-2
...
Usage
ppkg is intended to be inserted into the original shell scripts of Puppy Linux. But if you want to do the conversion by hand you have to proceed as follows (Example for Debian Lenny):

Download the Package-Info-file from the distribution repository
http://http.us.debian.org/debian/dists/ ... ckages.bz2

You convert it by calling:
ppkg -c debian -i lenny.ctg -x debug Packages Packages-debian-lenny-main

To get a short description of the options, call:
ppkg -?

To do
ppkg at it's present state can only convert Debian and Ubuntu packages. Using the concepts and algorithms I used for Debian it shouldn't be a big problem to the conversion of other package formats as there are:

1. Slackware
2. Arch

If you want to realize one of them let me know and I will support you.

The appended software includes all files needed to compile the program and do some tests.
Attachments
ppkg.tar.bz2
ppkg all files
(52.66 KiB) Downloaded 1085 times
Last edited by Wosh on Thu 15 Oct 2009, 13:00, edited 2 times in total.
Wosh

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#2 Post by BarryK »

Beautiful!

I've posted a response on my blog:

http://puppylinux.com/blog/?viewDetailed=01167
[url]https://bkhome.org/news/[/url]

Bruce_n_Duane
Posts: 71
Joined: Wed 02 Nov 2005, 01:40
Location: Washington State

Ubunt Repositories

#3 Post by Bruce_n_Duane »

Great Work!!!!

Will this work with the Ubuntu Repositories as the package source?

-Bruce.

User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

Categories in the Puppy Linux Package Management

#4 Post by Wosh »

@BarryK
It should be taken care that the category system of Puppy Linux will not be garbled by the inclusion of foreign packages. At the moment the categorisation within the Pet Package Mangager mirrors the desktop menu categories. This system will be messed by the inclusion of foreign packages because until now there is no solution how to integrate them into the Puppy Menu Category System.

One solution could be to precategorize them how you do it with find_cat and to create a tool that includes them into the Puppy Menu automatically (using a default icon e.g.).

Another option could be to evaluate a distribution package for it's desktop file and to convert the distribution category to the corresponding Puppy category.

If it's for the package manager only, Debian already has a categorisation see "Section:" Enhancing the Pet-Package-Manager with this system would make it easier to distinguish the source of a package. The advantage would be that distribution categories must not be converted. The Manager's category buttons could be replaced by a list.

I think there is some reflection necessary of how the integration of foreign packages can be done user friendly without messing up the whole system.

@ Bruce_n_Duane
I have not tried it but as the Ubuntu Package Description is the same as Debian's I think it will work. The only thing that has to be done is the categorisation. Depending on your expectation you can do the conversion in 3 ways (Ubuntu-Packages: Name of bunzipped Ubuntu Package Info):

1. To categorize all packages "BuildingBlock":
ppkg -c ubuntu -x debug Ubuntu-Packages Packages-ubuntu-jaunty-main

2. As Debian and Ubuntu should be about 90% identical concerning the packages, this call is worth a trial:
ppkg -c ubuntu -i lenny.ctg -x debug Ubuntu-Packages Packages-ubuntu-jaunty-main

3. To do it like Puppy Linux does it now is also no big thing. It only makes necessary a bit of manual work:
If you not have it yet, create the original Puppy Database file with 0-setup-cut (this will take about one hour and I have commented the package-name output. So wait or undo it to have some information about the progress)
0setup-cut ubuntu Ubuntu-Packages Packages-ubuntu-jaunty-main

Then you must cut the package-name together with it's category out of
Packages-ubuntu-jaunty-main. The packages must be sorted by category and to be inserted into a file jaunty.ctg following the example of lenny.ctg. Then you call:

ppkg -c ubuntu -i jaunty.ctg -x debug Ubuntu-Packages Packages-ubuntu-jaunty-main

and you get exactly the same structure as in Puppy Linux. Let me know if you have created jaunty.ctg.
Wosh

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#5 Post by BarryK »

Wosh,
I would like to make a request.

I would like to have a generic 'ubuntu.ctg' (or 'debian.ctg').

ppkg can lookup this file, but if no match found, can you then call my 'find_cat'.

Then ppkg can automatically add the category to ubuntu.ctg.

If your 'ppkg' is located in the 'support' folder in Woof, then it can easily find 'find_cat'.

If 0setup has a call to your ppkg in it, then the first time it runs it can generate ubuntu.ctg totally from scatch if it doesn't exist.

Another advantage is for the future, as someone moves up to a later version of Ubuntu (or Debian), new packages will get automatically added to ubuntu.ctg.
[url]https://bkhome.org/news/[/url]

User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

Category-File

#6 Post by Wosh »

@ BarryK
The category file structure is very simple as it only assigns package-names to their category. If all distribution categories once have been classified their source does not matter. The package-names are unique and hopefully have the same categorisation for each distribution. A universal puppy.ctg would be enough. It even could include Slackware and Arch as long as the categorisation has same structure for all distributions.

Your proposal has two disadvantages:
1. It would be necessary to include "BuildingBlock" into the category file (around 70% of all Lenny packages) to decide if the package is categorized or not, thus increasing the file and reducing the speed of the conversion.
2. find_cat must run for every conversion thus reducing speed. But the distributions are only updated once a while.

Maybe this proposal is more economic, but it needs your approval:
Using cut, your printcols function, sort and grep, the new categories easily can be extracted from the existing package-data-base files. After a run of find_cat they can be inserted into this universal space saving file. Maybe a new function in ppkg can automate as much as possible. I would like to realize it but I can't promiss when I will find time.

Once puppy.ctg is up to date this solution would be a smart:
A bzipped puppy.ctg together with it's MD5sum resides on the Puppy Mirrors. As the packages already must be downloaded a short deviation to a Puppy Mirror could do an automatic update if needed and ensure that the categories are always up to date.
Wosh

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

Re: Category-File

#7 Post by BarryK »

Wosh wrote:Your proposal has two disadvantages:
1. It would be necessary to include "BuildingBlock" into the category file (around 70% of all Lenny packages) to decide if the package is categorized or not, thus increasing the file and reducing the speed of the conversion.
2. find_cat must run for every conversion thus reducing speed. But the distributions are only updated once a while.

Maybe this proposal is more economic, but it needs your approval:
Using cut, your printcols function, sort and grep, the new categories easily can be extracted from the existing package-data-base files. After a run of find_cat they can be inserted into this universal space saving file. Maybe a new function in ppkg can automate as much as possible. I would like to realize it but I can't promiss when I will find time.
I don't understand your point 2.
ppkg would lookup 'puppy.ctg', if it finds a match, fine, if not then run 'find_cat'. So, for an up-to-date 'puppy.ctg', 'find_cat' would never run even once.
[url]https://bkhome.org/news/[/url]

User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

More Reflections about Categories

#8 Post by Wosh »

2. find_cat must run for every conversion thus reducing speed. But the distributions are only updated once a while.
This applies for the case that the info in puppy.ctg is older than the distribution converted, because ppkg has no information about the version of the package-info file. After further reflection I found a solution for this problem by an auto-update function if unknown packages are found. It could work this way:

If ppkg finds unknown packages it calls find_cat, categorizes them and stores them in an internal table. With the content of this table it automatically updates puppy.ctg.

But I am not really a friend of the current category concept for this reason:
The around 25000 packages of a Debian or Ubuntu distribution are like an elefant that stamps into a kennel called Pet Package Manager designed for a few hundred packages. Everything is smashed. The problem can be solved by these approaches. Shrink the elephant to size of a dog (that's what the current appoach does) or extend the kennel to an elephant stable. I am for the extension for these reasons:

1. At the moment find_cat only assigns a reduced number of these packages to categories. If you try to find a certain package in PPM it is probable that you won't find it even if you know it's name.
2. This could be improved by the assignment of more packages to the existing categories. But the consequence would be huge categories that are loaded slowly and are not really a great help for the user if she/he wants to find a certain package.

The current category-system is not designed for such a huge amount of packages. But it easily could be extended using this concept:
Puppy packages use the current category system, Debian packages use their system as it is represented in their package website
http://packages.debian.org/stable/
In the Debian package descriptions these categories are found behind the topic "Section:"

The representation of the categories could be realized rather simple and flexible by the exchange of the category-radio-buttons with a list that represents Debian-sections if the Debian repo is selected and Puppy-categories if the Puppy repo is selected. I do not know whether this concept collides with existing PPM concepts but the advantages are various:

1. No categorisation must be done by the conversion program.
2. The user gets the well known Debian system represented in PL.
3. All packages are categorized and selectable.
4. There is a clear separation in the PPM and in it's database concerning the categories of Debian/Ubuntu and Puppy.
Wosh

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#9 Post by BarryK »

If ppkg finds unknown packages it calls find_cat, categorizes them and stores them in an internal table. With the content of this table it automatically updates puppy.ctg.
Yes, that's what I wanted you to do.

A problem is that online repos change, new packages get added, so any 'puppy.ctg' may be out of date. There has to be this fallback that will automatically add to puppy.ctg, even if it is limited, as find_cat is.

...when will you implement that, so I can test it! :)
[url]https://bkhome.org/news/[/url]

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#10 Post by BarryK »

If ppkg finds unknown packages it calls find_cat, categorizes them and stores them in an internal table. With the content of this table it automatically updates puppy.ctg.
Yes, that's what I wanted you to do.

A problem is that online repos change, new packages get added, so any 'puppy.ctg' may be out of date. There has to be this fallback that will automatically add to puppy.ctg, even if it is limited, as find_cat is.

...when will you implement that, so I can test it! :)
[url]https://bkhome.org/news/[/url]

User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

Implementation of Category Conversion

#11 Post by Wosh »

As described two posts before the autoupdating categorisation requires to include all categories including "BuildingBlock" to the category file thus increasing it from now 86000 bytes to more than 200000 bytes. The implementation of this concept is a bit tricky and will require about two days leisure. At the moment my family, job and house make heavy demands on my time. At the next weekend I am already booked by my family.

But I found a more effective solution for this problem that could be easily implemented after work.
The current concept uses the category file as a the complete presentation of the category system. But now I want to use it as a fine tuning tool for a raw categorisation that is done this way:

1. The Debian sections are assigned to Puppy categories. Some of them will fit rather good to Puppy categories like "Editors", "Network", "Games" others won't.
2. For those that do not fit the category file is used. ppkg will check this file before it does the default categorisation.

The advantages of this concept:
1. It will work with new distributions also, because they hopefully will not contain to many new packages that must be forced into a certain category.
2. The category file becomes very small because it only contains the exceptions and not the rules.
3. The implementation will be very effective and fast because it can use conversion tables if a category is not found in the category file.
4. This concept can be implemented within a few hours.
Wosh

pstef
Posts: 18
Joined: Fri 31 Jul 2009, 19:47

#12 Post by pstef »

I'd like to help, but the coding style of ppkg source code is like huge festering gutpile in a septic pond in hot summer. Way too much comments makes it really hard to understand and impossible to apply different style with GNU indent or any other tool of that kind. I can't stand its control flow either, FIO_FileToMemory() shows perfectly how it shoudln't have been done.

User avatar
Wosh
Posts: 60
Joined: Mon 09 Jul 2007, 19:20
Location: Nuremberg Germany

PPKG Coding Style

#13 Post by Wosh »

@ pstef
If it's for the coding style it shouldn't put you off participating the project. I have some good reasons to this kind of coding style which to explain would fill a book. But I also don't wan't to be forced to a certain style be it the Windows Hungarian notation or the GNU coding style and I as well do not wan't to force others to my kind of coding style. So if you really are interested in the project this won't distract you as you can realize your part of the whole (e.g.: the Slackware conversion) in a own module using your preferred style.

I think the software is hard to read for another reason:
It uses some rarley used software techniques rather from the compiler's than a human's point of view because it's focus is nothing else than speed. But the principles used are quite simple. If I will have time I'll do some explanations that make the basics more understandable.

So excuse me if my horizont is a bit resticted to the speed optimisation due to several decades micro controller programming in C and Assembler.

PS I didn't think that exactly FIO_FileToMemory of all functions would cause problems in understanding because the real tricky functions are pkg_DebianConvert and pkg_DebianConvertInfo. So here are some approaches that might help you.

Note:
FIO_FileToMemory can't be really simple because it hides a lot of demands like opening the file, checking the file size, reading the file, memory allocation and error handling from the caller. But if you produce a more understandable version that is as effective as the current one it will make it into ppkg.

Code: Select all

//Example 1 FIO_FileToMemory in gnu coding style, structured but hard to read.

typedef enum
{
. FILE_RESULT_SUCCESS =        0,        //success
. FILE_RESULT_ERR_OPEN =        1,        //open-error
. FILE_RESULT_ERR_SIZE =         2,        //size-error
. FILE_RESULT_ERR_MEMORY =   3,        //memory-allocation-error
. FILE_RESULT_ERR_READ =        4,        //read-error
} FileioFileResult;                                //file-result enum-type

typedef enum
{
. OPTION_NONE =                   0,          //none
. OPTION_EXIT_ON_ERR =        1 << 0, //exit-on-error
. OPTION_OUTPUT_ERR =        1 << 1, //output-error
. OPTION_EOS_0 =                 1 << 2, //end-of-string-0
} FileioOption;                                   //option enum-type

//===========================================================================
FileioFileResult fileio_file_to_memory( //file-to-memory
. const char* const fileName_p,          //in: file-name pointer
. const FileioOption        option,          //in: option
. char** const                mem_pp,      //in/out: memory pointer-pointer
. size_t* const                memSize_p) //in/out: memory-size pointer
.                                                        //out: file-result
//write a whole file to memory and do some tedious error handling
{
. FileioFileResult fileResult;               //file-result
. int                file_h;                        //file-handle
. ulong                fileSize;                //file-size
.
. fileResult = FILE_RESULT_SUCCESS; //success
.
. if ((file_h = open(                          //open
.   (char*) fileName_p,                     //in: file-name pointer
.   O_RDONLY))                                //in: openKind
.                                                     //out: file handle
.   <= 0)                                         //failure?
.
.   fileResult = FILE_RESULT_ERR_OPEN; //open-error
. else
. {
.   if (fileio_file_size(                //file-size
.        file_h,                             //in: file handle
.        &fileSize)                       //in/out: file-size pointer
.                                             //out: size-valid
.        == GLB_RSL_FAILURE)    //failure?
.
.     fileResult = FILE_RESULT_ERR_SIZE; //file-size-error
.   else
.   {
.     size_t         bytesRead;            //bytes-read-from-file
.     size_t         memSize;              //memory-size
.
.     memSize = fileSize;                //memory-size
.
.     if ((option & OPTION_EOS_0) != 0) //end-of-string-0?
.        memSize++;                        //memory-size
.
.     if (fileSize > MEM_MEM_SIZE_MAX) //or file-size to big?
.     {
.        errno = ENOMEM;
.        fileResult = FILE_RESULT_ERR_SIZE; //file-size-error
.     }
.     else if ((*mem_pp = malloc(     //memory-allocate
.        memSize))                            //in: size
.                                                    //out: memory pointer
.
.        == GLB_NULL)                      //invalid memory pointer?
.
.     {
.        errno = ENOMEM;
.        fileResult = FILE_RESULT_ERR_MEMORY; //memory-allocation-error
.     }
.     else
.     {
.        bytesRead = read(                //read
.          file_h,                                  //in: file handle
.          *mem_pp,                           //in/out: data pointer
.          (unsigned) fileSize);            //in: count
.                                                    //out: number-read
.
.        if (bytesRead <= 0)              //number-read error?
.          fileResult = FILE_RESULT_ERR_READ; //read-error
.
.        else if (bytesRead != (size_t) fileSize) //not read completely?
.        {
.          errno = EILSEQ;
.          fileResult = FILE_RESULT_ERR_READ; //read-error
.        }
.        else
.        {
.          if ((option & OPTION_EOS_0) != 0) //end-of-string-0?
.            (*mem_pp)[memSize - 1] = '\0'; //end-of-string
.
.          if (memSize_p != GLB_NULL)      //valid memory-size pointer?
.            *memSize_p = memSize;        //memory-size pointer
.        }
.     }
.   }
.   close(                                //file-close
.     file_h);                             //in: file handle
.                                            //out: close-result
. }
. if (fileResult != FILE_RESULT_SUCCESS) //no success?
. {
.   if ((option & OPTION_OUTPUT_ERR) != 0) //output-error?
.     error_fatal_system(                //error-fatal-system
.        "Reading to memory",         //in: text pointer
.        fileName_p);                        //in: file-name pointer
.
.   if ((option & OPTION_EXIT_ON_ERR) != 0) //exit-on-error?
.     exit(fileResult);                        //exit with error
. }
. return fileResult;                        //file-result
}

//Example 2: FIO_FileToMemory in gnu coding style
//not correct in regard of structured design but more easy to read

//===========================================================================
FileioFileResult fileio_file_to_memory( //file-to-memory
. const char* const fileName_p,          //in: file-name pointer
. const FileioOption        option,         //in: option
. char** const                mem_pp,     //in/out: memory pointer-pointer
. size_t* const        memSize_p)        //in/out: memory-size pointer
.                                                       //out: file-result
.
//write a whole file to memory and do some tedious error handling
{
. FileioFileResult fileResult;                //file-result
. int                file_h;                         //file-handle
. ulong                fileSize;                 //file-size
. size_t         bytesRead;                  //bytes-read-from-file
. size_t         memSize;                    //memory-size
.
. fileResult = FILE_RESULT_SUCCESS; //success
.
. if ((file_h = open(                        //open
.   (char*) fileName_p,                   //in: file-name pointer
.   O_RDONLY))                              //in: openKind
.                                                   //out: file handle
.   <= 0)                                       //failure?
. {
.   fileResult = FILE_RESULT_ERR_OPEN; //open-error
.   goto CHECK_OPTIONS;
. }
. if (fileio_file_size(                        //file-size
.   file_h,                                       //in: file handle
.   &fileSize)                                 //in/out: file-size pointer
.                                                  //out: size-valid
.   == GLB_RSL_FAILURE)              //failure?
. {
.   fileResult = FILE_RESULT_ERR_SIZE; //file-size-error
.   goto CLOSE;
. }
. memSize = fileSize;                        //memory-size
.
. if ((option & OPTION_EOS_0) != 0)   //end-of-string-0?
.   memSize++;                                //memory-size
.
. if (fileSize > MEM_MEM_SIZE_MAX)  //or file-size to big?
. {
.   errno = ENOMEM;
.   fileResult = FILE_RESULT_ERR_SIZE; //file-size-error
.   goto CLOSE;
. }
. if ((*mem_pp = malloc(               //memory-allocate
.   memSize))                                //in: size
.                                                   //out: memory pointer
.   == GLB_NULL)                          //invalid memory pointer?
. {
.   errno = ENOMEM;
.   fileResult = FILE_RESULT_ERR_MEMORY; //memory-allocation-error
.   goto CLOSE;
. }
. bytesRead = read(                   //read
.   file_h,                                     //in: file handle
.   *mem_pp,                              //in/out: data pointer
.   (unsigned) fileSize);               //in: count
.                                                //out: number-read
.
. if (bytesRead <= 0)                 //number-read error?
. {
.   fileResult = FILE_RESULT_ERR_READ; //read-error
.   goto CLOSE;
. }
. if (bytesRead != (size_t) fileSize) //not read completely?
. {
.   errno = EILSEQ;
.   fileResult = FILE_RESULT_ERR_READ; //read-error
.   goto CLOSE;
. }
. if ((option & OPTION_EOS_0) != 0) //end-of-string-0?
.   (*mem_pp)[memSize - 1] = '\0'; //end-of-string
.
. if (memSize_p != GLB_NULL)        //valid memory-size pointer?
.   *memSize_p = memSize;          //memory-size pointer
.
. CLOSE:
. close(                                   //file-close
.   file_h);                                //in: file handle
.                                             //out: close-result
.
. CHECK_OPTIONS:
. if (fileResult != FILE_RESULT_SUCCESS) //no success?
. {
.   if ((option & OPTION_OUTPUT_ERR) != 0) //output-error?
.     error_fatal_system(                //error-fatal-system
.        "Reading to memory",         //in: text pointer
.        fileName_p);                       //in: file-name pointer
.
.   if ((option & OPTION_EXIT_ON_ERR) != 0) //exit-on-error?
.     exit(fileResult);                        //exit with error
. }
. return fileResult;                        //file-result
}
Cheers
Wosh

pstef
Posts: 18
Joined: Fri 31 Jul 2009, 19:47

#14 Post by pstef »

You have totally misunderstood my moans. It's not the technical aspect of your program that I can't understand, it's the source code itself.
If it's for the coding style it shouldn't put you off participating the project. I have some good reasons to this kind of coding style which to explain would fill a book.
I bet there are people with many good reasons not to wash their teeth, but even a dozen of books wouldn't convince me to follow them.
But I also don't wan't to be forced to a certain style be it the Windows Hungarian notation or the GNU coding style and I as well do not wan't to force others to my kind of coding style.
But that's exactly what you're doing. You've made it impossible to apply different coding style. Without those stupid comments it would be as simple as typing #indent -kr ppkg.c new-ppkg.c
I think the software is hard to read for another reason:
It uses some rarley used software techniques rather from the compiler's than a human's point of view because it's focus is nothing else than speed. But the principles used are quite simple. If I will have time I'll do some explanations that make the basics more understandable.
I didn't get to that, I'm stuck with the coding style.
So excuse me if my horizont is a bit resticted to the speed optimisation due to several decades micro controller programming in C and Assembler.
PS I didn't think that exactly FIO_FileToMemory of all functions would cause problems in understanding because the real tricky functions are pkg_DebianConvert and pkg_DebianConvertInfo. So here are some approaches that might help you.
No, FIO_FileToMemory() doesn't cause any problems in understanding, the coding style does. But I have mentioned FIO_FileToMemory() for another reason - control flow. It would be more obvious with tab indentation; look how deep it is at the end of the original function.

This is how I would rearrange FIO_FileToMemory(). Notice that if someone doesn't like its style, it can be easily reformatted with GNU indent. BTW, GNU indent is a program and can be found there: http://www.gnu.org/software/indent/

Code: Select all

//write a whole file to memory and do some tedious error handling
FIO_FILE_RSL_ET FIO_FileToMemory(
  const uchar* const fileName_p,
  const FIO_OPTION_ET	option,
  uchar** const		mem_pp,
  MEM_SIZE_T* const	memSize_p)
{
  int file_h;
  ulong fileSize;
  FIO_SIZE_T bytesRead;
  MEM_SIZE_T memSize;

  if ((file_h = open((char*)fileName_p, O_RDONLY)) <= 0)
    return FIO_FLR_ERR_OPEN;

  if (FIO_FileSize(file_h, &fileSize) == GLB_RSL_FAILURE)
    return FIO_FLR_ERR_SIZE;

  memSize = fileSize;

  if (option & FIO_OPT_EOS_0)
    memSize++;

  if (fileSize > MEM_MEM_SIZE_MAX)
  {
    errno = ENOMEM;
    return FIO_FLR_ERR_SIZE;
  }

  if ((*mem_pp = malloc(memSize)) == GLB_NULL)
  {
    errno = ENOMEM;
    return FIO_FLR_ERR_MEMORY;
  }

  if ((bytesRead = read(file_h, *mem_pp, (unsigned) fileSize)) <= 0)
    return FIO_FLR_ERR_READ;

  if (bytesRead != (FIO_SIZE_T) fileSize)
  {
    errno = EILSEQ;
    return FIO_FLR_ERR_READ;
  }

  if (option & FIO_OPT_EOS_0)
    (*mem_pp)[memSize - 1] = '\0';

  if (memSize_p != GLB_NULL)
    *memSize_p = memSize;

  close(file_h);

  return FIO_FLR_SUCCESS;
}

User avatar
Hugh
Posts: 138
Joined: Sat 24 Jun 2006, 21:53
Location: Imperial Warmongering Dystopia of Amerika

#15 Post by Hugh »

But that's exactly what you're doing. You've made it impossible to apply different coding style. Without those stupid comments it would be as simple as typing #indent -kr ppkg.c new-ppkg.c

We would like to believe that for programmers and coding nothing is impossible. Finding the 'solution' is not always easy but with sufficient thought it will come forth without fail.

Comments, of course, are always optional, however, they can be very helpful to get the gist of the routines quickly and easily; thus leading to the 'big picture' with ease.

pstef
Posts: 18
Joined: Fri 31 Jul 2009, 19:47

#16 Post by pstef »

Hugh wrote:
But that's exactly what you're doing. You've made it impossible to apply different coding style. Without those stupid comments it would be as simple as typing #indent -kr ppkg.c new-ppkg.c
We would like to believe that for programmers and coding nothing is impossible. Finding the 'solution' is not always easy but with sufficient thought it will come forth without fail.

Comments, of course, are always optional, however, they can be very helpful to get the gist of the routines quickly and easily; thus leading to the 'big picture' with ease.
http://lxr.linux.no/#linux+v2.6.31/Docu ... Style#L425

User avatar
Hugh
Posts: 138
Joined: Sat 24 Jun 2006, 21:53
Location: Imperial Warmongering Dystopia of Amerika

Coding Styles

#17 Post by Hugh »

Linux kernel coding style

This is a short document describing the preferred coding
style for the linux kernel.

Coding style is very personal, and I won't _force_ my views
on anybody, but this is what goes for anything that I have to
be able to maintain, and I'd prefer it for most other things too.

Please at least consider the points made here.
- - - - - - - - - - - - - - - - - - - - - -


Of course, there are several coding conventions and
for the sake of 'uniformity' certain styles are recom-
mended for certain projects.

But I haven't seen any examples of the 'alleged'
over-commenting.

With a little ingenuity, in the event the comments
prove an irritant, it is possible to eliminate them
with the editor 'search and replace' function.

With a little more ingenuity, it is even possible to
develop a program to re-style the source code.

For programmers, nothing is impossible.

pstef
Posts: 18
Joined: Fri 31 Jul 2009, 19:47

#18 Post by pstef »

I don't want to keep talking off the topic, just let me say that I disagree with you.

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

??

#19 Post by scsijon »

did this get any further or was it abandoned?

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#20 Post by BarryK »

There is new action in this area:

http://bkhome.org/blog2/?viewDetailed=00028
[url]https://bkhome.org/news/[/url]

Post Reply