Compiler GUI and other packaging tools

Stuff that has yet to be sorted into a category.
Message
Author
User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#76 Post by Gedrean »

Alright what's the format for LDFLAGS?

Your picture shows -Wl= but here's what I have so far in the original code:
LDFLAGS=" -Wl,-O4,-Os,-relax,--sort-common,--no-keep-memory,-s "

Anyhow I see your layout there... it's gonna be fun figuring this out.

I already had to fix a few "things" in my own -- found some more glitches!

Huzzah.

EDIT:
Also can you repost your window file as your png screenshot you threw up there has things the copy I grabbed apparently does not.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

TRAIN OF THOUGHT ON DYNAMIC VARIABLES

#77 Post by Gedrean »

My previous post question remains to be answered, as well as the request for reposting the new version you made of the window file ...

This post is a train-of-thought on how to format options, in cboxes and rbuttons and other formats, to allow dynamic adding of compiler options and other functions to the ./configure or other commands used in the pcompile script called pcomp_script - located in /usr/local/pcompile/ - to allow extension and expansion of functionality while reducing changes in core code.

Previously a fixed "indicator" string of "CB_" was affixed before certain items with a true/false value, namely radio buttons and checkboxes. Inside the full names of these items, a % was substituted for an =.
If the value was true, the CB_ was stripped from the name, any % converted to =, and the resulting name appended to a ./configure argument... meaning any argument could be added in by adding a new checkbox to the window code.

With the advent of changing compiler (CXXFLAGS, LDFLAGS, etc) flags, it is apparent that this system is no longer sufficient for our needs. These other systems follow a whole new format, and require added code to interpret and build, so the existing system is not enough.

Expanding upon that system would just make the interpretation code for the CB_ much more convoluted.

Since on a relatively recent (1.6 GHz single-core 32-bit 2 GB RAM) machine the runtime for the interpretation code is completely negligible, adding additional convolution to those codes would not add CPU time of a noticeable amount.

While that is definitely not representative of the CPU power or RAM of the common Puppy system, even a system running around 700 MHz with 512 MB of RAM should not experience much more than a couple extra seconds of script time with additional code.

In addition, memory usage is minimal.

Really, since our target audience is compiling packages for PET'ing, and probably aware that compiles take forever anyhow, 2-3 seconds won't matter.

However, good coding practice dictates we DO NOT over-convolute the code.
While trying to restrict ourselves to bash-internals is good, making the code as compact as possible so as to make it unreadable, and thus unchangeable later for improvements, is completely bad.

Just changing our format for a new option is also bad, because we have more and more risk of a "false positive" -- that is, something where an entry or a name incorrectly triggers a result, thus breaking functionality.

We require then a format that is sufficiently obscure that false positives are as unlikely as possible, but not so obscure that it is a pain to type out for every entry.

We also want a format with a character that, in a bash command string, cannot be included, because we then don't have our "custom options" field interfering with this either.

To mind, ampersand (&) works as it is the "Job in Background" character.
Pipe (|) works as well, as it is the pipe function in bash.
Percent (%) has worked so far that we know of, but not sure if % is a character that bash would break the command with.

& and | seem to be our likely results for what we want then.

A good overall format should contain multiple "sections", with each having a defined purpose, such that we can use these sections to determine a "dynamic variable" and determine what purpose it serves and how it should be passed.

First, we need a section to indicate it is a dynamic variable. A two-character string, for example "DV" for "Dynamic Variable", could work sufficiently for this.
Previously we had been using "CB" for "CheckBox" but that has become rather broken.
An alternate could be "PC" for "PCompile".

Second, a separator. I think the | character would work good for the separators, and is easy to pull out with our expansion functions.

Third, we need a section to indicate what the function of this particular DV is. Another two- or three-character string would be sufficient for this purpose, much more and the naming convention is too complicated.
This two or three character string should probably be an acronym or abbreviation of its function.
For example, Configure Option could be "CO", and LDFLAG could be "LDF".
These are just examples, and not recommendations.

Fourth, another separator. This separates these "indicators" from the actual name.

Finally, the name. Within the name, any = sign should be replaced by an &. The reason for this is that when passed back by gtkdialog, these options will be followed with ="true" or ="false", thus any = in the name would be harder to parse out. In comparison, once the value is determined to be "true", the & can be converted to an = and the whole name passed as an argument in whatever format is needed.

Since | and & would not be used in arguments to ./configure or any other bash commands, these two characters should suffice for these needs.

------------------------------------------------------------------

RECOMMENDATIONS FOR NAMING OF DYNAMIC OPTIONS:
Use | for separator and & for substitution character.
Use 3 total sections, separated by | separator.
Section 1 - Two-Char Indicator of Dynamic Variable.
Section 2 - 2/3-Char Indicator of DV Function.
Section 3 - Variable/Argument/Option Name.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#78 Post by technosaurus »

Very good point. I was forgetting the target audience... better to have some predefined selections that are known to work well and a user defined box for intermediate to advanced users... I have a lot of these predefined already - I can post when I get home tonight. The checkboxes were not well thought out - I don't know what I was thinking with that, but I came to the same conclusion when I started writing the bash for them.

also some programs keep their own FLAGS in the Makefile that override what you specify. Adding this line after ./configure will fix that:

Code: Select all

find . -iname \*akefile -execdir sed -i "s/-O[0-3]/ $PCOMPILE_CFLAGS /g" {} ';' -print
find . -iname \*.mk -execdir sed -i "s/-O[0-3]/ $PCOMPILE_CFLAGS /g" {} ';' -print
To remove debugging support (NOT default because it will break some programs for whatever reason):

Code: Select all

find . -iname \*akefile -execdir sed -i "s/ -g / /g" {} ';' -print
find . -iname \*.mk -execdir sed -i "s/ -g / /g" {} ';' -print
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#79 Post by Gedrean »

The checkboxes were not well thought out - I don't know what I was thinking with that, but I came to the same conclusion when I started writing the bash for them.
No it's honestly a great idea, it just requires making this a little more standardized and less... simplistic.

To add a new category, we'd just need to add another loop with the right identifiers, and the right compilations, and use that in some command as a var instead of a fixed string.

The flags are still a good idea, we just want to properly format it out.

Also, we want to make sure that the top line in the window for that particular tab indicates these are advanced options, and should be left default if the user is unsure what they do.
Also add in rbuttons for mtune as well as march :)
also some programs keep their own FLAGS in the Makefile that override what you specify. Adding this line after ./configure will fix that
A specific (non-dynamic) checkbox in the app to check to request "override makefile flags" would be good for that concept.
To remove debugging support
Also, the removal of debugging support isn't REALLY necessary. A lot of changes I did so that program-breaking wouldn't occur, like the moving to a sub-dir instead of the main dir (when I was compiling some lib it was like "Not in the main dir, *****!")

Tell ya what, lay out the checkboxes the way you want, in the name just call them CB_<whatevername> for each option, make sure the frame indicates where they should go... and post here the format used for each section/flags/whatever --
(( meaning the format used in bash command line to make those flags active, so like cxxflags="blahblah" -- make sure it's clear how it's supposed to go with multiple options )).

I'll write up the code tonight or tomorrow. It's not at ALL hard to write up anymore. I've gotten the coding bug back and it's just a matter of figuring out how to work it.

BTW, are there any other characters we could do besides pipe and ampersand? Not saying I don't like those but I think & is only bad if it's at the end of a line - I'm not sure if filenames and stuff can have an & in it, so is there any other "command-breaking" characters?

If there aren't and those work fine I'll just use those.
It's no big deal, I just wanna make sure what I can and cannot do with this.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#80 Post by technosaurus »

You seem to be working wonders with my simple code hacks and turning into an actual piece of software. Here is an updated window file, feel free to name the variables whatever you like.

everything in CFLAGS goes into

CFLAGS=" $VAR1 $VAR2....-O[s,2,3] -march=i[3,4,5,6]86 -mtune=i686" \
CXXFLAGS=$CFLAGS \
####where VAR1... takes the format

Code: Select all

 -fsee 
<--note spaces before and after
LDFLAGS= -Wl,-O4$VARX$VARY$VARZ \
####where VARZ... takes the format

Code: Select all

,--as-needed
as far as % and & ... \ works for some things (\n for new line) depends on what you use to parse it. To be honest I can't really write a line of code myself but I have a good memory for where to find code that does what I want it to do so I can cut, paste and hack until it works.
Attachments
window.gz
(1.41 KiB) Downloaded 374 times
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#81 Post by Gedrean »

To clarify then, CXXFLAGS and CFLAGS should be the same value.

CFLAGS should be separated by spaces, with a leading space at beginning of string and a trailing space at end of string.

LDFLAGS should be separated by commas, with a leading space at the beginning, and a trailing space at the end.

Am I correct on all that?

EDIT:

Oh and did a little research, | needs to be escape-char'd in command line most of the time, or double-quoted, and the way we're wording things the odds that those chars will interfere are VERY low, so I think the | and & should work. \ will be a pain in the ass because I'd have to do a lot of \-escaping the \'s, so... I would want to kill myself after a while. :)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#82 Post by technosaurus »

Gedrean wrote:... I would want to kill myself after a while. :)
I hear you - I hate picket fences. That's what initially scared me away from learning bash.

correct
CFLAGS and CXXFLAGS can use the same parameters but need to be specified seaparately


LDFLAGS is passed with -Wl$v1$v2$v3$v4$v5$v6$v7$v8$v9$v10 \
each variable should contain a preceding comma followed by the actual flag and nothing after

cflags template

if [ $CB_CFPIPE = "true" ];then
CB_CFPIPE =" -pipe " #description of pipe...
else
CB_CFPIPE ="" #setting it back to "null" allows us to pass all parameters without doing more "if thens"
fi

ldflags template

CB_LDO
if [ $CB_LDO = "true" ];then
CB_LDO=",-Os" #description of pipe...
else
CB_LDO="" #set back to "null"
fi


CFLAGS="$CFLAGS$CB_CFPIPE$CB_CFCOMBINE....$CB_CFMARCH -mtune=i686 " \
CXXFLAGS=$CFLAGS \
LDFLAGS=" -Wl,-04$CB_LDO$CBLDRELAX....$CB_LDSTRIP " \
./configure .....
(putting the proper separators in the string prevents)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#83 Post by Gedrean »

technosaurus wrote:
Gedrean wrote:... I would want to kill myself after a while. :)
I hear you - I hate picket fences. That's what initially scared me away from learning bash.

LDFLAGS is passed with -Wl$v1$v2$v3$v4$v5$v6$v7$v8$v9$v10 \
each variable should contain a preceding comma followed by the actual flag and nothing after

ldflags template

CB_LDO
if [ $CB_LDO = "true" ];then
CB_LDO=",-Os" #description of pipe...
else
CB_LDO="" #set back to "null"
fi


CFLAGS="$CFLAGS$CB_CFPIPE$CB_CFCOMBINE....$CB_CFMARCH -mtune=i686 " \
CXXFLAGS=$CFLAGS \
LDFLAGS=" -Wl,-04$CB_LDO$CBLDRELAX....$CB_LDSTRIP " \
./configure .....
(putting the proper separators in the string prevents)
Okay. So the LDFLAGS should be -quote--space-"-Wl," then -O whatever then everything else, all separated by commas, space preceding -Wl and following last argument. Got it.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#84 Post by Gedrean »

Alright. I'm gonna reformat things some in the window stuff, and build it like that in pcomp_script - hopefully it's self-explanatory what I do, but basically I'm not going to have the script "know" what the flag values or words are SUPPOSED to be, but instead just derive them from the varnames passed back by gtkdialog.

This way if later we add other flags at are off or on, there requires no code changes, just window changes.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#85 Post by Gedrean »

Lacking other input, I'm going with DV for Dynamic Variable (I'm cheap and lazy) and 2-character Function Code, again acronyms.

DV|FC|StringToBeUsedInTyping="true" etc...
If false, don't strip the string.
If true, strip the string and put it in.

For example, in the instance of -fomit-frame-pointer...

DV|CF|-fomit-frame-pointer="true" -- hopefully that makes sense.
The building script should insert the separating spaces, commas, spaces in right locations, etc.
It should be smart about those parts.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#86 Post by technosaurus »

It took me a bit to figure out what you were saying but now that I got it, it sounds good. What you are talking about could be pretty useful for some other programs.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#87 Post by Gedrean »

I figured I'd post a code snippet, in case other programmers read and are interested (HAH! That'll be the day!)

Code: Select all

# This is a template of how we're doing it now.
# This template covers EVERY loop for Dynamic Variables.
# This template also details variable naming, for coding style purposes.
# For this template, <FC> means function code, 2 digit code from var names.

# First, we declare our holder for all codes of this DV type.
# Call it: "DV_<FC>_CODE"
# Strip out the leading = and perform a grep of "DV\|<FC>\|" from CHOOSER2
# and put the result in this holder.
# Declare another variable, DV_<FC>_REMAINING, equal to this first holder.
# Declare a third variable, DV_<FC>_RESULT, for storing the end result in.
# This RESULT MUST start out null ("").  NO starter value!
# In a while loop, condition: [ "$DV_<FC>_REMAINING" != "" ]
#	Set a name equal to the first "name" in REMAINING:
#	NAME=${DV_<FC>_REMAINING%%=*}
#	grep the NAME from the DV_<FC>_CODE and put result to ENTRY and VALUE.
#	Perform strip command: pcomp_strip VALUE $NAME -- VALUE not $-expanded!
#	if condition: [ $VALUE == "true" ]
#		( We could also use other conditions if value isn't going to be ...
#		a true/false, but that's more complicated )
#		Strip 'DV|<FC>|' from NAME: NAME=${NAME#DV\|<FC>\|}
#		Replace & with = in NAME: NAME=${NAME/\&/=}
#		Format and add NAME to the end of DV_<FC>_RESULT
#	fi
#	Remove ENTRY from REMAINING: DV_<FC>_REMAINING=${DV_<FC>_REMAINING#$ENTRY}
#	Remove leading space: DV_<FC>_REMAINING=${DV_<FC>_REMAINING/?DV\|/DV\|}
# done
# if condition: [ $DV_<FC>_RESULT != "" ]
#	Perform any additional formatting to result (add starter string) ...
#	and add to final exec string
# fi
As you'll note, this is a code template, not actual code - but it should give a proper idea of how to parse out dynamic values like this from gtkdialog in ANY instance, with a little interpolation...

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#88 Post by Gedrean »

On a side note, you'll notice in the code snippet that I have the REMAINING variable simply for getting the "NAME" of the first entry from, and for stripping full entries from for the purposes of a really cheap counter
(I could do a for loop, now that I think of it, but I don't recall what happens with a for loop when passed an empty string-list, and I didn't want to code around with it, though I could definitely do that if it works... Hold on...)

As you may have noticed, during my massive rambling posts which don't make much sense to anyone, I tend to do posts on a train of thought, and rarely go back and delete stuff that is contradicted by later comments, trusting it will be taken like a conversation.

You'll now understand then, why I want to go cry in a corner. I spent nearly 4 hours trying to figure out why I couldn't just strip the value from the CODE variable.

I could have done it with a for loop.

It would have been easier.

None of that "stripping stuff out".

Anyhow.

Yeah. I'll post a later update with for loops.
God I feel like an idiot.

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#89 Post by Aitch »

Hi Gedrean
I'm not a programmer, but I share house with one named Pat
He uses xbasic, so I can follow the logic, if not the vision....
I always admire people like You, Techno, Aragon, Patriot, disciple, amigo, tronkel, pizzasgood, 01micko, ttuuxxx, etc, etc.....
[to name a few of the top of my head - please don't write me if you weren't included,.... nothing personal... :D ]
...and enjoy watching communication of this tricky concept between 2 [and more] people, trying to resolve a programming problem.
I've lost count of the number of times Pat has had one of those 'Doh!' moments - you are not alone :wink: :)
....and your conversational style makes a good read...thanks....even if I'm unlikely to use it.... :lol:

Keep up the good work, and the inspiration to those who will use it :D

Aitch :)

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#90 Post by Gedrean »

Okay, here a new template with FOR instead of While.

The template is much larger itself in text than the loop will be, because once this is documented only spot-comments will be needed for tricky lines.

Code: Select all

# This is a FOR loop template, much cleaner than before.
# First, we declare our holder for all codes of this DV type.
# Call it: "DV_<FC>_CODE"
# Strip out the leading = and perform a grep of "DV\|<FC>\|" from CHOOSER2
# and put the result in this holder.
# Declare a variable, DV_<FC>_RESULT="", for storing the end result in.
# This RESULT MUST start out null ("").  NO starter value!
# for COUNTER in DV_<FC>_CODE
#	Set a name equal to the name of the value in COUNTER:
#	NAME=${COUNTER%=*}
#	Set VALUE equal to COUNTER: VALUE=$COUNTER
#	Perform strip command: pcomp_strip VALUE $NAME -- VALUE not $-expanded!
#	if condition: [ $VALUE == "true" ]
#		( We could also use other conditions if value isn't going to be ...
#		a true/false, but that's more complicated )
#		Strip 'DV|<FC>|' from NAME: NAME=${NAME#DV\|<FC>\|}
#		Replace & with = in NAME: NAME=${NAME/\&/=}
#		Format and add NAME to the end of DV_<FC>_RESULT
#	fi
# done
# if condition: [ $DV_<FC>_RESULT != "" ]
#	Perform any additional formatting to result (add starter string) ...
#	and add to final exec string
# fi
Hope this is nicer. I haven't yet rebuilt any of the loop systems (so right now my pcompile copy is actually neutered) but they WILL be done sometime in the next 24-48. :) Soon's I get together a few free hours and the actual will to code. Exhausting day today, Rennaissance Festival was visited. :)

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#91 Post by Gedrean »

Alright, I have completed the loop for Configure Options.

Code: Select all

DV_CO_CODE="`echo "${CHOOSER2#=}" | grep DV\|CO\|`"
# DV_CO_CODE should have the list of all Config Options DV's
DV_CO_RESULT=""
for COUNTER in $DV_CO_CODE
do
	NAME=${COUNTER%=*} # Get everything on the left side of the =.
	VALUE=$COUNTER
	pcomp_strip VALUE $NAME # Now we have the value.
	if [ $VALUE == "true" ]; then
		NAME=${NAME#DV\|CO\|} # Strip the DV/CO indicator
		NAME=${NAME/\&/=} # Replace & with =
		NAME=$NAME" " # Format to include trailing space
		DV_CO_RESULT=$DV_CO_RESULT$NAME # Stick it together
	fi
done
Looks good to me, runs good too! :) And nice and short too!

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#92 Post by Gedrean »

Okay new questions.

I've got some concerns, one of these CFLAGS has an =2 and the other just has a 2, I'm not sure if these flags are exactly as you want them or not. For now, whatever's in the label is EXACTLY how it's getting pushed to the configure command line, so if you go through there and notice some arguments are wrong, they need to be fixed up.

Second, Optimization and Architecture - those are part of CFLAGS?

Third, should -O4 and -Os in LDFLAGS be together?

Also, the commas aren't gonna be in the label, they will be in the string overall, but thanks for the reminder.

Given that, I'm assuming Arch and Opt go to CFLAGS, and coding for that for now.

I like the user-defined sections.

I'm going to try and separate LDFlags and CFlags -- I'm not against a fourth tab, and I think the flow and spacing will work better, as it is CFlags has those three sets of args that are two-column instead of one, and they don't line up well -- admittedly gtkdialog has HORRIBLE layout options, so I'm gonna see just what I can make work to make everything look nice.
--That should also make the window a bit smaller -- it was smaller before, and while bigger isn't terrible, if we're pushing for Lowest Common Denominator I'd want to assume we're working with maybe an 800x600 screen at worst, maybe even 640x480, and so smaller is better if possible.

I don't think it's going bad though.

Keep up with it here, we're getting to a finish line and a release I think for v2. At this point we should release a v2 before moving on to further changes and alterations -- as it is the script code is nice and lean, I've stripped out a lot of things that aren't necessary... commented still but tried to get nicer.

So get back to me on those flags questions, and we'll see what we can pump out. :)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#93 Post by technosaurus »

they are correct:
mpreferred-stack-boundary is looking for a power of two - in this case 2^2

the other fbranch... tells gcc to optimize branches toward the end after a bunch of other optimizations have been done

,-O4,-Os is a really hacky way of ensuring that the code size doesn't accidentally get increased... when linking gcc is supposed to look for -O, but there are plans to add the option of a numerical value (like -O[0-9]) so using -O4 prevents accidental generation of large and often broken -O3 bytecode and the subsequent -Os is there for the same reason. With Puppy's current gcc ,-O4,-Os is no different than using ,-O

btw gcc 4.4.2 is out now. I may build that later today while I am on a long car trip to go hiking with the family. It would be good to test the options on older and newer versions. I also have gcc 3.4.6 statically compiled against dietlibc to see how backward compatible it is.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#94 Post by technosaurus »

I worked on dump2 pet (renamed to possibly keep backward compatibility via a check of /etc/puppyversion > 425)
Attachments
dmp2pet.gz
(2.09 KiB) Downloaded 370 times
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#95 Post by Gedrean »

There's an incredibly easy solution to the backwards compatibility:

Have dump2pet be a wrapper that checks puppyversion and calls one of two scripts in the usr local pcompile directory depending on result.

Post Reply