Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 13 Dec 2017, 18:58
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Suggestions
A local /usr/share/docs/packages.htm
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [7 Posts]  
Author Message
sc0ttman


Joined: 16 Sep 2009
Posts: 2548
Location: UK

PostPosted: Mon 11 Sep 2017, 12:28    Post subject:  A local /usr/share/docs/packages.htm
Subject description: A page listing all the packages in your Puppies repo, with links to download
 

Suggestion: An even easier way to download packages from online

SUMMARY:

A shell script to generate a static HTML page, maybe during a woof build
which lists a table of packages for each supported repo, with each pkg
name being clickable link, straight to pkg (.deb etc) download.

(This is a much simpler alternative to hosting a cool package search site
online somewhere, and a nice page to link to from the default homepage)

DETAILS:

One of Puppies main goals is ease of use.
Puppy often goes the extra mile to make things, easier, simpler, faster.


Many users (not myself, ever) go to Debian/Ubuntu/Slackware the big
package websites like https://packages.debian.org/ or
https://packages.ubuntu.com/) to download pkgs manually.

For some, this is reliable, and "just works" .. But it's a pain, and those
sites are not at all user friendly for non-Linux users or newbies.

Plus, you have to know about them and find them first..

!! For some, their preferred browser might not even work with those sites !!

Take for example, the Debian 'Ace of Penguins' package page:

https://packages.debian.org/stretch/games/ace-of-penguins

It lists some of deps that might break a Puppy users system.

The pkg itself is way down at the bottom, where it lists lots of debs, most for
other architectures, and also lots of irrelevant guff on the right.

Finding the correct deb is not easy for newbies, and a needless hassle for regulars..

...But we know we want to download the .deb for the running $ARCH..
.. and woof worked out which mirrors are working during build time..

Maybe links to the packages can be auto-generated with something like:
Code:
<a href="http://$MIRROR1/$DISTRO_COMPAT_VERSION/$DBIN_ARCH/$PKGNAME/download">i386</a>


So ... stay with me ...

Instead of the user:

- finding the Debian/Ubuntu/Slack/other packages site themselves,
- then finding the right DISTRO_BINARY_COMPAT version (stretch, trusty, wheezy, xenial, tahr, etc),
- then finding the package,
- then finding the right architecture,
- then choosing a mirror,
- and finally getting their download...

We could have a link in /usr/share/docs/index.htm to a page which
was generated in woof at build time, and contains a simple listing
of all pkgs from that puppies repo files.

For example, /usr/share/docs/packages.htm

Which could list ALL packages available in the repos, each repo in a
separate HTML table, with name, deps, size, description, and a clickable
link to download the package (woof knows the mirrors that worked at
build time, at least!).

Any deps listed could simply be anchor links (like #pkgname),
which takes the user straight to the package on that same page! How easy..

As in my other post about the default Puppy homepage, we know that
at build time, woof knows (more or less) enough about the Puppy being
built to generate links to specific, relevant web pages..


.... This (mostly generation of a big long HTML table in woof
somewhere at build time) may be a great way to easily download
pkgs from all over the place , and save Puppy users having to trawl
around the interwebz..

---- Improvements ----

However, lots of long table are boring, and hard to scroll around, so some
'accordion' CSS/JS could be used to make each repo table expand/contract
when its Heading is clicked ..

Same for package entries themselves - click a package row, it could
expand/contract to show/hide more details..

(So, something like the pkgs.org interface, but a local single HTML file,
listing all locally installed repos, and generated by a script)

The script that generated this packages.htm page could be re-ran to keep
it up to date (if its generated from the local config and repo files anyway)..

... Just a suggestion..

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search

Last edited by sc0ttman on Tue 12 Sep 2017, 12:16; edited 1 time in total
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2548
Location: UK

PostPosted: Mon 11 Sep 2017, 14:46    Post subject:  

Now more than just a suggestion...

DEMO ONLY

Some links work (pets download OK), but some don't..

The case statement which builds URLs need fixing/finishing..

Screenshot:


Larger image

Save the code below as make_packages_page.sh:

Then do:

Code:
chmod +x ./make_packages_page.sh


Code:
#!/bin/ash
#
#   makepackagespage.sh - builds an HTML page listing all packages in
#                    all available repo on the running system
#

. /etc/DISTRO_SPECS
. /root/.packages/PKGS_MANAGEMENT

rm -f /tmp/packages.htm

# create our new page
echo '<!DOCTYPE html>
<html>
<head>
<title>All Software Packages</title>
<style>
body {
font-size: 15px;
padding: 0;
margin: 0;
}
.intro {
background-color:#FFF3A3;
font-weight: bold;
margin: 0 auto;
margin-left:28px;
margin-right:28px;
padding:10px;
}
.repo {
margin-bottom: 28px;
}
table {
background-color:#EFEFEF;
margin: 0 auto;
margin-top: 20px;
width: 98%;
}
th {
background-color: #2D98FF;
color: white;
font-size:20px;
text-align: center;
padding-top:10px;
padding-bottom:10px;
}
tr {
margin-top:5px;
margin-bottom:5px;
padding-top:5px;
padding-bottom:5px;
}
.n {
font-weight: bold;
padding-left:6px;
padding-right:6px;
}
.s {
padding-right:6px;
}
.c {
padding-right:6px;
}
.d {
width: 50%;
min-width: 50%;
padding-right:6px;
}
.e {
padding-right:6px;
}
</style>
</head>
<body>
<center>
<h1>'$DISTRO_NAME' Packages</h1>
<div class="intro">
Here are all the packages available for your system.
</div>
</center>' > /tmp/packages.htm

pkgurl=''

for RF in $PKG_REPOS_ENABLED
do
   echo '<div class="repo_block">'    >> /tmp/packages.htm
   
   # create container and table for this repo
   echo '<div class="repo">'         >> /tmp/packages.htm

   echo '<table>'                  >> /tmp/packages.htm
   echo '<thead>'                  >> /tmp/packages.htm
   echo '<tr>'                     >> /tmp/packages.htm
   echo '<th colspan="5">'            >> /tmp/packages.htm
   echo $RF                     >> /tmp/packages.htm
   echo '</th>'                  >> /tmp/packages.htm
   echo '</tr>'                  >> /tmp/packages.htm
   echo '<tr>'                     >> /tmp/packages.htm
   echo '</thead>'                  >> /tmp/packages.htm

   echo "Building pkg list for `basename $RF`, please wait."
      
   case $RF in
      Packages-puppy*noarch*) pkgurl="http://ftp.nluug.nl/os/Linux/distr/puppylinux/pet_packages-noarch";;
      Packages-puppy*common*) pkgurl="http://ftp.nluug.nl/os/Linux/distr/puppylinux/pet_packages-common";;
      Packages-puppy*slacko*) pkgurl="http://distro.ibiblio.org/puppylinux/pet_packages-slacko$DISTRO_COMPAT_VERSION";;
      Packages-*slackware*)    pkgurl="http://slackware.mirrors.tds.net/pub/slackware/slackware-$DISTRO_COMPAT_VERSION";;
      Packages-*blah*)       pkgurl="http://distro.ibiblio.org/more/links";;
      Packages-other*)       pkgurl='' ;;
   esac
   
   cat /root/.packages/$RF | while read pkgline
   do
      # get pkg details
      pkgname=`echo $pkgline | cut -f2  -d'|'`
      pkgfile=`echo $pkgline | cut -f8  -d'|'`
      pkgcat=` echo $pkgline | cut -f5  -d'|'`
      pkgdir=` echo $pkgline | cut -f7  -d'|'`; [ "$pkgdir" != '' ] && pkgdir="$pkgdir/"
      pkgsize=`echo $pkgline | cut -f6  -d'|'`
      pkgdeps=`echo $pkgline | cut -f9  -d'|'`
      pkgdesc=`echo $pkgline | cut -f10 -d'|'`
      
      # build our pkg URL
      pkgurlfull="$pkgurl/$pkgdir$pkgfile"
      
      pkgrow=''
      # each each pkg to its repo table
      pkgrow="$pkgrow"'<tr>'
      pkgrow="$pkgrow"'<td class="n">'
      pkgrow="$pkgrow"'<a href="'$pkgurlfull'">'$pkgname'</a>'
      pkgrow="$pkgrow"'</td>'
      pkgrow="$pkgrow"'<td class="c">'
      pkgrow="$pkgrow"''$pkgcat
      pkgrow="$pkgrow"'</td>'
      pkgrow="$pkgrow"'<td class="s">'
      pkgrow="$pkgrow"''$pkgsize
      pkgrow="$pkgrow"'</td>'
      pkgrow="$pkgrow"'<td class="d">'
      pkgrow="$pkgrow"''$pkgdesc
      pkgrow="$pkgrow"'</td>'
      pkgrow="$pkgrow"'<td class="e">'
      pkgrow="$pkgrow"`echo $pkgdeps | head -c 50`
      pkgrow="$pkgrow"'</td>'
      pkgrow="$pkgrow"'</tr>'
      
      #echo "$pkgname $pkgurlfull $pkgcat $pkgsize $pkgdeps $pkgdesc" | head -c 78
      
      echo "$pkgrow" >> /tmp/packages.htm
   done

   # finished adding pkgs, close table and container
   echo '</table>'   >> /tmp/packages.htm
   echo '</div>'   >> /tmp/packages.htm
done

# finish our page
echo '</body></html>' >> /tmp/packages.htm

# final checks
if [ -f /tmp/packages.htm ];then
   mv /tmp/packages.htm /usr/share/doc/packages.htm
   echo "Page created: /tmp/packages.htm"
   # open it
   defaultbrowser /usr/share/doc/packages.htm
else
   echo "Page NOT created!"
   exit 1
fi


exit 0


EDIT: updated code above, less spacing in final HTML file.

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search

Last edited by sc0ttman on Mon 11 Sep 2017, 20:03; edited 3 times in total
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 3036
Location: Worcestershire, UK

PostPosted: Mon 11 Sep 2017, 15:21    Post subject:  

Interesting idea....

Well it worked and created a nice webpage (on LxPupSc).....

but oh-boy took a longggggg time and created a 3.8MB file so not something you'd want to add to a build...

The repos included should probably be constrained to be the same as PPM by:
$PKG_REPOS_ENABLED
from /root/.packages/PKGS_MANAGEMENT ??

Cheers
peebee
Screenshot.png
 Description   
 Filesize   151.76 KB
 Viewed   131 Time(s)

Screenshot.png


_________________
LxPup = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
Keef


Joined: 20 Dec 2007
Posts: 874
Location: Staffordshire

PostPosted: Mon 11 Sep 2017, 15:30    Post subject:  

Ran this on Quirky, and it also took a while to generate. Slow to load too. 32MB!
The URLs are not being made properly though.
eg First link is for 915resolution, but the URL is:
Code:
file:///915resolution-20110617-x86_64.pet


All other links start with file:/// as well.
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2548
Location: UK

PostPosted: Mon 11 Sep 2017, 15:50    Post subject:  

Keef wrote:
Ran this on Quirky, and it also took a while to generate. Slow to load too. 32MB!
The URLs are not being made properly though.
eg First link is for 915resolution, but the URL is:
Code:
file:///915resolution-20110617-x86_64.pet


All other links start with file:/// as well.


Yeah its just demo code...

The case statement needs to be fixed to account for the various repos,
and their various URL syntaxes, vars, etc...


On Slacko 6.9 with the code above, I get a 2133k file, took about 4 mins!

Possible improvements

- build different HTML pages for each repo..
- use *something* like the following instead of reading each repo line:

Code:
cut -f2,5,6,7,8,9,10 -d'|' ~/.packages/Packages-puppy-slacko14.2-official | sed -e "s/^/<td>/" -e "s/|/<\/td><td>/g" -e "s/\$/<\/td>/g"


The above will cut the fields we need, then convert ^ (start of line), the | and also
$ (end of line) to HTLM <td> stuff .. It is much faster as it does the file all in one
go (not line by line).. but won't give you links/urls ..

EDIT:

FWIW, I cleaned up the file a little, keeping the HTML as concise as
possible, got the file down to ~1.8mb, then made an sfs of the .htm
file to see (very roughly) how small it would compress:

# du -h packages.sfs
224K packages.sfs

EDIT:

Updated to use Peebees $PKG_REPOS_ENABLED suggestion.

Could also remove deps and/or description to make things faster and smaller.

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2548
Location: UK

PostPosted: Mon 11 Sep 2017, 22:30    Post subject:  

Updated, in case anyone is interested in doing something with this..

- separate repos into their own pages
- generate an index page to choose repos

The repos index page (/usr/share/doc/packages.htm):


.. still slow.. still large HTML files (but they compress well, and could make a
small PET package).

Attached a fake gz ...

Download and remove the extension, then

Code:
chmod +x makepackagepages.sh
mv ./makepackagepages.sh /usr/sbin/make_packages_pages.sh
make_packages_pages.sh


Your files end up in /usr/share/doc/packages*.htm
makepackagespage.sh.gz
Description 
gz

 Download 
Filename  makepackagespage.sh.gz 
Filesize  4.88 KB 
Downloaded  17 Time(s) 

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2548
Location: UK

PostPosted: Tue 12 Sep 2017, 12:07    Post subject:  

If I was gonna extend this, I would progressively enhance with JS:

1. Add a search filter
- add a <script> tag just before the closing </body> tag of each page
- use JS to put a search box as the top of each repo page
- inside the <script> tag:
- use JS to get search term from search input field
- hide (add css display:none) to all non-matching <tr> elems
- clear search field to unhide all elems

2. Don't list deps in the main tables
- list deps as data-deps="" in the <tr>s instead
- click a package, get a popup modal with all pkg details
- modal lists deps as links, to easily download package and its deps all in one place
- popup modal can be achieved with a totally centered <div>, plus CSS to hide/show it
- click the DOWNLOAD button in the modal popup to download the pkg

3. Auto generate links to more packages
- as in this thread, generate links to Slackware/Ubu/Deb package sites
- show links to extra packages under the repos list, in same style

With 'progressive enhancement', features 1 and 2 would only appear if the
users browser supported JS, otherwise they would get just the static
HTML pages as they are now.


...Alternatively, the sh script could be altered to generate a PERL .cgi file,
and the page loaded up on http://localhost, and then more dynamic
behaviours and backend cleverness could be added that way (query strings, calling cmds on the system, etc).

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [7 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Suggestions
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1862s ][ Queries: 15 (0.0120s) ][ GZIP on ]