LazY FReD 2.0.0 - Free Repository Downloader

Miscellaneous tools
Message
Author
User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#16 Post by RSH »

Hi.

If you want to use LazY FReD in MacPup, here is the runtt21 MacPup Repository at http://macpup.org/runtt21/. Just copy the attached list to /usr/local/LazY-FReD/server_lists.

MacPup Repo of runtt21 - .pet in the right frame
Image

RSH
Attachments
runtt21.lst.gz
.lst file - remove.gz
(319 Bytes) Downloaded 698 times
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#17 Post by RSH »

Uuuppsss...

AFI 1.0.2 has had a wrong link in /root/.config/rox.sourceforge.net/OpenWith/.application_x-cd-image/ - so, the frugal install on right-click doesn't work

Is now fixed and new AFI 1.0.2 PET and SFS are already uploaded (using the same version number).

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#18 Post by SFR »

Hey RSH
RSH wrote:Must have / still needed:

- access to the files of the murga forum ! ! !
(Is this possible? Any Suggestion/Idea?)
Are you still interested in that option?

The most convinient solution would be if there is (somewhere?) an index file of all forum attachments.
I was hoping someone knows something about this mysterious index and will post some info, but apparently noone knows. :lol:

In this case the only option I see is to create one.
This exemplary script should do that task, but it'll take a really long time since the first attachment ever seems to have id=100 and, for instance, your LazY-FReD.pet has id=58993.
(If anyone knows faster way than utilized by me - let us know).
Second thing - attachments aren't only .pet, .sfs, etc. there are also pictures and (this will be difficult to filter) fake extensions (mostly .gz).
The script, as it stands, doesn't filter them in any way.
Third thing - there's a lot of missing attachments.
Try to set COUNT=5500 to see several of them in a row.
Currently the script handles that manually, but it'd be more reasonable to set a given upper limit and skip the missing ones automatically.

Here's the script:

Code: Select all

#!/bin/bash

# This script creates the index file of murga-linux forum attachments.
# BTW, to download an attachment, use (for example):
# curl -J -O "http://www.murga-linux.com/puppy/viewtopic.php?mode=attach&id=100"

INDEXFILE="murga-linux_attachments_index.txt"
BASE="http://www.murga-linux.com/puppy/viewtopic.php?mode=attach&id="

clear
case "$1" in
  new)
    echo "Creating new index..."
    [ -e "$INDEXFILE" ] && rm -f $INDEXFILE
    COUNT=100
  ;;
  resume)
    echo "Resuming old index..."
    [ -e "$INDEXFILE" ] && COUNT=$((`tail -1 $INDEXFILE | cut -d " " -f1`+1))
    [ ! $COUNT ] && echo "Something's wrong with the index file, perhaps doesn't exist - check it." && exit 1
  ;;
  *)
    echo "This program creates index of murga-linux forum attachments"
    echo "and saves the file to the current directory"
    echo -e "Usage:\n$0 new - to create new index\n$0 resume - to resume old index"
    exit
  ;;
esac

echo -e "Press x to exit\n"

while true; do
  LINK="$BASE$COUNT"
  TEST=`curl -s -I $LINK | grep 'filename=' | cut -d '=' -f2 | tr -d "\r"`
  if [ ! $TEST ]; then
    echo "Attachemnt $COUNT doesn't exist or reached the last one! Skip it or break (s/b)?"
    read -sn1 KEY
    [ $KEY = "b" ] && break || echo "Skipped..."
  else
    echo $COUNT" "$TEST | tee -a $INDEXFILE
  fi
  read -s -t0.1 -n1  KEY
  [ "$KEY" = "x" ] && break
  ((COUNT++))
done
exit
Of course, you should modify it to your needs, it's just an example. :wink:

Very nice tool, by the way. Congrats! :)

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#19 Post by RSH »

SFR wrote:Are you still interested in that option?
YESSSSS! Absolutely! :P

Uuhhh.... seems to be a long term project. :o

I could imagine there would be a huge number of files available. :lol:

Currently running your script and looks like it needs hours to get the whole list. :shock:

But i think, it should be ok, to create the whole list once (maybe a bit sorted out), split this list into parts (let's say lists of 1000 files in each list) and then only from time to time getting the last 100 files. If any older one changes or will get lost and download fails - doesn't matter.

Maybe list of 100 files will be better - don't know, how many objects can be used in gtkdialog.

Anyway...

...thank you for the script. It will be very useful to get that feature into the LazY FReD and to make it well rounded. :D
SFR wrote:Very nice tool, by the way. Congrats!
Thanks. I think, it's the best application i ever wrote for puppy os and wherein st90's two lines back-end code could find its place ("bestens aufgehoben" - as we say in German). :lol:

Greetings

RSH

P.S.

Getting around 1000 files in about 12 minutes, so it might take somewhat around 12 hours (if i would skip every missed attachement immediately by key "s". :roll:

Oh my god...

...what did i ask for? :lol:
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#20 Post by RSH »

Ok, i see!

This one is the address of the database of murga-linux.com - or better say of a database entry
http://www.murga-linux.com/puppy/viewto ... attach&id=

Each entry of Database must have more information - not only attach&id
Information is shown here
Image
Must be something like (don't know it though):
- attach&id&description
- attach&id&filename
- attach&id&filesize
- attach&id&downloaded

Must be a structure similar to a record in PASCAL and must also use id to make sure to show the correct data.

So, it would be good to know how to get this information (how the access point is really called (named) inside the record).

Also would be good to know if there is more information available f.e. like the forum, where it has been attached.
Might be something like
- attach&id&forum

If so, then it could be sorted in categories like multimedia, graphics etc.pp. That would be very nice. :D

So, anyone any suggestion/idea/knowledge on that issue? Please post it!

I will post this also in the Programming forum

---

Currently 6407 files detected at http://murga-linux.com - and counting (did make comments out of the skip/break functions)
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#21 Post by RSH »

Ok.

Got it.
8)

No, not the application/function ---> the index list. :lol:

Some statistical data?

Here it is
(i do use German form instead of US form (we use "." to mark 1000s)):

- Index of first file: 100
- Index of last file: 59.159

- Number of all files: 59.059

- Number of lines in index file: 43.059

- Difference: 16.000


So as a first result, we have to accept, that murga-forum has lost 16.000 attached files!

Or in other words: 27,091 % of all files are lost (US form: 27.091%)
.
I will write now a script to sort this list into different lists, each for every file type (sfs, pet, archives (excluding .gz because of the fake .gz files), images etc.pp).

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#22 Post by RSH »

Ok,

index list sorted out (a bit).

Here some new statistical data (can include doubles though, because f.e. files like Whippet.png are currently listed in the png-list and also in the pet-list):

pet list: 10.354 files
sfs list: 260 files
gz list: 2400 files (this list contains also the sfs.gz files and surely the fake .gz)
tar.gz list: 4868 files
tgz list: 16 files (all doubles/wrong ---> not really one .tgz file available)
zip list: 1605 files

jpg list 10.893 files
png list: 10.894 files
tif list: 70 files (one file really ---> the rest are doubles)
xpm list: 61 files

Need to refine the script and get the whole list again ---> takes hours!

---

LazY FReD Language Support - translations needed
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#23 Post by greengeek »

RSH wrote:Some statistical data?
Here it is
(i do use German form instead of US form (we use "." to mark 1000s))
Hi RSH, could you ask the Germans to change to using the "comma" to indicate the thousands please? It is too confusing to use the "point". I hope the change will be easy. Thanks :-)

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#24 Post by RSH »

greengeek wrote:Hi RSH, could you ask the Germans to change to using the "comma" to indicate the thousands please? It is too confusing to use the "point". I hope the change will be easy. Thanks
Do you mean to change it generally in our (the Germans) posts or somewhere inside LazY FReD? I'm not sure, what you mean.

Thanks.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#25 Post by greengeek »

Sorry - I meant the whole of Germany. It seems like you use the comma where we use the point and vice versa - Also you drive on the wrong side of the road... (I'm just joking of course...no offense intended...) :-)

User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:

#26 Post by ASRI éducation »

To permit French users to better understand Lazy-Fred, I opened a discussion on the forum fr.
cf. http://www.murga-linux.com/puppy/viewto ... 971#666971

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#27 Post by RSH »

Version 1.0.2-2 uploaded. Link and Info at first post.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:

#28 Post by ASRI éducation »

RSH wrote:Version 1.0.2-2 uploaded. Link and Info at first post.
Merci RSH.

User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:

#29 Post by ASRI éducation »

Sorry for my weak English, I'm French and I use an online translator.

I appreciate the application Lazyred and I believe that this utility could soon be present in the majority of puplets.
In the meantime, I would modestly suggest some changes / developments ...

1 - With small screens (800x600), it is difficult to use because LazyFred GUI is too broad. Ideally, we should reorganize the interface to reduce the space needed when viewing the screen. This makes me think that Ziggy had done with the PPM Puppy.

2 - Users often struggle to understand how to use the application, in particular the update servers (and display of available packages). Is it possible to improve this? Not easy, because it involves automating the update of the display when changing server, I do not know but I'm a novice GTK.

In all cases, LazyFred is a great idea!

Thank you HSN.

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#30 Post by RSH »

ASRI éducation wrote:Sorry for my weak English, I'm French and I use an online translator.

I appreciate the application Lazyred and I believe that this utility could soon be present in the majority of puplets.
In the meantime, I would modestly suggest some changes / developments ...

1 - With small screens (800x600), it is difficult to use because LazyFred GUI is too broad. Ideally, we should reorganize the interface to reduce the space needed when viewing the screen. This makes me think that Ziggy had done with the PPM Puppy.

2 - Users often struggle to understand how to use the application, in particular the update servers (and display of available packages). Is it possible to improve this? Not easy, because it involves automating the update of the display when changing server, I do not know but I'm a novice GTK.

In all cases, LazyFred is a great idea!

Thank you HSN.
Hello, ASRI éducation.

I'm sorry, bu I've now worked for about 16 Months almost every day several hours and sometimes up to 40 hours in a row on LazY Puppy and all "my" applications on the forum. I think, I need a break. I have to take a break.

A long term break from coding, working on the forum and also working on LazY Puppy. This might take a few months where I have to do different stuff from puppy - except: I want to use LazY Puppy, finally, within these break from coding etc.

So, we will see, when I'm back.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:

#31 Post by ASRI éducation »

@ RSH
Hello RSH
I understand that you want to pause.
I renew my congratulations for your various contributions.
More ... Once you have finished your break!

Pelo

Merci pour cette aide précieuse

#32 Post by Pelo »

Lazy Puppy please me more and more. And if there is a forum (even speaking english), it's wonderful.

Nice job.

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#33 Post by RSH »

[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#34 Post by RSH »

I have created a update of LazY-FReD - now 2.0.0.

The GUI is now much smaller and it searches now for some more file types.

It searches now e.g. also for .exe (for provided Puppy Win-Installer files) and an editable list gives option to increase file types to search for.

Version 2.0.0 is now uploaded to the first post of this thread (older version is still available).
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:

#35 Post by ASRI éducation »

RSH wrote:I have created a update of LazY-FReD - now 2.0.0.

The GUI is now much smaller and it searches now for some more file types.

It searches now e.g. also for .exe (for provided Puppy Win-Installer files) and an editable list gives option to increase file types to search for.

Version 2.0.0 is now uploaded to the first post of this thread (older version is still available).
Good new !
Thank you RSH.
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]

Post Reply