LazY FReD 2.0.0 - Free Repository Downloader

Miscellaneous tools
Message
Author
User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

LazY FReD 2.0.0 - Free Repository Downloader

#1 Post by RSH »

Hi!

LazY FReD 2.0.0 and AFI Automated Frugal Installer 1.0.2!

LazY FReD Language Support - translations needed
LazY MAID 0.1

Changes:

LazY FReD 2.0.0

- searches now for some more file types, e.g. .exe (for provided Puppy Win-Installer files)
- an editable list gives option to increase file types to search for
- GUI refined and smaller in its screen-size
- Chinese locales added
- again updated version of getbootdir script
- - (if using older version of getbootdir script (a few of my apps are using this), replace it with this one included in LazY FReD 2.0.0)

LazY FReD 1.0.2-2

- French and Spanish locales added
- updated version of getbootdir script

---

LazY FReD is a tool to download iso, sfs, pet, deb, tar.gz, .tar.bz2, .tgz, .gz and zip files from "free repositories" like smokey01.com. The image shows directory of 01micko's lucid files.

First Tab shows already available sfs files at boot directory in the left frame , the middle frame shows sfs files on the server and the right frame shoes pet files on the server.

Image

Second Tab shows iso files in the left frame, archive files in the middle frame and deb files in the right frame.

Image

Third Tab shows some Information on Data and Application.

Image

Fourth Tab shows some quick settings from the preferences gui

Image

You can select multiple files (from each and every category/frame together) and download all of them. All downloaded files are written into log files which are used to let you load all downloaded sfs files using sfs_load, install all downloaded pet files using petget and extract all downloaded archives using pupzip after the LazY FReD is closed.

Best of all:

I have made an update on the AFI Frugal Installer, so you can install all downloaded iso files onto your HD and/or USB Stick, after the LazY FReD is closed. It uses the name of the iso to make the directory for installing it. Also it can be used now as rox right-click application. Do a right-click on an iso file and choose AFI 1.0.2 install iso.


You can choose where to save the downloaded files. You can choose three different locations. One for SFS files, one for ISO files and one for applications and archive files.

For the SFS files you can set also option to save these files to the boot directory instead of a different directory.

All directories can/must be chosen by a file selector gui - nothing is automatically mounted (this is your job - i don't like auto mounting)

If you are starting LazY FReD, it runs first into the preferences to let you choose the directories and some options on which files to be installed automatically after downloading and closing the LazY FReD.

Image

You can select server addresses from a list (in the main gui) and you are allowed to edit and modify this list to create your personal repository for your self made puppy derivative. I have predefined some server addresses at smokey01.com, so you can play with it. Shinobar's address works now - thanks to stu90(91). Zigbert's web address works now as well - thanks again to stu90(91).

Image

You can have multiple lists to load and to use, containing different server addresses.

Image

I have added 175 different server addresses and mirrors that will work right after installing LazY FReD or loading the SFS file of LazY FReD.

LazY FReD 1.0.2 has now LazY ReCo, a new tool (thanks for the xclip tip to seaside), which can save Server-Addresses from clipboard to a selected Server-List.

Image

It should work in every puppy and i hope you'll find this one useful anyhow to use it for your own purposes.

LazY FReD Language Support - translations needed
LazY MAID 0.1

RSH

By the way: again inspired from a script by stu90 :D

...and please, be kind to FReDdy and ReCo :wink:
Attachments
LazY-FreD-2.0.0.pet
Installation File / Installationsdatei
(93.19 KiB) Downloaded 693 times
LP2_LazY-FreD-2.0.0.sfs.gz
SFS File - remove .gz / Programm-Modul - .gz entfernen
(80 KiB) Downloaded 656 times
LP2_AFI-1.0.2.sfs.gz
SFS File - remove .gz / Programm-Modul - .gz entfernen
(12 KiB) Downloaded 1050 times
AFI-1.0.2.pet
Installationsdatei / Install file
(10.55 KiB) Downloaded 1145 times
LazY-FReD-1.0.2-2.pet
Installationsdatei / Install file
(47.88 KiB) Downloaded 1577 times
LP2_LazY-FReD-1.0.2-2.sfs.gz
SFS File - remove .gz / Programm-Modul - .gz entfernen
(52 KiB) Downloaded 1113 times
Last edited by RSH on Sat 17 May 2014, 00:54, edited 22 times in total.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#2 Post by RSH »

Hi.

Version 0.2 uploaded. Link is in the first post.

Image

CHANGES:

Version 0.2

- now able to get access to shinobar's server (thanks to stu91(90?))
- pet list on server shows now icon for pet files
- can now choose a directory to save to in gtkdialog3

Still needed:

- access to zigbert's server http://www.meownplanet.net/zigbert/
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

hexabit
Posts: 15
Joined: Sat 01 Sep 2012, 12:03
Location: Sweden
Contact:

#3 Post by hexabit »

Works great! :D

backi
Posts: 1922
Joined: Sun 27 Feb 2011, 22:00
Location: GERMANY

#4 Post by backi »

Hi RSH !
Appreciate your work on Lazypup .

LazY FReD downloader..... fantastico......Apllause apllause applause .!!!!
Image

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#5 Post by RSH »

backi wrote:Hi RSH !
Appreciate your work on Lazypup .

LazY FReD downloader..... fantastico......Apllause apllause applause .!!!!
Image
Thank you very much. The whole thing is mainly based on a single line, taken from a script by stu90 (commented line) and modified (second line) a little.

Code: Select all

#AVSFS="` curl -s $SERVER | awk -F '"' '/href=".*\.sfs/ {print $8}' | sort -f -u | tr '\n' '!' | sed '$s/.$//' `"
AVSFS="` curl -s $NEWSERVER | awk -F '"' '/href=".*\.sfs/ {print $8}' | sort -f -u | tr '\n' ' ' | sed '$s/.$//' `"
Inspired by this thread.

By now stu90 did made work on this line (two times) and so i am glad to announce version 0.3!

Changes:

Version 0.3

- access to zigbert's server

Link is in the first post.
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
ally
Posts: 1957
Joined: Sat 19 May 2012, 19:29
Location: lincoln, uk
Contact:

#6 Post by ally »

wonderful pet

thanks rsh.......

:)

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#7 Post by RSH »

Hi.

Version 0.4 uploaded.

CHANGES:

Version 0.4

- shows wrong files in .pet list on server (fixed)
- refresh is now faster - local sfs list is now only created at first start
- all tmp files are now written to /tmp
- German help updated

Must have / still needed:

- access to the files of the murga forum ! ! !
(Is this possible? Any Suggestion/Idea?)

RSH
Attachments
wrong-files-in-list.jpg
(12.3 KiB) Downloaded 2934 times
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
darkcity
Posts: 2534
Joined: Sun 23 May 2010, 19:16
Location: near here
Contact:

#8 Post by darkcity »

great idea, there may be some more sites here-
http://puppylinux.org/wikka/PuppySites

to search murga forum for pets I think you would need to look at the phpBB database.

Somebody wrote a tool for saving forum topics to a document, but I can't find it now. That script would have some hints, so if someone remembers please post a link :idea:

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#9 Post by RSH »

Hi. Version 0.5 uploaded.

Link is in the first post.

CHANGES:

Version 0.5

- automated install of .sfs and .pet files did not work in 0.4 - FIXED - sorry

Must have / still needed:

- access to the files of the murga forum ! ! !
(Is this possible? Any Suggestion/Idea?)
darkcity wrote:great idea, there may be some more sites here-
http://puppylinux.org/wikka/PuppySites

to search murga forum for pets I think you would need to look at the phpBB database.

Somebody wrote a tool for saving forum topics to a document, but I can't find it now. That script would have some hints, so if someone remembers please post a link.
Thanks. Will have a look.

I think you do mean ThreadGet. It's already installed in LazY Puppy.

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
darkcity
Posts: 2534
Joined: Sun 23 May 2010, 19:16
Location: near here
Contact:

#10 Post by darkcity »

thanks, just spent an age looking for that one.

I was thinking of thread saver, which is based on thread get ; -)
http://www.murga-linux.com/puppy/viewtopic.php?t=74404

User avatar
oldyeller
Posts: 889
Joined: Tue 15 Nov 2011, 14:26
Location: Alaska

#11 Post by oldyeller »

RSH wrote:Hi. Version 0.5 uploaded.

Link is in the first post.

CHANGES:

Version 0.5

- automated install of .sfs and .pet files did not work in 0.4 - FIXED - sorry

Must have / still needed:

- access to the files of the murga forum ! ! !
(Is this possible? Any Suggestion/Idea?)
darkcity wrote:great idea, there may be some more sites here-
http://puppylinux.org/wikka/PuppySites

to search murga forum for pets I think you would need to look at the phpBB database.

Somebody wrote a tool for saving forum topics to a document, but I can't find it now. That script would have some hints, so if someone remembers please post a link.
Thanks. Will have a look.

I think you do mean ThreadGet. It's already installed in LazY Puppy.

RSH
Ok my friend you owe me for all the digging that I have done for the two scripts that you have modified and made better. :lol: :lol: :lol: :lol:

the frugal install and this one. :lol: :lol:

So I am thinking that maybe you could help me with a small project for my Manna build that I am getting ready to do. pm me and let me know

User avatar
vicmz
Posts: 1262
Joined: Sun 15 Jan 2012, 22:47

#12 Post by vicmz »

RSH, this is very clever 8)
I always have to search those repos manually and takes much time. Thank you!
I'm looking forward to the internationalized version, non-english-speaking users will love this app. :D
[url=http://murga-linux.com/puppy/viewtopic.php?t=76948]Puppy Linux en español[/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#13 Post by RSH »

vicmz wrote:RSH, this is very clever 8)
I always have to search those repos manually and takes much time. Thank you!
I'm looking forward to the internationalized version, non-english-speaking users will love this app. :D
Thank you very much.

As for the gui: if you see any program of mine the screenshot will show a German gui. But all my programs are already translated to English language. Might not be perfect, but -i think- good enough to understand and to work with.

If you want to help to translate it, you are welcome to do so. Might be good to make a post in the Localization Project of the forum - will do this soon.

I am currently working on updated version 0.6, which comes with a lot of new server addresses (by now i have included 38 addresses that will work and there are more to improve), a better fix for the wrong-files-in-pet-list-on-server problem (less code - thanks again to stu90(91)) and new gtkdialog gui object notepad, which gives a second tab for iso and tar.gz files.

Will be easy to expand then for more file types - if needed.

I think, i will nominate this one as the penthouse pet of the month ...

...aahhmmm i mean the murga forum pet of the month.

Even if we do not have such an event. :lol:

So, 0.6 is on the way.

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#14 Post by RSH »

Hi.

LazY FReD is grown to version 1.0.0 and it is right on the way!

Please read updated information in the first post...

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#15 Post by RSH »

Hi.

LazY FReD 1.0.2 is now ready for download. Link is in the first post. Please read updated information in the first post.

LazY FReD 1.0.2 has now LazY ReCo, a new tool (thanks for the xclip tip to seaside), which can save Server-Addresses from clipboard to a selected Server-List.

Image

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#16 Post by RSH »

Hi.

If you want to use LazY FReD in MacPup, here is the runtt21 MacPup Repository at http://macpup.org/runtt21/. Just copy the attached list to /usr/local/LazY-FReD/server_lists.

MacPup Repo of runtt21 - .pet in the right frame
Image

RSH
Attachments
runtt21.lst.gz
.lst file - remove.gz
(319 Bytes) Downloaded 698 times
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#17 Post by RSH »

Uuuppsss...

AFI 1.0.2 has had a wrong link in /root/.config/rox.sourceforge.net/OpenWith/.application_x-cd-image/ - so, the frugal install on right-click doesn't work

Is now fixed and new AFI 1.0.2 PET and SFS are already uploaded (using the same version number).

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#18 Post by SFR »

Hey RSH
RSH wrote:Must have / still needed:

- access to the files of the murga forum ! ! !
(Is this possible? Any Suggestion/Idea?)
Are you still interested in that option?

The most convinient solution would be if there is (somewhere?) an index file of all forum attachments.
I was hoping someone knows something about this mysterious index and will post some info, but apparently noone knows. :lol:

In this case the only option I see is to create one.
This exemplary script should do that task, but it'll take a really long time since the first attachment ever seems to have id=100 and, for instance, your LazY-FReD.pet has id=58993.
(If anyone knows faster way than utilized by me - let us know).
Second thing - attachments aren't only .pet, .sfs, etc. there are also pictures and (this will be difficult to filter) fake extensions (mostly .gz).
The script, as it stands, doesn't filter them in any way.
Third thing - there's a lot of missing attachments.
Try to set COUNT=5500 to see several of them in a row.
Currently the script handles that manually, but it'd be more reasonable to set a given upper limit and skip the missing ones automatically.

Here's the script:

Code: Select all

#!/bin/bash

# This script creates the index file of murga-linux forum attachments.
# BTW, to download an attachment, use (for example):
# curl -J -O "http://www.murga-linux.com/puppy/viewtopic.php?mode=attach&id=100"

INDEXFILE="murga-linux_attachments_index.txt"
BASE="http://www.murga-linux.com/puppy/viewtopic.php?mode=attach&id="

clear
case "$1" in
  new)
    echo "Creating new index..."
    [ -e "$INDEXFILE" ] && rm -f $INDEXFILE
    COUNT=100
  ;;
  resume)
    echo "Resuming old index..."
    [ -e "$INDEXFILE" ] && COUNT=$((`tail -1 $INDEXFILE | cut -d " " -f1`+1))
    [ ! $COUNT ] && echo "Something's wrong with the index file, perhaps doesn't exist - check it." && exit 1
  ;;
  *)
    echo "This program creates index of murga-linux forum attachments"
    echo "and saves the file to the current directory"
    echo -e "Usage:\n$0 new - to create new index\n$0 resume - to resume old index"
    exit
  ;;
esac

echo -e "Press x to exit\n"

while true; do
  LINK="$BASE$COUNT"
  TEST=`curl -s -I $LINK | grep 'filename=' | cut -d '=' -f2 | tr -d "\r"`
  if [ ! $TEST ]; then
    echo "Attachemnt $COUNT doesn't exist or reached the last one! Skip it or break (s/b)?"
    read -sn1 KEY
    [ $KEY = "b" ] && break || echo "Skipped..."
  else
    echo $COUNT" "$TEST | tee -a $INDEXFILE
  fi
  read -s -t0.1 -n1  KEY
  [ "$KEY" = "x" ] && break
  ((COUNT++))
done
exit
Of course, you should modify it to your needs, it's just an example. :wink:

Very nice tool, by the way. Congrats! :)

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#19 Post by RSH »

SFR wrote:Are you still interested in that option?
YESSSSS! Absolutely! :P

Uuhhh.... seems to be a long term project. :o

I could imagine there would be a huge number of files available. :lol:

Currently running your script and looks like it needs hours to get the whole list. :shock:

But i think, it should be ok, to create the whole list once (maybe a bit sorted out), split this list into parts (let's say lists of 1000 files in each list) and then only from time to time getting the last 100 files. If any older one changes or will get lost and download fails - doesn't matter.

Maybe list of 100 files will be better - don't know, how many objects can be used in gtkdialog.

Anyway...

...thank you for the script. It will be very useful to get that feature into the LazY FReD and to make it well rounded. :D
SFR wrote:Very nice tool, by the way. Congrats!
Thanks. I think, it's the best application i ever wrote for puppy os and wherein st90's two lines back-end code could find its place ("bestens aufgehoben" - as we say in German). :lol:

Greetings

RSH

P.S.

Getting around 1000 files in about 12 minutes, so it might take somewhat around 12 hours (if i would skip every missed attachement immediately by key "s". :roll:

Oh my god...

...what did i ask for? :lol:
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#20 Post by RSH »

Ok, i see!

This one is the address of the database of murga-linux.com - or better say of a database entry
http://www.murga-linux.com/puppy/viewto ... attach&id=

Each entry of Database must have more information - not only attach&id
Information is shown here
Image
Must be something like (don't know it though):
- attach&id&description
- attach&id&filename
- attach&id&filesize
- attach&id&downloaded

Must be a structure similar to a record in PASCAL and must also use id to make sure to show the correct data.

So, it would be good to know how to get this information (how the access point is really called (named) inside the record).

Also would be good to know if there is more information available f.e. like the forum, where it has been attached.
Might be something like
- attach&id&forum

If so, then it could be sorted in categories like multimedia, graphics etc.pp. That would be very nice. :D

So, anyone any suggestion/idea/knowledge on that issue? Please post it!

I will post this also in the Programming forum

---

Currently 6407 files detected at http://murga-linux.com - and counting (did make comments out of the skip/break functions)
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

Post Reply