Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 21 Dec 2014, 02:10
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Wget 1.13.4 ( Lucid 528-005 )
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [3 Posts]  
Author Message
chrome307


Joined: 15 Jan 2009
Posts: 713

PostPosted: Fri 11 May 2012, 09:04    Post subject:  Wget 1.13.4 ( Lucid 528-005 )
Subject description: Command line retrieving files using HTTP, HTTPS and FTP
 

* Updated package - Lucid 528-005 uses 1.12



Homepage:

http://www.gnu.org/software/wget/

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including:

    Can resume aborted downloads, using REST and RANGE
    Can use filename wild cards and recursively mirror directories
    NLS-based message files for many different languages
    Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
    Runs on most UNIX-like operating systems as well as Microsoft Windows
    Supports HTTP proxies
    Supports HTTP cookies
    Supports persistent HTTP connections
    Unattended / background operation
    Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
    GNU Wget is distributed under the GNU General Public License.


NB wget-1.13.4-i686 is the binary file, the other two are not required unless you want the documentation and native language support (non-english)
libiconv-1.14-i686.pet
Description 
pet

 Download 
Filename  libiconv-1.14-i686.pet 
Filesize  1.37 MB 
Downloaded  279 Time(s) 
wget_NLS-1.13.4-i686.pet
Description  Native Language Support
pet

 Download 
Filename  wget_NLS-1.13.4-i686.pet 
Filesize  444.84 KB 
Downloaded  259 Time(s) 
wget_DOC-1.13.4-i686.pet
Description  Documentation
pet

 Download 
Filename  wget_DOC-1.13.4-i686.pet 
Filesize  89.32 KB 
Downloaded  246 Time(s) 
wget-1.13.4-i686.pet
Description  18-Sep-2011
pet

 Download 
Filename  wget-1.13.4-i686.pet 
Filesize  150.03 KB 
Downloaded  273 Time(s) 

Last edited by chrome307 on Mon 14 May 2012, 03:59; edited 1 time in total
Back to top
View user's profile Send private message 
RetroTechGuy


Joined: 15 Dec 2009
Posts: 2673
Location: USA

PostPosted: Sun 13 May 2012, 21:18    Post subject:  

And how about a sample batch file (I don't recall what version of wget I'm using, but I doubt there have been any significant changes)

I have a file called something like "get50" (as in get 50 MB, then exit). Create a file, cut and paste the following text into it.

Code:
wget --mirror --recursive --continue --proxy=on --convert-links --quota=50M --timeout=5 --tries=30 --input-file=urls.txt --output-file=get.log


What I do from the command line is:

Code:
source ./get50


Looking at the line, you can see the quota adjust (adjust to your preferred level).

You insert a stack of URLS in a file called "urls.txt", which must sit in the local directory. Note: if the files in the urls.txt are already "gotten", wget skips and then continues on the next file/url. No limit to the number of URLS in the file (that I'm aware of...).

The log file is also written to the local directory.

How the 50 MB limit works under wget is: it "gets" files until the download total exceeds 50 MB, and then continues until the current file is complete.

So, you could start the above on a typical Puppy ISO (~100MB), and it would complete a total of 1 file (50MB or 1 file, whichever is greater).

Keep in mind that not all sites tolerate "robots", so wget may not be welcome at all of them.

In years past, I would often do something of the nature (command line only, no GUI):

Code:
source ./get50 ; poweroff


Which would complete the operation, then perform a machine shutdown.

_________________
Add swapfile
Back to top
View user's profile Send private message 
chrome307


Joined: 15 Jan 2009
Posts: 713

PostPosted: Sun 19 Aug 2012, 19:58    Post subject:  

wget-1.14-i686.pet
Description  05-Aug-2012
pet

 Download 
Filename  wget-1.14-i686.pet 
Filesize  749.48 KB 
Downloaded  196 Time(s) 
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [3 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0524s ][ Queries: 12 (0.0052s) ][ GZIP on ]