Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 01 Jul 2015, 23:38
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Wget 1.13.4 ( Lucid 528-005 )
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 1 Posts_count  
Author Message

Joined: 15 Jan 2009
Posts: 713

PostPosted: Fri 11 May 2012, 09:04    Post_subject:  Wget 1.13.4 ( Lucid 528-005 )
Sub_title: Command line retrieving files using HTTP, HTTPS and FTP

* Updated package - Lucid 528-005 uses 1.12



GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including:

    Can resume aborted downloads, using REST and RANGE
    Can use filename wild cards and recursively mirror directories
    NLS-based message files for many different languages
    Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
    Runs on most UNIX-like operating systems as well as Microsoft Windows
    Supports HTTP proxies
    Supports HTTP cookies
    Supports persistent HTTP connections
    Unattended / background operation
    Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
    GNU Wget is distributed under the GNU General Public License.

NB wget-1.13.4-i686 is the binary file, the other two are not required unless you want the documentation and native language support (non-english)
Description  Documentation

Filename  wget_DOC-1.13.4-i686.pet 
Filesize  89.32 KB 
Downloaded  282 Time(s) 
Description  18-Sep-2011

Filename  wget-1.13.4-i686.pet 
Filesize  150.03 KB 
Downloaded  307 Time(s) 

Back to top
View user's profile Send_private_message 

Joined: 15 Dec 2009
Posts: 2780
Location: USA

PostPosted: Sun 13 May 2012, 21:18    Post_subject:  

And how about a sample batch file (I don't recall what version of wget I'm using, but I doubt there have been any significant changes)

I have a file called something like "get50" (as in get 50 MB, then exit). Create a file, cut and paste the following text into it.

wget --mirror --recursive --continue --proxy=on --convert-links --quota=50M --timeout=5 --tries=30 --input-file=urls.txt --output-file=get.log

What I do from the command line is:

source ./get50

Looking at the line, you can see the quota adjust (adjust to your preferred level).

You insert a stack of URLS in a file called "urls.txt", which must sit in the local directory. Note: if the files in the urls.txt are already "gotten", wget skips and then continues on the next file/url. No limit to the number of URLS in the file (that I'm aware of...).

The log file is also written to the local directory.

How the 50 MB limit works under wget is: it "gets" files until the download total exceeds 50 MB, and then continues until the current file is complete.

So, you could start the above on a typical Puppy ISO (~100MB), and it would complete a total of 1 file (50MB or 1 file, whichever is greater).

Keep in mind that not all sites tolerate "robots", so wget may not be welcome at all of them.

In years past, I would often do something of the nature (command line only, no GUI):

source ./get50 ; poweroff

Which would complete the operation, then perform a machine shutdown.

Add swapfile
WellMinded Search
Back to top
View user's profile Send_private_message 

Joined: 15 Jan 2009
Posts: 713

PostPosted: Sun 19 Aug 2012, 19:58    Post_subject:  

Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 1 of 1 Posts_count  
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Jump to:  

You cannot attach files in this forum
You can download files in this forum

Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0504s ][ Queries: 13 (0.0056s) ][ GZIP on ]