Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 25 Oct 2014, 00:39
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Wget 1.13.4 ( Lucid 528-005 )
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 1 Posts_count  
Author Message
chrome307


Joined: 15 Jan 2009
Posts: 713

PostPosted: Fri 11 May 2012, 09:04    Post_subject:  Wget 1.13.4 ( Lucid 528-005 )
Sub_title: Command line retrieving files using HTTP, HTTPS and FTP
 

* Updated package - Lucid 528-005 uses 1.12



Homepage:

http://www.gnu.org/software/wget/

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including:

    Can resume aborted downloads, using REST and RANGE
    Can use filename wild cards and recursively mirror directories
    NLS-based message files for many different languages
    Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
    Runs on most UNIX-like operating systems as well as Microsoft Windows
    Supports HTTP proxies
    Supports HTTP cookies
    Supports persistent HTTP connections
    Unattended / background operation
    Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
    GNU Wget is distributed under the GNU General Public License.


NB wget-1.13.4-i686 is the binary file, the other two are not required unless you want the documentation and native language support (non-english)
libiconv-1.14-i686.pet
Description 
pet

 Download 
Filename  libiconv-1.14-i686.pet 
Filesize  1.37 MB 
Downloaded  269 Time(s) 
wget_NLS-1.13.4-i686.pet
Description  Native Language Support
pet

 Download 
Filename  wget_NLS-1.13.4-i686.pet 
Filesize  444.84 KB 
Downloaded  254 Time(s) 
wget_DOC-1.13.4-i686.pet
Description  Documentation
pet

 Download 
Filename  wget_DOC-1.13.4-i686.pet 
Filesize  89.32 KB 
Downloaded  243 Time(s) 
wget-1.13.4-i686.pet
Description  18-Sep-2011
pet

 Download 
Filename  wget-1.13.4-i686.pet 
Filesize  150.03 KB 
Downloaded  268 Time(s) 

Edited_time_total
Back to top
View user's profile Send_private_message 
RetroTechGuy


Joined: 15 Dec 2009
Posts: 2668
Location: USA

PostPosted: Sun 13 May 2012, 21:18    Post_subject:  

And how about a sample batch file (I don't recall what version of wget I'm using, but I doubt there have been any significant changes)

I have a file called something like "get50" (as in get 50 MB, then exit). Create a file, cut and paste the following text into it.

Code:
wget --mirror --recursive --continue --proxy=on --convert-links --quota=50M --timeout=5 --tries=30 --input-file=urls.txt --output-file=get.log


What I do from the command line is:

Code:
source ./get50


Looking at the line, you can see the quota adjust (adjust to your preferred level).

You insert a stack of URLS in a file called "urls.txt", which must sit in the local directory. Note: if the files in the urls.txt are already "gotten", wget skips and then continues on the next file/url. No limit to the number of URLS in the file (that I'm aware of...).

The log file is also written to the local directory.

How the 50 MB limit works under wget is: it "gets" files until the download total exceeds 50 MB, and then continues until the current file is complete.

So, you could start the above on a typical Puppy ISO (~100MB), and it would complete a total of 1 file (50MB or 1 file, whichever is greater).

Keep in mind that not all sites tolerate "robots", so wget may not be welcome at all of them.

In years past, I would often do something of the nature (command line only, no GUI):

Code:
source ./get50 ; poweroff


Which would complete the operation, then perform a machine shutdown.

_________________
Add swapfile
Back to top
View user's profile Send_private_message 
chrome307


Joined: 15 Jan 2009
Posts: 713

PostPosted: Sun 19 Aug 2012, 19:58    Post_subject:  

wget-1.14-i686.pet
Description  05-Aug-2012
pet

 Download 
Filename  wget-1.14-i686.pet 
Filesize  749.48 KB 
Downloaded  190 Time(s) 
Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 1 of 1 Posts_count  
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0552s ][ Queries: 13 (0.0092s) ][ GZIP on ]