The time now is Sun 19 May 2013, 03:37
All times are UTC - 4 |
| Author |
Message |
chrome307

Joined: 15 Jan 2009 Posts: 713
|
Posted: Fri 11 May 2012, 09:04 Post subject:
Wget 1.13.4 ( Lucid 528-005 ) Subject description: Command line retrieving files using HTTP, HTTPS and FTP |
|
* Updated package - Lucid 528-005 uses 1.12
Homepage:
http://www.gnu.org/software/wget/
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.
GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy, including:
Can resume aborted downloads, using REST and RANGE
Can use filename wild cards and recursively mirror directories
NLS-based message files for many different languages
Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
Runs on most UNIX-like operating systems as well as Microsoft Windows
Supports HTTP proxies
Supports HTTP cookies
Supports persistent HTTP connections
Unattended / background operation
Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
GNU Wget is distributed under the GNU General Public License.
NB wget-1.13.4-i686 is the binary file, the other two are not required unless you want the documentation and native language support (non-english)
 |
| Description |
|

Download |
| Filename |
libiconv-1.14-i686.pet |
| Filesize |
1.37 MB |
| Downloaded |
193 Time(s) |
| Description |
Native Language Support
|

Download |
| Filename |
wget_NLS-1.13.4-i686.pet |
| Filesize |
444.84 KB |
| Downloaded |
184 Time(s) |
| Description |
Documentation
|

Download |
| Filename |
wget_DOC-1.13.4-i686.pet |
| Filesize |
89.32 KB |
| Downloaded |
173 Time(s) |
| Description |
18-Sep-2011
|

Download |
| Filename |
wget-1.13.4-i686.pet |
| Filesize |
150.03 KB |
| Downloaded |
199 Time(s) |
Last edited by chrome307 on Mon 14 May 2012, 03:59; edited 1 time in total
|
|
Back to top
|
|
 |
RetroTechGuy

Joined: 15 Dec 2009 Posts: 2298 Location: USA
|
Posted: Sun 13 May 2012, 21:18 Post subject:
|
|
And how about a sample batch file (I don't recall what version of wget I'm using, but I doubt there have been any significant changes)
I have a file called something like "get50" (as in get 50 MB, then exit). Create a file, cut and paste the following text into it.
| Code: | | wget --mirror --recursive --continue --proxy=on --convert-links --quota=50M --timeout=5 --tries=30 --input-file=urls.txt --output-file=get.log |
What I do from the command line is:
Looking at the line, you can see the quota adjust (adjust to your preferred level).
You insert a stack of URLS in a file called "urls.txt", which must sit in the local directory. Note: if the files in the urls.txt are already "gotten", wget skips and then continues on the next file/url. No limit to the number of URLS in the file (that I'm aware of...).
The log file is also written to the local directory.
How the 50 MB limit works under wget is: it "gets" files until the download total exceeds 50 MB, and then continues until the current file is complete.
So, you could start the above on a typical Puppy ISO (~100MB), and it would complete a total of 1 file (50MB or 1 file, whichever is greater).
Keep in mind that not all sites tolerate "robots", so wget may not be welcome at all of them.
In years past, I would often do something of the nature (command line only, no GUI):
| Code: | | source ./get50 ; poweroff |
Which would complete the operation, then perform a machine shutdown.
_________________ Wellminded Search
Add swapfile
|
|
Back to top
|
|
 |
chrome307

Joined: 15 Jan 2009 Posts: 713
|
Posted: Sun 19 Aug 2012, 19:58 Post subject:
|
|
| Description |
05-Aug-2012
|

Download |
| Filename |
wget-1.14-i686.pet |
| Filesize |
749.48 KB |
| Downloaded |
129 Time(s) |
|
|
Back to top
|
|
 |
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|