Tag Archives: wget

How to Set Wget Target Directory Tips, Tricks and Tutorials 12 JUN 2014

If I ever need to pull something down from a website while working on an Ubuntu Server instance, then without a doubt, wget is my go to guy. From the man pages:

GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.

Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site (recursive downloading). Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.

Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.

So all in all, a VERY useful tool. The question being solved today is how to go about setting or specifying the target directory wget should be saving the downloaded files to. The short answer is that in the strictest sense you can’t – but you can specify a string prefix to add to all downloaded files, thereby allowing you to essentially set the wget target directory!

Again from the man pages:

-P prefix
--directory-prefix=prefix
Set directory prefix to prefix.  The directory prefix is the directory where all other files and sub-directories will be saved to, i.e. the top of the retrieval tree.  The default is . (the current directory).

In practice, if we want to download a file to /home/craiglotter then we would add -P /home/craiglotter/ to our command. In practice:

wget -P /home/craiglotter/ http://www.craiglotter.co.za/wp-uploads/image.jpg

The above would download the image.jpg file to /home/craiglotter/image.jpg.

Useful to know.

ubuntu black and orange logo

Related Link: http://manpages.ubuntu.com/manpages/trusty/man1/wget.1.html

How to Download to a Specific Directory using Wget for Windows Tips, Tricks and Tutorials 26 APR 2010

The Wget for Windows application is of course exactly that: the classic GNU/Linux wget application built for Windows command line usage.

Of course, this means that most if not all command line switches are still available to you, just as what they would be had you been running the latest wget from your Linux distro of choice.

The question I’m answering today is how to specify which directory wget is to save what it is downloading to.

Normal usage sees wget placing whatever it grabs in the directory from which it is called.

You can of course specify a specific file to which to save your downloaded web page when downloading a single page by making use of the -O switch, resulting in something that looks like this:

"C:\\wget.exe" -O "C:\\downloadpage.txt" "http://www.download.co.za/index.html"

If however we want to specify the directory to which wget must save its downloaded content to, we need to make use of the -P switch, which the results in a call looking like:

"C:\\wget.exe" -P "C:\\download" "http://www.download.co.za/index.html"

So at the end of the operation specified above, we would be sitting with a file named index.html in the directory C:\\download.

And that’s how you specify to which directory wget must drag its captures down to!

Related Link: http://gnuwin32.sourceforge.net/packages/wget.htm

Download Webpages with Wget for Windows CodeUnit 20 APR 2010

If you are a well established Linux user then you’ll already be pretty familiar with the awesome little wget application that gives you a command line interface with which to download content from web servers via either HTTP, HTTPS and FTP protocols.

Amongst its many features is the ability to set up recursive downloading, conversion of links for offline viewing of local HTML and support for proxies. It is pretty much ubiquitous across most major GNU/Linux distributions and is written in portable C, making it a fairly simpe affair to port it to other operating systems like Microsoft Windows and Mac OS X.

Now for those of you who would like to use something like this little robust, command line and self terminating application for a scheduled task on your great big Windows Server or desktop machine, rejoice because someone has gone to the effort and created a great little port project that offers you a complete wget package for Windows, even going so far as giving you a proper installer to drop everything in its right place.

Simple to install, and even easier to use, go grab your copy of GNU Wget 1.11.4 now! :)

gnu wget install screen

Related Link: http://gnuwin32.sourceforge.net/packages/wget.htm