File leacher

[pooky2483]

Looks like I'm at a dead end then. Back to manually downloading them...

[SeZo]

Looks like I'm at a dead end then. Back to manually downloading them...

Wget follows the breadcrumbs (links) in the pages it comes accross.

I guess if the pages are loaded by script, then you are outta luck.
However if you would start with a known subsection which contains the links to the pdfs then you might get somewhere.

Code: [Select]

wget --random-wait --limit-rate=20k -r --no-parent -l10 -A.pdf http://website.com/subfolder

That should start from the specified url (ignoring the directories above) and will go to 10 level deep.

Like Mark said, they might just tell you to sod off and do the downloading like everyone else does.

[Mark Greaves (PCNetSpec)]

I had a "quick" look at the site in question, and the pdf always seems to get oaded from the same link, which "becomes" a link to the selected pdf .. so I'm guessing it's ll scripted.

Nor wll it let you "browse" the directory it says the pdf comes from.

That said, playing with wget::gui and trying a few random options seemed to download quite a few directories .. but no pdf's, just a bunch of xml files, even though I'd told it to get pdf's
(maybe I just didn't give it enough time to follow all the links).

I'd also guess there will be gigabytes of pdf's on that site .. which would explain them not wanting 1000's of people all downloading the lot in one go .. they'd lock up the site for hours.

[pooky2483]

Just thought it'd be worth a try to get the PDF's in one go but if it can't be done, nevermind.
Thanks anyways for trying.