How to simulate a referer with Perl and Lynx

Public Archives
#1

Some of this German search engines have a new joke.

For luck not something as complicate as Altavista, so it should be
possible to solve:

When I type the URL to submit a page, the search engine tells “URL not
correct entered”.

The same when I am on their site, it works. I fill in the form, press submit,
and at the confirmation page appears exactly the same URL.

Only difference can be: the referer. So how can I simulate a referer in Perl
and Lynx ?!?

–
Roland Mösl http://pege.org Clear targets for a confused civilization
http://BeingFound.com Web Design starts at the search engine

#2

Re: How to simulate a referer with Perl and Lynx

How are you using Perl to send the http request? If it’s with LWP
(a.k.a. libwww-perl), there’s a user_agent variable to set – check the LWP docs

If your perl program is making the actual headers by itself, you should have a
user-agent header like this: User-Agent: Mozilla/4.x (Win95)

It may have been the lack of this or another header that “gave you away.”
Another difference between your browser and the script is the IP address –
maybe they blocked your IP?

#3

Re: How to simulate a referer with Perl and Lynx

“Winston D. Neutel” wrote:
>
> How are you using Perl to send the http request? If it’s with LWP
> (a.k.a. libwww-perl), there’s a user_agent variable to set

Actually, he’d want to look for the HTTP_REFERER variable :wink:

-Mike K.

>
> If your perl program is making the actual headers by itself, you should have a
> user-agent header like this: User-Agent: Mozilla/4.x (Win95)
>
> It may have been the lack of this or another header that “gave you away.”
> Another difference between your browser and the script is the IP address –
> maybe they blocked your IP?

#4

Re: How to simulate a referer with Perl and Lynx

“Winston D. Neutel” <winston@dramaturgy.net> wrote in message
news:99d5gr$24i$1@constitution.worldwebserver.com…
> How are you using Perl to send the http request? If it’s with LWP
> (a.k.a. libwww-perl), there’s a user_agent variable to set – check the LWP
> docs
>
> If your perl program is making the actual headers by itself, you should have a
> user-agent header like this: User-Agent: Mozilla/4.x (Win95)

I use Lynx started by Perl and returning the page to Perl

> It may have been the lack of this or another header that “gave you away.”
> Another difference between your browser and the script is the IP address –
> maybe they blocked your IP?

No, tested both from my computer.

–
Roland Mösl http://pege.org Clear targets for a confused civilization
http://BeingFound.com Web Design starts at the search engine