How does changing an HTTP referrer header help circumvent crawler blocking

I have been researching different ways a web crawler might be blacklisted/blocked from a web-server and how to potentially circumvent that. One of those ways is to change the referrer header on the request. I have been going to various places trying to figure out the benefit of doing this, but I believe I am thinking about it too hard and have tunnel vision.

How to get the previous url using PHP

Suppose my site’s url is given as hyperlink on some page on the internet; that page could be anything on internet – blog, orkut, yahoo, even stackoverflow etc, and someone clicks on it,and visited my site. So can we know, using php, the previous url from which the visitor came to my page?

URLReferrer is null when page is HTTPS

We use the URLReferrer and a code passed in on the query string to produce online videos so that only our paid clients can link to our video playback page. This system has worked well for some time. I know the URL referrer can be spoofed, but who would tell their clients to do such a thing to access a video ? It’s worked well for us.