Ping All Search Engines
Nothing much to see here. Needed this little puppy for work and figured I and others will need it for other projects. Enjoy!
- search
- all
- ping
- engines
- yahoo
- live
- ask
Snippet List
Web crawler/bot detection and blocking middleware
Sets request.is_crawler Allow bot lockout from certain urls in urlconf ,add view parameter 'deny_crawlers' ex. url(r'^foo/$', 'views.foo',{'deny_crawlers' : True},name='foo') view param is removed after middleware pass.
- middleware
- bot
- yahoo
- crawler
- robot
- googlebot
- cuil
- slurp