Opened 18 years ago
Last modified 10 years ago
#5177 new enhancement
norobots option for: raw source code, rss, old source code
| Reported by: | ThurnerRupert | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | next-major-releases |
| Component: | version control/browser | Version: | 0.10.3 |
| Severity: | normal | Keywords: | robots noindex |
| Cc: | osimons | Branch: | |
| Release Notes: | |||
| API Changes: | |||
| Internal Changes: | |||
Description
it would be beneficial if there would be an option to prevent search engines from spidering:
- rss
- raw contents if the contents is displayed beautifully rendered in trac
- old versions out of svn
via e.g. norobots or similar.
Attachments (0)
Change History (9)
comment:1 by , 18 years ago
comment:3 by , 18 years ago
According to this page, rel="nofollow" does not mean the same thing as the nofollow that you would put in a typical robots.txt file.
It does not mean "do not follow this link"; rather, it means "do not add weight from this link".
comment:4 by , 16 years ago
| Cc: | added |
|---|
On the subject of robots, I followed a bot grabbing one of my projects today, and it kept hitting 'Previous period' in timeline back to 1970 until it got an error due to invalid date. The problem with this link, and many of the old version links is that they only show in query string - like ?version=22 - and it is impossible to keep them from bots using just HTML directives and `robots.txt.
Good ideas are needed to make progress on this ticket.
comment:5 by , 15 years ago
| Keywords: | robots added |
|---|---|
| Milestone: | next-minor-0.12.x → unscheduled |
If someone thinks he has a good idea on this topic, PatchWelcomed, but keep in mind #6124.
comment:6 by , 14 years ago
| Component: | general → version control/browser |
|---|---|
| Keywords: | noindex added |
| Milestone: | unscheduled → 0.13 |
| Owner: | changed from to |
We should at least do the same as for older wiki pages (#478), i.e. don't allow indexing source pages obtained via an explicit ?rev.
comment:7 by , 14 years ago
| Milestone: | 0.13 → next-major-0.1X |
|---|
comment:8 by , 11 years ago
#11251 was closed as a duplicate. It suggested not generating the all links if the user agent is a robot.
See also #9029, which forbids robots to index Wiki history pages and edited pages.
Also comment:11:ticket:4022 suggests it may not (anymore?) be impossible to disallow certain query parameters in robots.txt.
comment:9 by , 10 years ago
| Owner: | removed |
|---|
better: all "download in other format" links, not just rss.