Block robots from ticket (comment) edit history

[anonymous]

To reduce server load caused by crawlers.
To disarm spam links in ticket comment history (that is, if spam comments have not been properly deleted).

Add a rel="nofollow" attribute to the link to the ticket description edit history. (The page it points to has <meta name="ROBOTS" content="NOINDEX, NOFOLLOW" /> already.)

Add a rel="nofollow" attributes to the links to ticket comment edit history.

Add <meta name="ROBOTS" content="NOINDEX, NOFOLLOW" /> to ticket comment edit history pages.

AFAIK, there are crawlers that won't access nofollow-links if there are no other references. The semantics however just says to not add weight to such links in ranking, which is also desired here.

[Jun Omae]

It seems that it makes sense.

The <meta name="ROBOTS" content="NOINDEX, NOFOLLOW" /> line is declared at source:/tags/trac-1.6/trac/templates/diff_view.html@:27​. The template is used in trac/ticket/web_ui.py:

$ git grep -n 'diff_view\.html' trac-1.6 -- '*.py'
trac-1.6:trac/ticket/web_ui.py:1047:        return 'diff_view.html', data
trac-1.6:trac/ticket/web_ui.py:1181:        return 'diff_view.html', data

trac/ticket/templates/ticket_box.html

@@ -163 +163 @@
         # if description_change:
         <a href="${href.ticket(ticket.id, action='diff',
                  version=description_change.cnum)}"
+           rel="nofollow"
            class="lastmod trac-diff"
            title="${description_change.date}">
           # trans author = authorinfo(description_change.author)

trac/ticket/templates/ticket_change.html

@@ -242 +242 @@
   #   endif
   #   if comment_version is greaterthan(0):
   (<a href="${href.ticket(ticket.id, action='comment-diff', cnum=cnum,
-                          version=comment_version)}">${
-    _("diff")}</a>)
+                          version=comment_version)}"
+      rel="nofollow">${_("diff")}</a>)
   #   endif
 </div>
 #   endif

However, crawlers can follow the diff links in <link rel="prev" /> and <link rel="next" />.

[Jun Omae]

Workaround is to add the following lines to your robots.txt:

Disallow: /ticket/*?action=*
Disallow: /ticket/*?cnum_hist=*
Disallow: /ticket/*?cversion=*
Disallow: /ticket/*?replyto=*
Disallow: /ticket/*?version=*

[Philippe Cloutier <chealer@…>]

Do you acknowledge that your ticket is about several issues? Please clarify which issues you report, ideally showing at least 1 site affected for each one. And please indicate which issue each one of your suggestions would address.

Do you acknowledge that none of your suggestions blocks robots?

If you are brought here by performance/load issues on Trac instances you manage, be aware that there are nowadays multiple spiders which unfortunately do not respect crawling directives (see AI web crawlers are destroying websites in their never-ending hunger for any and all content). AI crawlers also pose particular issues (see Cloudflare Content Signals).

This comment is from Philippe "Chealer" Cloutier. I am subscribing to this ticket, but notifications seem to be broken, and I struggle to display my full email address or link to my contact information from this comment. My email address is available on Kune ni povos’s contact page. All of my comments and contributions in this ticket are offered under the terms of CC0 1.0 (unless otherwise noted).