Page MenuHomePhabricator
Create Task
Maniphest T184485

Stop logging autopatrol actions
Closed, ResolvedPublic
Actions

Assigned To
Ladsgroup
Authored By
kaldari
Jan 8 2018, 10:55 PM
Tags
Referenced Files
None
Subscribers
Aklapper
ArielGlenn
Bawolff
• brooke
• Catrope
Ciencia_Al_Poder
daniel
View All 27 Subscribers
Tokens
"Yellow Medal" token, awarded by kaldari."Piece of Eight" token, awarded by RandomDSdevel."Like" token, awarded by Liuxinyu970226."Like" token, awarded by jcrespo.

Description

Problem statement

  • The majority of log entries for most wikis (especially RCPatrol wikis) are autopatrol actions. (Note that before April 2016, these were not distinct from explicit patrol actions.) There doesn't appear to be much value in logging these.
    • The data is almost identical to the revision row, except with some substituted localisation messages.
    • The only unique data in the logging row is the (boolean) fact that it was an autopatrolled revision.

Solution

  • Stop logging autopatrol actions (and possibly purge all the previous autopatrol actions).
  • Store the fact that it was autopatrolled in the rc_patrolled field instead (currently boolean tinyint, could add a third state for autopatrolled).

The proposed expansion of rc_patrolled is logically similar to the previous expansion we did of the patrol log type (from patrol/patrol, to also differentiating patrol/autopatrol).

Original task description by @kaldari

It seems that the majority of log entries for most wikis (especially RCPatrol wikis) are autopatrol actions. (Note that these were not distinct from patrol actions prior to April 2016.) I don't see any actual reason why we want to log autopatrol actions. The data is basically useless after 30 days and we have an rc_patrolled flag in recentchanges that is more useful.

Considering that some logging tables are approaching 1 billion records (see T12331#3876482) and that this is (soft-)blocking deployment of new, more useful, logging types, I think we should just stop logging autopatrol actions entirely (and possibly purge all the previous autopatrol actions while we're at it).

Is there any reason we need this data in the logging table?

See also

Details

SubjectRepoBranchLines +/-
Enable deleteAutoPatrolLogs to delete pre-2011 autopatrol logs as wellmediawiki/coremaster+52 -10
Clean up old config for logging autopatrol actionsoperations/mediawiki-configmaster+0 -5
Remove the feature switch for disabling autopatrol loggingmediawiki/coremaster+3 -25
Stop logging autopatrol actions everywhereoperations/mediawiki-configmaster+1 -3
Disable logging autopatrol actins in commonswikioperations/mediawiki-configmaster+1 -0
Change default of $wgLogAutopatrol from true to falsemediawiki/coremaster+7 -1
Stop logging autopatrol actions in wikidatawikioperations/mediawiki-configmaster+6 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Stashbot added a comment.Apr 4 2018, 9:25 AM

Mentioned in SAL (#wikimedia-operations) [2018-04-04T09:25:17Z] <Amir1> end of the deleteAutoPatrolLogs.php script on mediawikiwiki (T184485)

gerritbot added a comment.Apr 5 2018, 12:46 PM

Change 424293 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Stop logging autopatrol actions in wikidatawiki

https://gerrit.wikimedia.org/r/424293

gerritbot added a comment.Apr 5 2018, 1:07 PM

Change 424293 merged by jenkins-bot:
[operations/mediawiki-config@master] Stop logging autopatrol actions in wikidatawiki

https://gerrit.wikimedia.org/r/424293

Stashbot added a comment.Apr 5 2018, 1:09 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-05T13:09:54Z] <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:424293|Stop logging autopatrol actions in wikidatawiki (T184485)]] (duration: 01m 16s)

gerritbot added a comment.Apr 5 2018, 1:45 PM

Change 403115 merged by jenkins-bot:
[mediawiki/core@master] Change default of $wgLogAutopatrol from true to false

https://gerrit.wikimedia.org/r/403115

ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)).Apr 5 2018, 2:00 PM
gerritbot added a comment.Apr 5 2018, 5:21 PM

Change 424362 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Disable logging autopatrol actins in commonswiki

https://gerrit.wikimedia.org/r/424362

Stashbot added a comment.Apr 5 2018, 5:33 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-05T17:33:26Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --before 20180223210426 (T184485)

Stashbot added a comment.Apr 5 2018, 5:56 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-05T17:56:45Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --before 20180223210426 --from-id 156008475 (T184485)

gerritbot added a comment.Apr 5 2018, 6:06 PM

Change 424362 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable logging autopatrol actins in commonswiki

https://gerrit.wikimedia.org/r/424362

Stashbot added a comment.Apr 5 2018, 6:08 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-05T18:08:30Z] <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Disable logging autopatrol actions on commonswiki (T184485) (duration: 01m 17s)

Stashbot added a comment.Apr 6 2018, 11:50 AM

Mentioned in SAL (#wikimedia-operations) [2018-04-06T11:50:10Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=fawiki --before 20180223210426 --sleep 2 (T184485)

Stashbot added a comment.Apr 9 2018, 6:57 AM

Mentioned in SAL (#wikimedia-operations) [2018-04-09T06:57:59Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=zhwiktionary --check-old --before 20180223210426 --sleep 2 (T184485)

Krinkle moved this task from In progress to Implemented on the TechCom-RFC (TechCom-RFC-Closed) board.Apr 11 2018, 6:44 PM
gerritbot added a comment.Apr 11 2018, 10:43 PM

Change 425719 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Stop logging autopatrol actions everywhere

https://gerrit.wikimedia.org/r/425719

kaldari added a comment.Apr 11 2018, 10:55 PM

The server kitties will be so happy!
Macro server_kitty:

gerritbot added a comment.Apr 11 2018, 11:03 PM

Change 425719 merged by jenkins-bot:
[operations/mediawiki-config@master] Stop logging autopatrol actions everywhere

https://gerrit.wikimedia.org/r/425719

Stashbot added a comment.Apr 11 2018, 11:09 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-11T23:09:53Z] <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: Stop logging autopatrol actions everywhere (T184485) (duration: 01m 18s)

EddieGP subscribed.Apr 12 2018, 9:35 AM

T192063: API misbehaves on FlaggedRevs wikis after logging autopatrol actions was disabled

Zache subscribed.Apr 12 2018, 10:19 AM

Btw. By disabling the logging we lose the ability do stats/graphs how much of the edits were patrolled/autopatrolled/not patrolled if needed outside of the 30 days recentchanges table window.

Ladsgroup added a comment.Apr 12 2018, 11:59 AM
In T184485#4126426, @Zache wrote:

Btw. By disabling the logging we lose the ability do stats/graphs how much of the edits were patrolled/autopatrolled/not patrolled if needed outside of the 30 days recentchanges table window.

Graph-wise it will be hard but stats-wise, you can just get list of autopatrolled users and consider their actions as patrolled. This is pretty straightforward thing to do

Lucas_Werkmeister_WMDE subscribed.Apr 12 2018, 12:58 PM
gerritbot added a comment.Apr 12 2018, 7:05 PM

Change 425880 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@master] Drop code for the feature switch of logging autopatrol actions

https://gerrit.wikimedia.org/r/425880

WMDE-leszek added a project: Wikidata-Ministry-Of-Magic-Tech-Debt.Apr 13 2018, 10:27 AM
gerritbot added a comment.Apr 13 2018, 9:27 PM

Change 425880 merged by jenkins-bot:
[mediawiki/core@master] Remove the feature switch for disabling autopatrol logging

https://gerrit.wikimedia.org/r/425880

ReleaseTaggerBot edited projects, added MW-1.31-release-notes (WMF-deploy-2018-04-17 (1.31.0-wmf.30)); removed MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)).Apr 13 2018, 10:00 PM
Lucas_Werkmeister_WMDE moved this task from User Stories to In Progress on the Wikidata-Ministry-Of-Magic board.Apr 17 2018, 2:39 PM
Stashbot added a comment.Apr 18 2018, 9:46 AM

Mentioned in SAL (#wikimedia-operations) [2018-04-18T09:46:21Z] <Amir1> start of deleting auto patrol actions in small wikis (T184485)

GTrang subscribed.Apr 19 2018, 10:49 PM

@Ladsgroup The next patch you might want to upload is to request that the following lines (110 to 113) be completely removed from the InitialiseSettings.php file.

# T184485
'wgLogAutopatrol' => [
	'default' => false,
],

After removing the wgLogAutopatrol variable, the above lines are not needed any more and should be completely removed.

Ladsgroup added a comment.Apr 20 2018, 2:34 PM

This is on my radar. Will get to it once the changes hit production.

EddieGP unsubscribed.Apr 20 2018, 2:50 PM
gerritbot added a comment.Apr 24 2018, 12:53 PM

Change 428624 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Clean up old config for logging autopatrol actions

https://gerrit.wikimedia.org/r/428624

gerritbot added a comment.Apr 24 2018, 1:34 PM

Change 428624 merged by jenkins-bot:
[operations/mediawiki-config@master] Clean up old config for logging autopatrol actions

https://gerrit.wikimedia.org/r/428624

Stashbot added a comment.Apr 24 2018, 1:37 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-24T13:37:53Z] <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:428624|Clean up old config for logging autopatrol actions (T184485)]] (duration: 00m 58s)

He7d3r subscribed.May 8 2018, 8:01 PM

Is this task related to the fact that no autopatrol actions are shown on ptwiki anymore? This list is empty (and takes forever to load):
https://pt.wikipedia.org/wiki/Special:Log/patrol?subtype=autopatrol

Ladsgroup added a comment.May 8 2018, 8:04 PM

Yes, We have been removing autopatrol actions (at least anything after September 2011) as well.

MGChecker added a comment.May 8 2018, 9:01 PM

I think this subtype should be deactivated to prevent this query to be run. It is of no use at all now.

Ladsgroup added a comment.May 8 2018, 9:03 PM

Yes, I make a phabricator ticket and make it happen ASAP.

Ladsgroup mentioned this in T194213: Disable autopatrol subtype in Special:Log.May 8 2018, 9:04 PM
GTrang mentioned this in T136493: Earlier autopatrolled revisions should go under "Automatic patrol".May 10 2018, 9:49 PM
GTrang added a comment.May 12 2018, 2:19 AM

All automatic patrols on Wikipedia since 1 March 2012 (when Wikipedia started using MediaWiki 1.19) are now gone. Next, we should make the deleteAutopatrolLogs.php script also detect and get rid of pre-1.19 automatic patrols as well. Then, we could start running the script on wikis that existed prior to 2012 to completely get rid of all very old automatic patrols.

Ladsgroup added a comment.May 12 2018, 6:59 PM

I have already deleted 1.1B rows from the database and deleting those rows that at the most will be about 10% of what we have accomplished by now will gain us next to none. I can't put too much time on it unfortunately. I will try to poke it when I have some free time but it's not high priority.

Ciencia_Al_Poder subscribed.May 13 2018, 5:29 PM
gerritbot added a comment.May 18 2018, 1:47 PM

Change 433714 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@master] Enable deleteAutoPatrolLogs to delete pre-2011 autopatrol logs as well

https://gerrit.wikimedia.org/r/433714

Ladsgroup closed subtask T189596: Run deleteAutopatrolLogs script for Wikidata (WMF) as Resolved.May 19 2018, 12:10 PM
Lydia_Pintscher closed subtask T194213: Disable autopatrol subtype in Special:Log as Resolved.May 23 2018, 9:51 AM
gerritbot added a comment.May 23 2018, 11:48 PM

Change 433714 merged by jenkins-bot:
[mediawiki/core@master] Enable deleteAutoPatrolLogs to delete pre-2011 autopatrol logs as well

https://gerrit.wikimedia.org/r/433714

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-05-29 (1.32.0-wmf.6)).May 24 2018, 12:01 AM
Nirmos added a comment.May 24 2018, 12:27 AM
This comment was removed by Nirmos.
Anomie closed subtask T57377: API: Implement rctitles parameter in query=recentchanges as Resolved.May 24 2018, 2:42 PM
Ladsgroup closed this task as Resolved.May 28 2018, 9:24 AM
Ladsgroup claimed this task.
Ladsgroup moved this task from In Progress to Done on the Wikidata-Ministry-Of-Magic board.

It is officially done now 🎉🎉🎉🎉

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptMay 28 2018, 9:25 AM
daniel added a comment.May 28 2018, 9:59 AM

wooot!

Am 28.05.2018 um 11:26 schrieb Ladsgroup:

Ladsgroup closed this task as "Resolved".
Ladsgroup claimed this task.
Ladsgroup added a comment.
Herald added a project: User-Ladsgroup.

It is officially done now 🎉🎉🎉🎉

*TASK DETAIL*
https://phabricator.wikimedia.org/T184485

*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

*To: *Ladsgroup
*Cc: *Ciencia_Al_Poder, He7d3r, GeoffreyT2000, Lucas_Werkmeister_WMDE, Zache,
Stashbot, Liuxinyu970226, daniel, Nirmos, brion, Tgr, Nikerabbit, Ladsgroup,
Framawiki, ArielGlenn, MGChecker, MZMcBride, Nemo_bis, WMDE-leszek, Catrope,
gerritbot, MaxSem, Bawolff, Krinkle, jcrespo, Aklapper, kaldari, Versusxo,
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Baloch007, Darkminds3113,
Bsandipan, Lordiis, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99,
Maathavan, Johan, SBisson, Luke081515, GWicke, jayvdb, fbstj, Lydia_Pintscher,
santhosh, TheDJ, Jdforrester-WMF, Rxy, Jay8g, Ltrlg, bd808, Legoktm

kaldari awarded a token.May 28 2018, 1:18 PM
Liuxinyu970226 unsubscribed.Jun 3 2018, 6:23 AM
GTrang mentioned this in T196743: Clean up automatic patrol log entries from MW 1.18 and earlier on Wikimedia wikis.Jun 8 2018, 2:28 PM
Halfak mentioned this in T196547: [Epic] Extension:JADE scalability concerns.Jun 13 2018, 5:02 PM
• 238482n375 removed Ladsgroup as the assignee of this task.Jun 15 2018, 8:02 AM
• 238482n375 lowered the priority of this task from High to Lowest.
• 238482n375 moved this task from Next Up to In Code Review on the Analytics-Kanban board.
• 238482n375 added projects: Analytics-Kanban, acl*security, Wikimedia-VE-Campaigns (S2-2018), Scap (Scap3-Adoption-Phase2), AbuseFilter, Data-release, Hashtags, LabsDB-Auditor, Ladies-That-FOSS-MediaWiki, Language-2018-Apr-June, Language-2018-Jan-Mar, HHVM, HAWelcome.
• 238482n375 edited subscribers, added: • 238482n375; removed: Aklapper.
This comment was removed by Vgutierrez.
• 238482n375 set Security to Software security bug.Jun 15 2018, 8:06 AM
• 238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".
This comment was removed by Vgutierrez.
Lucas_Werkmeister_WMDE assigned this task to Ladsgroup.Jun 15 2018, 10:15 AM
Lucas_Werkmeister_WMDE raised the priority of this task from Lowest to High.
Lucas_Werkmeister_WMDE removed projects: HAWelcome, HHVM, Language-2018-Jan-Mar, Language-2018-Apr-June, Ladies-That-FOSS-MediaWiki, LabsDB-Auditor, Hashtags, Data-release, AbuseFilter, Scap (Scap3-Adoption-Phase2), Wikimedia-VE-Campaigns (S2-2018), acl*security, Analytics-Kanban.
Lucas_Werkmeister_WMDE edited subscribers, added: Aklapper; removed: • 238482n375.
Restricted Application added a project: acl*security. · View Herald TranscriptJun 15 2018, 10:15 AM
Lucas_Werkmeister_WMDE mentioned this in T197456: Remove security vandalism of various tasks (2018-06-15).Jun 15 2018, 10:16 AM
Vgutierrez removed a project: acl*security.Jun 15 2018, 10:26 AM
Vgutierrez changed the visibility from "Custom Policy" to "Public (No Login Required)".
Vgutierrez subscribed.
Vgutierrez unsubscribed.
jcrespo mentioned this in T197459: Optimize logging table.Jun 15 2018, 10:39 AM
• Vvjjkkii reopened subtask T194213: Disable autopatrol subtype in Special:Log as Open.Jul 1 2018, 1:11 AM
CommunityTechBot closed subtask T194213: Disable autopatrol subtype in Special:Log as Resolved.Jul 2 2018, 1:51 PM
GTrang mentioned this in T198809: Introduce "unpatrol" feature.Jul 4 2018, 6:18 PM
GTrang mentioned this in T199755: Clean up autopatrol-related code related to old log entries, such as in PatrolLog and PatrolLogFormatter.Jul 16 2018, 10:20 PM
ArielGlenn moved this task from Watching now to Done on the User-ArielGlenn board.Aug 7 2018, 10:00 AM
Quiddity mentioned this in T215922: Community input for patrol log reviews in 'all public logs' .Feb 16 2019, 8:36 PM
Ladsgroup mentioned this in T233625: Change PK and remove partitions from the logging table.Sep 27 2019, 12:25 PM
Maintenance_bot removed a project: Patch-For-Review.Sep 27 2019, 1:11 PM
DannyS712 mentioned this in T243642: Do patrol log entries still need the `auto` parameter saved.Jan 24 2020, 10:57 PM
Maintenance_bot moved this task from Incoming to Done on the User-Ladsgroup board.Jan 24 2020, 11:15 PM
Jony subscribed.Jun 30 2020, 10:57 AM
ArielGlenn mentioned this in T258108: Ignore auto-watchlist preferences for bots.Oct 26 2020, 4:54 PM
Krinkle mentioned this in T149886: Mediawiki 1.27.1 bug: all patrolled edits turned off in LocalSettings.php and it keeps creating patrol logs..May 11 2022, 6:04 PM
Krinkle mentioned this in T13030: Special:Patrollist.May 11 2022, 6:30 PM
Ladsgroup edited projects, added User-notice-archive; removed User-notice.Aug 13 2022, 1:53 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits