Need
Automoderator has been deployed on the the first pilot wikis, with plans to rollout to couple of other wikis soon. At the moment we're tracking the pilot metrics manually with a weekly run. Eventually, a publicly accessible dashboard as communities would also be interested in AM's activity is needed, that is updated at least daily.
Solution
- A public Superset dashboard to monitor activity of Automoderator, both by the team and the community as required.
- The dashboard will live at https://superset.wmcloud.org/
- Addition of new wikis, which mainly a work on the data pipeline side should be easy.
Steps involved
| # | Step | Task | Notes |
|---|---|---|---|
| 1 | Identify all the metrics (+dependent data sources) to be tracked with v1 of the dashboard | T371981 | Pilot metrics are in T362610 (weekly reports) |
| - | Make an exhaustive list of available metrics for future consideration | ✅ | |
| 2 | mockup based on testwiki (iterations can be done after the data is available) | ||
| 3 | Develop Airflow-based pipelines for the necessary metrics to be calculated | T374705 | sub-tasks: T375062, T375153 |
| 4 | Privacy review of the data to be published (L3SC request as it would be Medium Risk according to guidelines) | request ✅ | |
| 5 | - | this will not be a separate job | |
| 6 | Jobs to read and update Toolforge ToolsDB from the published datasets | T374705 | |
| 7 | Develop the queries required for the dashboard on Superset | ✅ | |
| 8 | Build and publish the first version of the dashboard & communicate | ✅ | |
Note: This is a goal for FY 24 Q1, so this should be ready by the end of quarter.
- Automoderator ToolsDB database: s56213__unified_automod_metrics_p
- Web publication: https://analytics.wikimedia.org/published/datasets/automoderator/