ReviewLoop

A production-minded Rust CLI/daemon for paperreview.ai submission and review retrieval.

Most paper review automation breaks in boring ways: duplicate submissions, lost tokens, noisy polling, and zero traceability.

ReviewLoop gives you a durable loop with guardrails:

• Queue reviews from Git tags or PDF hash changes
• Persist every transition in SQLite
• Pull tokens from Gmail OAuth or IMAP
• Write reproducible artifacts (review.json, review.md, meta.json)
• Recover from failures with explicit retries and fallback submission

Why This Project Exists

Reviewing pipelines are usually a pile of scripts plus cron plus hope. ReviewLoop is built for the opposite:

• predictable state transitions
• low default provider pressure
• human approval gates where it matters
• clear local evidence of what happened and why

If you want reliable, low-drama automation for paperreview.ai, this is the tool.

1-Minute Quick Start

# 1) install (choose one)
# after public release
brew tap acture/ac && brew install reviewloop
# OR
cargo install reviewloop

# 2) initialize global paths + current repo project config
reviewloop init
reviewloop init project --project-id main

# 3) register paper
reviewloop paper add \
  --paper-id main \
  --path paper/main.pdf \
  --backend stanford

# 4) optional: add a custom git-tag trigger for this paper
reviewloop paper add \
  --paper-id camera_ready \
  --path build/camera_ready.pdf \
  --backend stanford \
  --tag-trigger "custom-review/camera_ready/*"

# 5) submit and run daemon
# `paper add` will prompt whether to submit immediately
reviewloop daemon install --start true

Installation

Homebrew (recommended on macOS)

# after public release
brew tap acture/ac
brew install reviewloop

Upgrade:

reviewloop self-update --yes
# or force Homebrew path
reviewloop self-update --method brew --yes

Cargo

# after public release
cargo install reviewloop

Upgrade:

reviewloop self-update --yes
# or force Cargo path
reviewloop self-update --method cargo --yes

Build From Source

git clone https://github.com/Acture/reviewloop.git
cd reviewloop
cargo build --release
./target/release/reviewloop --help

Command Surface

Global usage:

reviewloop [--config /path/to/override.toml] <command>

Core commands:

reviewloop init
reviewloop init project --project-id <id> [--project-root <path>] [--force]
reviewloop paper add --paper-id <id> --path <pdf-or-build-artifact> --backend <backend> [--watch true|false] [--tag-trigger "<pattern>"] [--submit-now] [--no-submit-prompt]
reviewloop paper watch --paper-id <id> --enabled <true|false>
reviewloop paper remove --paper-id <id> [--purge-history]
reviewloop daemon run
reviewloop daemon run --panel false
reviewloop daemon install [--start true]
reviewloop daemon uninstall
reviewloop daemon status
reviewloop submit --paper-id main [--force]
reviewloop approve --job-id <job-id>
reviewloop import-token --paper-id main --token <token> [--source email]
reviewloop check [--job-id <job-id> | --paper-id <paper-id>] [--all-processing]
reviewloop status [--paper-id main] [--json] [--show-token]
reviewloop retry --job-id <job-id> [--override-rate-limit]
reviewloop complete --job-id <job-id> [--summary-text <text> | --summary-url <url> | --empty-summary] [--score <value>]
reviewloop config init
reviewloop config init project --project-id <id> [--project-root <path>] [--force]
reviewloop config migrate-project --project-id <id> [--project-root <path>]
reviewloop email login --provider google
reviewloop email status
reviewloop email switch --account <account-id-or-email>
reviewloop email logout [--account <account-id-or-email>]
reviewloop self-update [--method auto|brew|cargo] [--yes] [--dry-run]

self-update only replaces the executable. It does not delete:

• global config (~/.config/reviewloop/config.toml)
• global data directory (database, artifacts, logs)
• project-local configs

Runtime Model

Daemon tick interval: every 30 seconds.

Each tick performs:

1. Trigger scan (git tags, PDF hash changes)
2. Optional Gmail OAuth + IMAP token ingestion
3. Timeout marking
4. Submission processing (QUEUED -> SUBMITTED/PROCESSING)
5. Poll processing (PROCESSING -> COMPLETED/FAILED/...)

Manual immediate poll:

• reviewloop check --job-id <id> forces one check now for that processing job (ignores next_poll_at)
• reviewloop check --paper-id <paper-id> checks the latest processing job for that paper
• reviewloop check --all-processing checks all current processing jobs

Output artifacts per completed job:

• <state_dir>/artifacts/<job-id>/review.json
• <state_dir>/artifacts/<job-id>/review.md
• <state_dir>/artifacts/<job-id>/meta.json

What Makes It Reliable

• State machine, not ad-hoc scripts: jobs move through explicit statuses (PENDING_APPROVAL, QUEUED, PROCESSING, COMPLETED, etc.)
• Duplicate guard: prevents repeated submissions for the same project_id + paper_id + backend + pdf_hash + version_key
• Load-aware polling: default schedule starts at 10 minutes with jitter/cooldown behavior
• Recovery built in: every transition is evented, retries are explicit
• Fallback path: optional Node + Playwright submit path when provider API flow fails

Triggering Modes

Git tag trigger

Supported patterns:

• review-<backend>/<paper-id>/<anything>
• review-<backend>/<anything> (uses the first configured paper of that backend)
• optional per-paper custom pattern via paper add --tag-trigger "<pattern>" (supports *)

Example:

review-stanford/main/v1

PDF change trigger

• Computes SHA256 for configured PDFs
• New hash enqueues job
• Default status is PENDING_APPROVAL (manual approve required)

Email Token Ingestion

ReviewLoop can attach review tokens from email to open jobs.

IMAP mode (built in)

Default token pattern includes Stanford:

[imap.backend_patterns]
stanford = "https?://paperreview\\.ai/review\\?token=([A-Za-z0-9_-]+)"

Recommended defaults:

• imap.header_first = true to scan headers first
• imap.max_lookback_hours = 72
• imap.max_messages_per_poll = 50

Gmail OAuth mode

Configure:

[gmail_oauth]
enabled = true
client_id = "your-google-oauth-client-id"
client_secret = "your-google-oauth-client-secret"
token_store_path = "~/.review_loop/oauth/google_token.json" # optional
poll_seconds = 300
mark_seen = true
max_lookback_hours = 72
max_messages_per_poll = 50
header_first = true

[gmail_oauth.backend_header_patterns]
stanford = "(?is)(from:\\s*.*mail\\.paperreview\\.ai|subject:\\s*.*paper review is ready)"

[gmail_oauth.backend_patterns]
stanford = "https?://paperreview\\.ai/review\\?token=([A-Za-z0-9_-]+)"

You can also provide credentials via environment variables:

• REVIEWLOOP_GMAIL_CLIENT_ID
• REVIEWLOOP_GMAIL_CLIENT_SECRET

For official CI-built binaries, these same variable names can be injected at compile time via GitHub Actions secrets.*; runtime env/config can still override them.

Then login:

reviewloop email login --provider google

email login will try to open your default browser automatically and wait in CLI for OAuth completion.

ReviewLoop runs Gmail API polling first when available, then IMAP fallback.

Configuration Highlights

ReviewLoop uses two config files with separate responsibilities:

• global config: $XDG_CONFIG_HOME/reviewloop/config.toml or ~/.config/reviewloop/config.toml
• project config: <repo-root>/reviewloop.toml

There is no global-overrides-project merge chain. Instead:

• global config owns machine/user concerns such as core.*, logging.*, polling.*, retention.*, imap.*, gmail_oauth.*, and Stanford provider connection defaults
• project config owns repo concerns such as project_id, papers, paper_watch, paper_tag_triggers, trigger.*, and Stanford venue
• --config /path/to/reviewloop.toml explicitly points to a project config file
• reviewloop init initializes the global config/data paths
• reviewloop init project --project-id <id> initializes the current repo's project config
• reviewloop daemon install can run in global-only mode when no project config is present; if a project config is found, it binds the daemon to that project config

Project commands require a non-empty project_id in the project config. Jobs, events, dedupe, and status views are isolated inside the shared global DB by project_id.

Paper registration:

• start with an empty papers[]
• add papers through reviewloop paper add ...
• remove papers through reviewloop paper remove --paper-id ...
  • add --purge-history to also delete DB jobs/events/reviews and local artifacts for that paper
• control PDF watcher per paper with reviewloop paper watch ...

Safe defaults:

• core.max_concurrency = 2
• core.max_submissions_per_tick = 1
• core.state_dir = "~/.review_loop" (or REVIEWLOOP_STATE_DIR when set)
• core.db_path = "~/.review_loop/reviewloop.db" (or <REVIEWLOOP_STATE_DIR>/reviewloop.db)
• core.review_timeout_hours = 48
  • for stanford, timeout is linearly scaled by PDF page count up to 20 pages
• polling.schedule_minutes = [10, 20, 40, 60]
• polling.jitter_percent = 10
• retention.enabled = true
• retention.prune_every_ticks = 20 (10 minutes with 30s tick)
• retention.email_tokens_days = 30
• retention.seen_tags_days = 90
• retention.events_days = 30
• retention.terminal_jobs_days = 0 (disabled by default)
• trigger.pdf.auto_submit_on_change = false
• trigger.pdf.max_scan_papers = 10
• trigger.git.tag_pattern = "review-<backend>/<paper-id>/*"
• trigger.git.auto_create_tags_on_pdf_change = false
• trigger.git.auto_delete_processed_tags = false

providers.stanford defaults:

• base_url = "https://paperreview.ai"
• fallback_mode = "node_playwright"
• fallback_script = "tools/paperreview_fallback.mjs"
• email optional (falls back to active email account)
• venue = "ICLR" (project config)

Logging:

• logging.output = "stdout" | "stderr" | "file"
• file mode default path: <state_dir>/reviewloop.log

CI/CD and Release Flow

This repository ships with GitHub Actions for both quality gates and release automation.

CI (.github/workflows/ci.yml)

On pull requests and pushes to main/master:

• cargo fmt --all -- --check
• cargo clippy --all-targets -- -D warnings
• cargo test --all-targets --locked

Runs on both Ubuntu and macOS.

The same gate is shared locally via ./scripts/quality-gates.sh. To enable it in the standard pre-commit framework:

• pre-commit install
• commits will then run the repository-local reviewloop quality gates hook before creating the commit

Release (.github/workflows/release.yml)

On tag push like v0.1.0:

1. Verify tag version matches Cargo.toml
2. Run quality gates again
3. Publish crate to crates.io
4. Update Homebrew tap formula in Acture/homebrew-ac
5. Create GitHub Release with generated notes

Required secrets:

• CARGO_REGISTRY_TOKEN
• HOMEBREW_TAP_GITHUB_TOKEN

Optional secrets (compile-time OAuth defaults for release/CI builds):

• REVIEWLOOP_GMAIL_CLIENT_ID
• REVIEWLOOP_GMAIL_CLIENT_SECRET

Optional repo variables:

• HOMEBREW_TAP_REPO (default: Acture/homebrew-ac)
• HOMEBREW_FORMULA_PATH (default: Formula/reviewloop.rb)

Fallback Requirements

When API submit fails and fallback is enabled:

• Node.js must be available
• Playwright runtime dependencies must be installed
• script path defaults to tools/paperreview_fallback.mjs

Responsible Use

ReviewLoop is intentionally conservative.

Please keep it that way:

• use it only for authorized submissions/retrieval
• keep concurrency and submit rate low unless provider approves otherwise
• do not aggressively shorten poll cadence
• respect provider Terms of Service and fair-use boundaries

Current Scope

• Supported backend: stanford (paperreview.ai)
• Database: SQLite (global state path by default, supports :memory:)
• Interface: CLI + daemon

License

GPL-3.0