Repository navigation
feat(findall): show cost and enrichments before a run, add --no-enrich and preview - #166
Open
vidiyala99 wants to merge 1 commit into
Open
vidiyala99 wants to merge 1 commit into
vidiyala99 wants to merge 1 commit into
Conversation
…h and preview A default `findall run` (core, -n 10) can bill more than the published formula because it applies whatever enrichments ingest suggests, one Task run per match, without saying so. Ingest is not deterministic: the same objective returned 0, 1, then 0 suggested enrichments, so `--dry-run` can't promise what the real run does. - Print the enrichments a run will add, and an estimated max cost, as soon as ingest returns (and in --dry-run, text and JSON). - Add --no-enrich to skip suggested enrichments. - Offer the preview generator on `findall run` only (5-10 candidates, $0.10). It stays out of FINDALL_GENERATORS since parallel-web#58 found it doesn't support enrichment reliably, and enrichment is always skipped for it. - Fix dry-run listing enrichments as "- : " (it read name/description keys that ingest enrichments don't have). - Add estimate_findall_cost() and summarize_enrichments() to core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vidiyala99
force-pushed
the
findall-cost-visibility
branch
from
November 10, 2026 17:06
1d1d126 to
dd880bf
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I ran my first
parallel-cli findall runwith the defaults (core,-n 10) on a new account. The published formula says that's at most $2.00 + 10 x $0.15 = $3.50. The run was billed $3.75, and the Usage page showed "Task API v1: 11" on a day I never called the Task API.The difference is enrichments.
run_findallapplies whatever enrichments ingest suggests (enrich=True), each one billed per match at its Task processor rate. For my objective ingest added afounding_dateenrichment oncore, so 10 x $0.025 = $0.25. The CLI never mentions this, and there's no way to turn it off.--dry-rundoesn't help either, because ingest runs again when the real run starts and isn't deterministic: the same objective came back with 0, 1, then 0 suggested enrichments across three calls.At
-n 10it's a quarter. At-n 1000with a couple ofcoreenrichments it's an extra $50 nobody asked for.What this changes, all in
findall run:Est. cost: up to $3.75 ($2.00 + 10 matches x $0.175). The same estimate is in--dry-run, in both text and--json(estimated_cost,apply_enrichments).--no-enrichskips suggested enrichments.-g previewis back forfindall run. The docs say to "Always run withgenerator: "preview"first", but the CLI only acceptedbase|core|pro. Preview was removed in feat: add FindAll to DuckDB integration + new CLI commands #58 because it doesn't support enrichment reliably, so it stays out ofFINDALL_GENERATORS(the DuckDB paths are unchanged), enrichment is always skipped for it, and--match-limitmust be 5-10.--dry-runprinting enrichments as- :. It readname/descriptionkeys that ingest enrichments don't have; it now lists the fields and processor.The prices come from https://docs.parallel.ai/getting-started/pricing and live in
FINDALL_GENERATOR_PRICINGandTASK_PROCESSOR_PRICE_PER_RUNincore/findall.py, withestimate_findall_cost()andsummarize_enrichments()alongside. An unknown generator or enrichment processor makes the estimate say "unknown" rather than guess. I know hardcoded prices can drift, so if there's an API or a better source for these I'm happy to switch to it. That would also help with #158 and parallel-sdk-python#17.Live run with this branch:
The same objective on the defaults took 5m20s and $3.75.
Testing: 25 new tests in
tests/test_findall.py(cost estimates including the $3.75 case above,run_findallwithenrich=Falseandpreview, and the CLI flag, dry-run and validation paths).ruff check,ruff format --check,tach checkandty checkare clean. Fullpytest: 864 passed, 14 failed, and the same 14 fail onmainon my Windows machine (Unix permission, symlink and DuckDB event loop tests), so they're unrelated.I used AI assistance while working on this and reviewed and tested the change myself.
🤖 Generated with Claude Code