Add support for exists and notExists query types in Mongo

[ArnabChatterjee20k]

Query added

Query::exists,
Query::notExists

Adapter

Only mongodb

Summary by CodeRabbit

• New Features

  • Added exists() and notExists() query methods to check for field presence or absence in MongoDB queries.

• Tests

  • Added end-to-end tests for existence-based queries on schemaless collections.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)

• main
• 0.69.x

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough

Walkthrough

This PR introduces support for EXISTS and NOT_EXISTS query types across the database system, enabling field existence checks in MongoDB while explicitly rejecting them in SQL adapters. The changes extend the Query class with new constants and helper methods, update validators to recognize the new types, and implement adapter-specific logic for handling existence checks.

Changes

Cohort / File(s)	Summary
Core Query Types src/Database/Query.php	Added two new public constants TYPE_EXISTS and TYPE_NOT_EXISTS, plus corresponding static helper methods exists() and notExists() to instantiate queries of these types. Updated isMethod() to recognize the new types as valid observable methods.
Query Validation src/Database/Validator/Queries.php	Extended the query type-to-method mapping to include TYPE_EXISTS and TYPE_NOT_EXISTS alongside TYPE_VECTOR_EUCLIDEAN, all routing to METHOD_TYPE_FILTER.
Filter Validation src/Database/Validator/Query/Filter.php	Added special handling for EXISTS and NOT_EXISTS methods: skip value validation and validate only the attribute; expanded allowed methods for array attributes to include EXISTS/NOT_EXISTS.
MongoDB Adapter src/Database/Adapter/Mongo.php	Extended operator handling to map TYPE_EXISTS/TYPE_NOT_EXISTS to MongoDB's $exists operator; updated buildFilter() to apply existence-based filters with appropriate boolean values.
SQL Adapter src/Database/Adapter/SQL.php	Added explicit rejection of EXISTS and NOT_EXISTS query types by throwing DatabaseException, consistent with vector-based method handling.
End-to-End Tests tests/e2e/Adapter/Scopes/SchemalessTests.php	Added two new test methods: testSchemalessExists() and testSchemalessNotExists() validating existence checks on optional and non-existent fields in schemaless collections. ⚠️ Note: Duplicate test block detected—testSchemalessExists() appears to be defined twice.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Feat mongodb #721: Directly overlaps with EXISTS/NOT_EXISTS additions across Query, Validator, and Mongo adapter code paths, suggesting coordinated feature development.
• Feat mongo tmp #647: Modifies Mongo.php query filter and operator logic in overlapping sections; relevant for operator mapping consistency.
• Not query types implementation #652: Updates Query.php and Validator/Queries.php with new query method types and validation logic, similar pattern to this PR.

Suggested reviewers

• abnegate
• fogelito

Poem

🐰 Fields that exist, fields that don't,
MongoDB knows, SQL won't,
A query whispers: "Are you there?"
We check with care, we validate with flair!
Existence proved, logic refined—
Peace of mind, perfectly designed! ✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 76.92% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title accurately summarizes the main change: adding support for exists and notExists query types in MongoDB adapter. The title is concise, clear, and specific.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ArnabChatterjee20k]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/Database/Validator/Query/Filter.php (1)

80-99: EXISTS/NOT_EXISTS validation may bypass relationship guards and accepts stray values

The new handling for exists/notExists mostly makes sense, but there are two behavior changes worth tightening:

1. Virtual relationship guard is skipped for EXISTS/NOT_EXISTS

• isValidAttributeAndValues() now returns early for TYPE_EXISTS / TYPE_NOT_EXISTS after isValidAttribute($attribute).
• That means the later block that rejects queries on virtual relationship attributes (child side of 1‑1, certain 1‑N / N‑1, and M‑M) is never reached for these methods.
• Previously, all filter methods (including null checks) respected those guards. Allowing exists('virtualRelation') while other filters are forbidden is likely inconsistent with the “cannot query virtual relationship attribute” rule.

Consider removing the early return and instead:

• Let isValidAttributeAndValues() run as usual for EXISTS/NOT_EXISTS, but
• Skip only the per‑value type validation, e.g. by short‑circuiting the foreach ($values as $value) block when the method is EXISTS/NOT_EXISTS.

2. EXISTS/NOT_EXISTS currently allow non‑empty values arrays

• isValid() routes EXISTS/NOT_EXISTS through isValidAttributeAndValues() but does not assert that values is empty.
• If someone sends raw JSON like {"method":"exists","attribute":"x","values":[1,2]}, it will be silently accepted even though the values are ignored.

You may want to explicitly enforce that no values are supplied, e.g.:

• In isValid() branch for TYPE_EXISTS / TYPE_NOT_EXISTS, return false if count($value->getValues()) !== 0, then call isValidAttributeAndValues().

This preserves your new allowance for array attributes while keeping behavior consistent for relationship attributes and catching malformed queries.

Also applies to: 215-263, 359-364

🧹 Nitpick comments (3)

src/Database/Adapter/SQL.php (1)

1773-1804: Explicitly rejecting EXISTS/NOT_EXISTS in SQL looks consistent

Mapping Query::TYPE_EXISTS / Query::TYPE_NOT_EXISTS to a DatabaseException in getSQLOperator() mirrors how vector queries are rejected and makes the limitation explicit at the adapter boundary. If you ever plan per‑adapter support, you might want a more specific message (e.g. include the actual $method), but as‑is this is clear and safe.

src/Database/Adapter/Mongo.php (1)

28-46: Mongo $exists wiring for exists/notExists queries is coherent

• Adding '$exists' to $this->operators and mapping TYPE_EXISTS / TYPE_NOT_EXISTS to '$exists' in getQueryOperator() plus true/false in buildFilter() correctly yields filters of the form ['field' => ['$exists' => true|false]].
• Including '$exists' in the exclusion list for replaceInternalIdsKeys() avoids mangling the operator name when rewriting $‑prefixed attribute keys.
• The dedicated $operator === '$exists' branch in buildFilter() currently does the same as the final else and is redundant, but harmless if you prefer to keep it for clarity.

Also applies to: 2371-2385, 2440-2444, 2457-2483

tests/e2e/Adapter/Scopes/SchemalessTests.php (1)

1159-1219: Consider adding a combination query test for consistency.

The testSchemalessNotExists method includes a combination test with both exists and notExists queries (lines 1277-1281). Consider adding a similar combination test here for symmetry and completeness, such as combining Query::exists('optionalField') with Query::equal('name', ['doc1']).

🔎 Example combination test

Add before line 1218:

+        // Test combination of exists with other query
+        $documents = $database->find($colName, [
+            Query::exists('optionalField'),
+            Query::equal('name', ['doc1']),
+        ]);
+        $this->assertEquals(1, count($documents)); // Only doc1
+        $this->assertEquals('doc1', $documents[0]->getId());
+
         $database->deleteCollection($colName);

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c8c1b2f and 5e0008e.

📒 Files selected for processing (6)

• src/Database/Adapter/Mongo.php
• src/Database/Adapter/SQL.php
• src/Database/Query.php
• src/Database/Validator/Queries.php
• src/Database/Validator/Query/Filter.php
• tests/e2e/Adapter/Scopes/SchemalessTests.php

🧰 Additional context used

🧠 Learnings (3)

📓 Common learnings

Learnt from: ArnabChatterjee20k
Repo: utopia-php/database PR: 747
File: src/Database/Adapter/Mongo.php:1449-1453
Timestamp: 2025-10-29T12:27:57.071Z
Learning: In src/Database/Adapter/Mongo.php, when getSupportForAttributes() returns false (schemaless mode), the updateDocument method intentionally uses a raw document without $set operator for replacement-style updates, as confirmed by the repository maintainer ArnabChatterjee20k.

📚 Learning: 2025-10-03T02:04:17.803Z

Learnt from: abnegate
Repo: utopia-php/database PR: 721
File: tests/e2e/Adapter/Scopes/DocumentTests.php:6418-6439
Timestamp: 2025-10-03T02:04:17.803Z
Learning: In tests/e2e/Adapter/Scopes/DocumentTests::testSchemalessDocumentInvalidInteralAttributeValidation (PHP), when the adapter reports getSupportForAttributes() === false (schemaless), the test should not expect exceptions from createDocuments for “invalid” internal attributes; remove try/catch and ensure the test passes without exceptions, keeping at least one assertion.

Applied to files:

• src/Database/Validator/Query/Filter.php
• tests/e2e/Adapter/Scopes/SchemalessTests.php

📚 Learning: 2025-10-29T12:27:57.071Z

Learnt from: ArnabChatterjee20k
Repo: utopia-php/database PR: 747
File: src/Database/Adapter/Mongo.php:1449-1453
Timestamp: 2025-10-29T12:27:57.071Z
Learning: In src/Database/Adapter/Mongo.php, when getSupportForAttributes() returns false (schemaless mode), the updateDocument method intentionally uses a raw document without $set operator for replacement-style updates, as confirmed by the repository maintainer ArnabChatterjee20k.

Applied to files:

• src/Database/Adapter/Mongo.php
• tests/e2e/Adapter/Scopes/SchemalessTests.php

🧬 Code graph analysis (5)

src/Database/Adapter/SQL.php (1)

src/Database/Query.php (1)

• Query (8-1207)

src/Database/Validator/Query/Filter.php (2)

src/Database/Query.php (1)

• Query (8-1207)

src/Database/Validator/Query/Order.php (1)

• isValidAttribute (30-55)

src/Database/Adapter/Mongo.php (1)

src/Database/Query.php (1)

• Query (8-1207)

tests/e2e/Adapter/Scopes/SchemalessTests.php (4)

src/Database/Adapter/Mongo.php (2)

• getSupportForAttributes (2654-2657)
• exists (344-362)

src/Database/Adapter/SQL.php (2)

• getSupportForAttributes (949-952)
• exists (180-222)

src/Database/Query.php (3)

• Query (8-1207)
• exists (1192-1195)
• notExists (1203-1206)

src/Database/Document.php (2)

• getId (63-66)
• getAttributes (194-212)

src/Database/Query.php (6)

src/Database/Adapter/Mongo.php (1)

• exists (344-362)

src/Database/Adapter/SQL.php (1)

• exists (180-222)

src/Database/Database.php (1)

• exists (1496-1501)

src/Database/Adapter.php (1)

• exists (501-501)

src/Database/Adapter/Pool.php (1)

• exists (138-141)

src/Database/Adapter/SQLite.php (1)

• exists (76-106)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Setup & Build Docker Image

🔇 Additional comments (2)

src/Database/Validator/Queries.php (1)

80-127: Routing EXISTS/NOT_EXISTS to FILTER validators is correct

Adding Query::TYPE_EXISTS and Query::TYPE_NOT_EXISTS to the Base::METHOD_TYPE_FILTER branch aligns them with other filter methods, so they get processed by the same Filter validator chain as the rest. This wiring looks correct and complete.

tests/e2e/Adapter/Scopes/SchemalessTests.php (1)

1221-1284: LGTM! Comprehensive test coverage for notExists.

The test correctly verifies the MongoDB semantics where a field set to null is considered "existing" and therefore excluded from notExists results (line 1262). The combination test at lines 1277-1281 provides excellent coverage for real-world query scenarios.

[fogelito]

LGTM lets wait for @abnegate review as well