fix(python): Parse uppercase config keys

[ion-elgreco]

In delta-rs we also parse config keys that are uppercase, this provides more flexibility and allows people to use same storage options for delta-rs and pola-rs.

Additionally instead of raising for non existing config keys, we just filter them out. This would make it easier to keep engine specific keys in the storage options in python, since delta-rs has some of those to handle mounted storage.

• closes Scan_delta parameters are now lowercase since 1.14.0 for S3 #19849

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.46%. Comparing base (da38e37) to head (6ec5f32).
Report is 48 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #19852      +/-   ##
==========================================
+ Coverage   79.33%   79.46%   +0.13%     
==========================================
  Files        1548     1555       +7     
  Lines      214247   216315    +2068     
  Branches     2460     2456       -4     
==========================================
+ Hits       169971   171905    +1934     
- Misses      43718    43852     +134     
  Partials      558      558              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[ritchie46]

Thanks @ion-elgreco. Can you add a test as well?

[ion-elgreco]

@ritchie46 sure, Ill take a look Sunday.

Are there existing integration tests with any object storage?

[ritchie46]

I think implicitly in python by forcing async.

[ritchie46]

We can also add a simple rust unit test with a comment on why that is tested. E.g. test parsing an uppercase config.

[ion-elgreco]

We can also add a simple rust unit test with a comment on why that is tested. E.g. test parsing an uppercase config.

Sure will do that, don't have time until Sunday for it tho

[ritchie46]

Now invalid keys are silently ignored? That doesn't seem right.

[ion-elgreco]

I am doing it explicitly. In delta-rs for example we use quite some custom storage_options for our internal log/objectstore handling.

So when you do a read_delta and you pass those additional options from your DeltaTable object into pola-rs, it will error out, see this comment #19103 (comment)

[ritchie46]

Right, can you add a comment that some keys are read upstream and may silently be ignored there.

[ion-elgreco]

Donee

[ritchie46]

Thank you @ion-elgreco.