Extend JSON serialization capabilities #1612
Conversation
|
At a first quick glance, this PR looks great. Can I ask what kind of workflow you have in mind to make use of this feature? |
I'm trying to set up document nodes that reference files in |
|
Sounds like something Intake could do for you, but I might be slightly biased :) |
fsspec/spec.py
Outdated
| if (obj_cls := self.try_resolve_fs_cls(dct)) is not None: | ||
| return AbstractFileSystem.from_dict(dct) | ||
| if (obj_cls := self.try_resolve_path_cls(dct)) is not None: | ||
| return obj_cls(dct["str"]) |
There was a problem hiding this comment.
If these both fail, you end up with just a dictionary?
There was a problem hiding this comment.
Yes, that is the default behaviour when decoding a dictionary from JSON.
fsspec/spec.py
Outdated
| try: | ||
| path_cls = _import_class(fqp) | ||
| except Exception: | ||
| raise |
There was a problem hiding this comment.
why catch Exception only to raise it again, all within the outer suppress block?
There was a problem hiding this comment.
It's just to have the same control flow as the code for decoding the filesystem class. If you think it's too verbose then I can simplify it.
Sure - I'll move them to |
|
I have finished addressing your comments. |
Co-authored-by: Martin Durant <[email protected]>
#247 first introduced JSON serialization of filesystem objects. However, it fails to handle
storage_argsandstorage_optionsthat containPathor other filesystems. This can occur in a few cases, such as:Pathinstance to construct aDirFileSystem(not officially documented but thePathobject is automatically converted to a string in the initializer, so it is functional)CachingFileSystem,DirFileSystemandReferenceFileSystem.This PR makes it possible to serialize such filesystems.