Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow mnist data to be writen to remote file system. #3890

Merged
merged 1 commit into from
Apr 6, 2018

Conversation

qlzh727
Copy link
Member

@qlzh727 qlzh727 commented Apr 6, 2018

For b/77657597. Btw, I don't have a TPU instance to validate this change, I think the tf.gfile should do the trick with internal env.

@qlzh727 qlzh727 requested a review from saeta April 6, 2018 03:59
@qlzh727 qlzh727 requested review from k-w-w, karmel and nealwu as code owners April 6, 2018 03:59
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

@googlebot
Copy link

CLAs look good, thanks!

@googlebot googlebot added cla: yes and removed cla: no labels Apr 6, 2018
@robieta
Copy link
Contributor

robieta commented Apr 6, 2018

Is this sufficient? You're still relying on urllib to be able to write the archive to the file system.

Copy link
Contributor

@saeta saeta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should work. :-)

@qlzh727
Copy link
Member Author

qlzh727 commented Apr 6, 2018

@robieta, the urllib is only used to read the input file object, which does not change within this PR. The tf.gfile.Open should support different file system name space (eg, gs:/ in this case), whereas the default python open() only support local file system. You can read internal doc about gfile API to get more details.

@qlzh727 qlzh727 merged commit 2b4f7b4 into tensorflow:master Apr 6, 2018
omegafragger pushed a commit to omegafragger/models that referenced this pull request May 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants