Skip to content

Comments

fix: multiprocessing in generating crystal graphs#46

Merged
mzjb merged 1 commit intomzjb:mainfrom
PhysAIzhijiang:OptParallel
Jun 6, 2023
Merged

fix: multiprocessing in generating crystal graphs#46
mzjb merged 1 commit intomzjb:mainfrom
PhysAIzhijiang:OptParallel

Conversation

@bsplu
Copy link
Contributor

@bsplu bsplu commented Jun 6, 2023

The flag multiprocessing = True can cause the computer to run out of resources.

This is because:
The num_threads keyword in kernel.py sets the torch threads.
But the process_worker uses multiprocessing, which conflicts with the torch threads.

Solution:
I modified the multiprocessing flag from bool to int and I set the torch threads to one in data.py. I restored them after the process_worker finished.

Suggestion:
To optimize performance and memory, I suggest using multiprocessing = 0 and changing num_threads to set the threads. This avoids large memory consumption when creating crystal graphs.

@mzjb mzjb merged commit 6f4c144 into mzjb:main Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants