Say, you're working on Windows and continue to get a permission's error without a reason. Then it may be that a different Windows program is working on the folder (see earlier notes also). In the case that you can't find that program, the line
<?php closedir(opendir($dirname)); ?>
may solve the problem!
Make sure to write this before rmdir($dirname);.