Menu ▾ ▴

#400 dataflow slice crash around 2**31 boundary

critical
closed-fixed
Craig DeForest
None
9
2015-09-29
2015-09-27
Chris Marshall
No

I've run into a dataflow/slice crash in perl/PDL-2.007_02
for large piddles (>2GiB in size). Here is a program that
reproduces the crash for me on cygwin64 perl and on
SPP. Both on the same win7 system.

The failure seems to be related to dataflow and slice
but I really don't know enough about this affine slice
stuff and dataflow implementation to even start to
guess where to look. :-(

If you have a PDL-2.007_02 install and >4GiB of RAM,
I would appreciate if you would see if you can
reproduce the problem and to determine if it is
specific to windows somehow.

--Chris

~~~~~~

!/usr/bin/perl

use PDL;
use PDL::NiceSlice;

$PDL::BIGPDL = 1; # maybe this should be the default

$recs = ones(byte,3149824,998);

works for the original piddle

print "\$recs(:,900)->sum is ", $recs(:,900)->sum, "\n";

$ufr = $recs(29:-4068); # After this...perl crashes

crashes for me here

print "\$ufr(:,900)->sum is ", $ufr(:,900)->sum, "\n";
~~~~~

Related

Bugs: #400

Discussion

  • Chris Marshall

    Chris Marshall - 2015-09-27

    Given the plan to release PDL 2.014 with 64bit index support,
    this is a priority bug for that release.

     

    Last edit: Chris Marshall 2015-09-27

  • Chris Marshall

    Chris Marshall - 2015-09-27

    Since the failure happens when the operation crosses the 2**31 boundary, it is likely something in the code that is an int and needs to be PDL_Indx type.

     

  • Chris Marshall

    Chris Marshall - 2015-09-27

    Here is a simpler piece of code showing the problem which may be easier to debug:

    #!/usr/bin/perl
#
use PDL;
use PDL::NiceSlice;

$PDL::BIGPDL = 1;  # maybe this should be the default

$big = ones(byte,2**30,4);
print "\$big(:,3)->sum is ", $big(:,3)->sum, "\n";


$aaa = $big(3:-10);
print "\$aaa(:,3)->sum is ", $aaa(:,3)->sum, "\n";
     

  • Chris Marshall

    Chris Marshall - 2015-09-29

    Fixed in git master and will appear in the next PDL CPAN release.

     

  • Chris Marshall

    Chris Marshall - 2015-09-29
    • status: open --> closed-fixed
    • assigned_to: Craig DeForest
     

Log in to post a comment.