[Elastix] Problem with CUDA re-sampler

M.Staring at lumc.nl M.Staring at lumc.nl
Wed Jul 18 12:50:36 CEST 2012


IN addition, you used
TxMovingImageImage = recursiveMultiResolutionPyramidImageFilter->GetOutput(0);
which should probably be
TxMovingImageImage = recursiveMultiResolutionPyramidImageFilter->GetOutput(level);
Regards, Marius

From: elastix-bounces at bigr.nl [mailto:elastix-bounces at bigr.nl] On Behalf Of M.Staring at lumc.nl
Sent: woensdag 18 juli 2012 12:45
To: rtennakoon at swin.edu.au; elastix at bigr.nl
Subject: Re: [Elastix] Problem with CUDA re-sampler

Hi Ruwan,
I noticed that you create a new pyramid and use that result.
That is not needed. You can simply use the result of the pyramid that elastix currently uses. Something like:
currentImage = this->GetElastix()->GetElxMovingImagePyramidBase()->GetAsITKBaseType()->GetOutput( level );
There is also no need for the last level to be a special case. SetTransform and SetInterpolator are probably also not needed. You only need to set the input image and corresponding fixed image sizes, everything else has already been set before. If you fix these issues you may get a better behaving program.
HTH, Marius
From: elastix-bounces at bigr.nl<mailto:elastix-bounces at bigr.nl> [mailto:elastix-bounces at bigr.nl]<mailto:[mailto:elastix-bounces at bigr.nl]> On Behalf Of Ruwan Tennakoon
Sent: dinsdag 17 juli 2012 10:13
To: elastix at bigr.nl<mailto:elastix at bigr.nl>
Subject: Re: [Elastix] Problem with CUDA re-sampler

Dear Marius,
Thank you very much for your reply. I implemented the patch you send me.  I also added the  "LetResamplerFollowPyramid" part as you suggest (to makes the resampling in the initial step faster).
The problem is still there when a lower resolution image is used as input of the CUDA re-sampler in initial registration level. The program gets stuck when the resolution is changed (it does not happen when the defaultResampler is used instead of the CUDA re-sampler).
according to the analysis it gets stuck inside the function
::cudaCastToHost( const size_t sizevalue, const TInternalImageType* src, TInternalImageType* tmp_src, TImageType* dst )

As I can see the problem occurs due to a memory allocation problem for the output. To test this I added the following code to itkCUDAResampleImageFilter.hxx GenerateData() function

  this->AllocateOutputs();
  InputPixelType* data = this->GetOutput()->GetBufferPointer(); //I suspect memory for data is not allocated properly

  std::cout<< "Size of data in  itkCUDAResampleImageFilter.hxx genarateData()" << this->GetOutput()->GetLargestPossibleRegion().GetSize()<<std::endl;

  if (this->GetOutput()->GetLargestPossibleRegion().GetSize()[0] > 250) //check if in level 1
  {
        // the image in level 0 is [128 128 47] with total noof voxels 770048
        // the image in level 1 is [256 256 94] with total noof voxels 6160384
        std::cout << ".....START Write value to lower part of data" << std::endl;
        data[770000] = static_cast<InputPixelType>( 234 );
        std::cout << ".....END Write value to lower part of data" << std::endl;

        std::cout << ".....START Write value to upper part of data" << std::endl;
        data[1540099] = static_cast<InputPixelType>( 234 );
        std::cout << ".....END Write value to upper part of data" << std::endl;
  }
When the program is executed the size of data is shown as [256 256 94] but the  program gets stuck before the second data write command (data[1540099]). It seems that the memory allocated to the data is still 128*128*47 .
The modified version of the elxResampleBase.hxx and elxResampleBase.h files are attached herewith together with the parameter file and a log file. I would be very grateful if you could help me resolve this issue. thank you very much.

Kind Regards,
Ruwan Tennakoon
Email: rtennakoon at swin.edu.au<mailto:rtennakoon at swin.edu.au>

From: M.Staring at lumc.nl<mailto:M.Staring at lumc.nl> [mailto:M.Staring at lumc.nl]<mailto:[mailto:M.Staring at lumc.nl]>
Sent: Monday, 16 July 2012 6:32 PM
To: elastix at bigr.nl<mailto:elastix at bigr.nl>; Ruwan Tennakoon
Subject: RE: [Elastix] Problem with CUDA re-sampler

Hi Ruwan,
elastix indeed did not support the writing of resultant images after each iteration (only at the end and after each resolution). I added a function that does this, see the attached patch.
I only tested this for the normal resampler. It should automatically also work for the CUDAResampler, but please test this.
Note that we always resample the full moving image to the full fixed image region. This is specified in the function BeforeRegistrationBase(). If you want you can add an option "LetResamplerFollowPyramid" to the resampler base. This can be done by adding a function BeforeEachResolutionBase(), with contents similar to BeforeRegistrationBase(), but instead specifying the blurred & resized moving image, and corresponding sizes of the fixed image pyramid.
Also note that you don't need to put cuda specific code in the resampler base. In fact this is not the intention of the base class. Just implement the options you want with a normal resampler in mind and it should also work for the cuda resampler.
Regards, Marius
From: elastix-bounces at bigr.nl<mailto:elastix-bounces at bigr.nl> [mailto:elastix-bounces at bigr.nl]<mailto:[mailto:elastix-bounces at bigr.nl]> On Behalf Of Ruwan Tennakoon
Sent: maandag 16 juli 2012 7:25
To: elastix at bigr.nl<mailto:elastix at bigr.nl>
Subject: [Elastix] Problem with CUDA re-sampler

Hi,
I'm trying to generale the transformed moving image after every iteration in a multi-resolution (two levels) registration process using the CUDA re-sampler. in order to do this I've modified the elxResamplerBase.hxx and added the function in code.txt file to it. in it I set the input image to a copy of the fixed image down converted to the correct resolution of the registration level.

but when the program is running it get stuck as soon as the resolution is changed to level 2 from level 1 (illustrate in the log file attached).

according to the analysis it gets stuck inside the function
::cudaCastToHost( const size_t sizevalue, const TInternalImageType* src, TInternalImageType* tmp_src, TImageType* dst )

in cudaResampleImageFilter.cu. it seams like the output image is not defined with enough memory (its defined with the size m_MaxNumberOfVoxelsPerIteration which is still set to the no of voxels in the previous level input image). I also tried changing the output image definitions in ::cudaMallocImageData() function to

this->m_OutputImage = cuda::cudaMalloc<TInternalImageType>(static_cast<unsigned int>( nrOfOutputVoxels ) ); and also tryied to force the m_MaxNumberOfVoxelsPerIteration  = static_cast<unsigned int>( nrOfOutputVoxels ) ;

but the problem was not resolved. I would be very grateful if you could help me resolve this issue. thank you very much.




Best Regards,
Ruwan Tennakoon
PhD candidate
Faculty of Engineering and Industrial Sciences
Swinburne University of Technology
(CRICOS Provider 00111D)
John Street Hawthorn VIC 3122
Australia
Email: RTENNAKOON at groupwise.swin.edu.au<mailto:RTENNAKOON at groupwise.swin.edu.au>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bigr.nl/pipermail/elastix/attachments/20120718/1972008c/attachment.html>


More information about the Elastix mailing list