DICOM @ OFFIS

Discussion Forum for OFFIS DICOM Tools - For registration, send email with desired user name to the OFFIS DICOM team
It is currently Mon, 2017-12-18, 17:38

All times are UTC + 1 hour




Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Mon, 2017-09-25, 22:16 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
Hello,

a while ago I had a similar problem on Windows - using 64bit binaries solved the problem. Now it appears on Mac OS. The DICOM source file contains >1800 frames at 30fps and 1280x960. This results in a segmentation fault both on v3.6.0 and 3.6.2. Splitting the frames would be possible, but is it needed.

Hardware: Macbook Pro, 16GB RAM, 2,8 GHz Intel Core i7. Could this be a memory exhausting problem?

Code:
macbook:~ jb$ ls -lh ./A0000
-rwxr-xr-x@ 1 jb  staff   222M 25 Sep 23:07 ./A0000
macbook:~ jb$ file ./A0000
./A0000: DICOM medical imaging data
macbook:~ jb$./dcmj2pnm +Fa +op -O --log-level trace A0000 ../mf
D: Start Of Frame 0xc0: width=1280, height=960, components=3
D:     Component 0: 2hx1v q=0
D:     Component 1: 1hx1v q=1
D:     Component 2: 1hx1v q=1
D: Define Huffman Table 0x00
T:           0   1   5   1   1   1   1   1
T:           1   0   0   0   0   0   0   0
D: Define Huffman Table 0x10
T:           0   2   1   3   3   2   4   3
T:           5   5   4   4   0   0   1 125
D: Define Huffman Table 0x01
T:           0   3   1   1   1   1   1   1
T:           1   1   1   0   0   0   0   0
D: Define Huffman Table 0x11
T:           0   2   1   2   4   4   3   4
T:           7   5   4   4   0   1   2 119
D: Start Of Scan: 3 components
D:     Component 0: dc=0 ac=0
D:     Component 1: dc=1 ac=1
D:     Component 2: dc=1 ac=1
D:   Ss=0, Se=63, Ah=0, Al=0
Segmentation fault: 11


Many thanks for any hints.


Jens


Top
 Profile  
 
PostPosted: Tue, 2017-09-26, 21:21 
Offline
DCMTK Developer

Joined: Fri, 2004-11-05, 13:47
Posts: 1650
Location: Oldenburg, Germany
Hi Jens,

can you provide the image, maybe anonymized? If possible, you can send it (via download link) to onken ät open-connections dot de, I'll share it with the rest of the (small) DCMTK team. It seems the segfault happens in the 3rd-party JPEG decoder we integrated into DCMTK :/

Best,
Michael


Top
 Profile  
 
PostPosted: Wed, 2017-09-27, 05:35 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
Hi Michael,

the download link has been sent.

Many thanks

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 14:05 
Offline
OFFIS DICOM Team
OFFIS DICOM Team

Joined: Tue, 2004-11-02, 17:22
Posts: 1208
Location: Oldenburg, Germany
It seems that you have found a bug in DCMTK here. The sample image you sent cannot be converted to an uncompressed DICOM format because the pixel data would be larger than 4 GBytes (1802 frames, 1280x960, RGB results in ca. 6.3 GBytes uncompressed). The dcmj2pnm tool is not too clever when it comes to handling multi-frame images, it will attempt to decompress the complete multi-frame image, in memory. This process should actually fail with an error message that the image is too large for that, but it seems that some calculation overflows or gets cut off through a typecast. So DCMTK allocates unsufficient memory and then happily starts filling that buffer with image information until the point (around 2 GBytes) where it starts writing beyond the buffer size and causes a segfault. We will fix that, but that will not help you much in this specific case, because instead of crashing dcmj2pnm will just refuse to process the image.


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 14:15 
Offline
OFFIS DICOM Team
OFFIS DICOM Team

Joined: Tue, 2004-11-02, 17:22
Posts: 1208
Location: Oldenburg, Germany
The bug has been registered in our bug tracker as issue #793: http://support.dcmtk.org/redmine/issues/793


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 15:45 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
Hello Marco,

Marco Eichelberg wrote:
It seems that you have found a bug in DCMTK here. The sample image you sent cannot be converted to an uncompressed DICOM format because the pixel data would be larger than 4 GBytes (1802 frames, 1280x960, RGB results in ca. 6.3 GBytes uncompressed). The dcmj2pnm tool is not too clever when it comes to handling multi-frame images, it will attempt to decompress the complete multi-frame image, in memory. This process should actually fail with an error message that the image is too large for that, but it seems that some calculation overflows or gets cut off through a typecast. So DCMTK allocates unsufficient memory and then happily starts filling that buffer with image information until the point (around 2 GBytes) where it starts writing beyond the buffer size and causes a segfault. We will fix that, but that will not help you much in this specific case, because instead of crashing dcmj2pnm will just refuse to process the image.


Is there another or even better tool I should use for this purpose? Is there any workaround you know of?

I need access to each frame of these multiframe images in order to anonymize them. In the meanwhile I would try to extract the frames in parts of ca. 1000 Frames, which seems to work, but I have to re-enumerate the .ppm files to re-concatenate them ...

Do you have a better idea to anonymize multiframe US images?

Thanks for your support anyway,

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 16:50 
Offline
DCMTK Developer

Joined: Tue, 2011-05-03, 14:38
Posts: 1882
Location: Oldenburg, Germany
I would write a little program for this purpose. The DicomImage class, which is used by dcmj2pnm, is able to access only parts of the pixel data element if initialized appropriately (see this Howto).

However, are you sure that using rendered pixel data (as created by dcmj2pnm and the DicomImage class) is a good idea when creating a DICOM image as output? How do you ensure that the describing header information still fits the value of the Pixel Data element?

Have you already checked David Clunie's "DicomCleaner"?


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 17:44 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
My little 'software' consists of some shell scripts. I'm radiologist with a little bit of experience in shell programming.

Unfortunately, I have no experience with programming C.

The created output using ffmpeg reading ppm output of dcmj2pnm is not DICOM!
Target format is quicktime for Mac OS, or avi with wmv2 codec for Windows. The goal is to have ANONYMIZED, most compatible movie files running smoothly in Apple Keynote or MS Powerpoint.

The main script (dcm2mov.sh) extracts the multiframe pixel data, generates one or multiple overlay images (depending on ultrasound machine) to
remove the name of the patient from the US image file and concatenates the ppm files with ffmpeg to quicktime ...

David Clunies DICOMCleaner does not help in this case, because the patient name is coded in the pixel data.

Another (simpler) approach is highly appreciated!

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 19:18 
Offline
DCMTK Developer

Joined: Tue, 2011-05-03, 14:38
Posts: 1882
Location: Oldenburg, Germany
Did you miss this section? http://www.dclunie.com/pixelmed/softwar ... l#Blackout

I have no Mac but OsiriX might also provide a solution (including export to MPEG4 and Quicktime).


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 19:40 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
Yes, I missed this section. Sorry.

My 'software' is working since five years for a bunch of people very well, for sone users with storescp support and automagic conversion after receiving the data from the ultrasound machine via ethernet. There is no need to install Java. It is absolutely no user interaction required for anonymization and conversion, the user will find the movie files on their harddisk ready for insert into Keynote. The generated overlays are not recognizable by the naked eye. Most users of my software are running Mac OS X. They are physicians with lots of exams per day needed to be converted without any interaction.

OSiriX is a nice software, but here is also too much user interaction needed.

My approach seems to be not the most elegant, but I found no better yet for this purpose. Still looking for a more efficient way ...

Many thanks,

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 19:46 
Offline
DCMTK Developer

Joined: Tue, 2011-05-03, 14:38
Posts: 1882
Location: Oldenburg, Germany
A more efficient way would probably require programming (see above). For example, I did this many years ago for the dcm2avi tool and underlying library (Windows only, though).


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 20:04 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
I really would like to transform my shell scripts into a much more efficient program using a mixture of storescp, dcmdump, dcm2jpnm, convert, identify, ffprobe and ffmpeg, but this would require learning C, which would cost me years. Unfortunately I'm fully absorbed with my daily work as a radiologist. Shell programming is much more simple for me and does the job.

Until now.

Wouldn't you like to write such a program? ;-)

Best,

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 20:08 
Offline
DCMTK Developer

Joined: Tue, 2011-05-03, 14:38
Posts: 1882
Location: Oldenburg, Germany
Quote:
Wouldn't you like to write such a program? ;-)

Sure, but not for free ;-) See: https://www.jriesmeier.com/development/


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 20:24 
Offline

Joined: Thu, 2015-09-03, 09:36
Posts: 26
If I understand the bug tracker ticket right, buffer size > 4 GByte is not permitted in DICOM? Does this mean, the problem with such a large DICOM multiframe file is not to solve?

Why is it needed to decompress all images into the RAM?

Wouldn't it be possible to write the multiframe ppm file directly to the harddisk?

Jens


Top
 Profile  
 
PostPosted: Thu, 2017-09-28, 21:05 
Offline
DCMTK Developer

Joined: Tue, 2011-05-03, 14:38
Posts: 1882
Location: Oldenburg, Germany
Quote:
If I understand the bug tracker ticket right, buffer size > 4 GByte is not permitted in DICOM? Does this mean, the problem with such a large DICOM multiframe file is not to solve?

Yes, DICOM does not allow element values to be larger than 4 GB - 1, but no, this does not mean that the problem cannot be solved. It's just the reason for the internal limitation of the dcmj2pnm tool (and its underlying library), i.e. the segfault is also caused by the current implementation (which stores the decompressed pixel data in a DICOM element value as described above).

Quote:
Why is it needed to decompress all images into the RAM?

It is not needed. This is just the way it was implemented (15 or 20 years ago, when the DICOM multi-frame images were not that large).

Quote:
Wouldn't it be possible to write the multiframe ppm file directly to the harddisk?

There is no multi-frame PPM file (as far as I know) and it's not the PPM file that causes the segfault (see above).

I would rather recommend to implement the following command line option from dcm2avi also for dcm[j]2pnm:
Code:
memory handling:

  -fm   --frames-in-memory  [f]rame count: integer
          number of frames in memory (default: all)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 21 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group