DicomImage constructor slow opening JPEG multiframe image

All other questions regarding DCMTK

Moderator: Moderator Team

Post Reply
Message
Author
mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

DicomImage constructor slow opening JPEG multiframe image

#1 Post by mkostrze »

Hi list!

I have 50+ frames jpeg compressed US multiframe image. It is slow to create new DicomImage object from it (regardless of number of frames passed in as parameter). For my image it takes 2+ secs on my machine. This also pertains to dcmj2pnm program - the phase 'preparing pixel data' lasts also this long.

- Am I right thinking that it tries to decompress *all* the frames during the constructor regardless of that I said it to create DicomImage of, say, 5th frame only?

- Is there any way to postpone the decompression? or do it one frame at once, not all of them at once?

thank you in advance and best regards

Marco Eichelberg
OFFIS DICOM Team
OFFIS DICOM Team
Posts: 1437
Joined: Tue, 2004-11-02, 17:22
Location: Oldenburg, Germany
Contact:

#2 Post by Marco Eichelberg »

DicomImage indeed decompresses all frames during the constructor, no matter which part of the pixel data you really want to access later. Currently there is no API in DCMTK that would allow for a selective decompression of individual frames, you can only decompress all or nothing. There is no way to postpone the decompression once you are executing the DicomImage constructor. The only workaround possible would be to remove all the JPEG fragments you don't need from the underlying dataset before feeding the remaining dataset to DicomImage. This will force you to determine which fragments belong to which frame, and while considering how to compute this in the possible case where the offset table is empty you might start to understand why we have not implemented that ourselves... :wink:

mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

#3 Post by mkostrze »

Thanks a lot for the answer, I can imagine the what the problem is, and I don't blame dcmtk by no means. I was using other toolkit (dcm4che) which is capable of doing this (at least with my images ;) but I need something in C not java. The problem with long lasting constructor is that user has to wait until it finishes while she may want to switch to looking at another image in the meanwhile and 2+ sec period of waiting for images is too long for MD's ;)
What about a patch to DCMTK that would decompress only needed frames in the simple case and decompress all as a fallback? Any suggestions to the brave man (me) who tries to do it? (I'll gladly give it back to you when (if?) it works/is feasible)

best regards,

mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

#4 Post by mkostrze »

Postscriptum:

Also - what do you think about another solution, that determines the frame starting points dynamically, based on previously decoded frames? Frames are usually read from beginning to the end, so after decoding frame #1 you'll know when frame #2 begins, w/o parsing them all in the constructor. So when somebody wants #5 from the beginning, she may need to decompress #1..#5, (but only to #5). But when she accesses the frames sequentially (like in typical case when showing a clip playing) it'll be all ok.
Do you think writting such 'improvement' is viable with current dcmtk api?

thank you, and best regards

Marco Eichelberg
OFFIS DICOM Team
OFFIS DICOM Team
Posts: 1437
Joined: Tue, 2004-11-02, 17:22
Location: Oldenburg, Germany
Contact:

#5 Post by Marco Eichelberg »

Any suggestions to the brave man (me) who tries to do it?
Our principle is to always encourage the brave :D
Right now the problem is that there is no direct relationship between the DicomImage class and the (various) decompression codecs, which are plugged into module dcmdata by means of a plugin concept. That means that the constructor of class DicomImage always asks the underlying DcmDataset to convert "itself" to an uncompressed representation before even trying to access the underlying pixel data. The DcmCodec class (and its child classes) which do the compression as such always decompress a complete DICOM object (i.e. all frames of all pixel data attributes in the complete dataset) and do not have an API to call for selective decompression.

Now, things are probably not as bad as it sounds in the first place. One would need to implement new code in two places:
  • In class DcmCodec and derived classes, implement a new API that does essentially the same thing as the current decode() routine, but only decompresses a single frame and does not store this frame in the DcmDataset structure (the DcmPixelData element, to be precise) but instead passes back the decompressed frame to the caller. This would allow for a decompression of individual frames and also acknowledge the fact that the structure of class DcmDataset and relatives does not allow for the representation of a "partially decompressed" element - an element is either compressed, or uncompressed, or both, but there are no partial things.
  • In the constructor of class DicomImage, when a frame number is passed, do not directly access the PixelData attribute, but instead use the new codec API to retrieve only the decompressed data for a single frame.
Concerning the issue of the possible fragmentation of the compressed data for a single frame and the optionality of the offset table, one could distinguish the following situations:
  • There is only one frame (NumberOfFrames=1) - no problem, all fragments "belong" to frame 1.
  • The caller tries to decode the frames in increasing order, starting from frame 1. This is also fine (and actually what the DcmCodec implementations do - they never look at the offset table). You simply start with the first fragment and "consume" fragments until one frame is decompressed. You then ignore the rest of the fragment (which might contain trailing garbage, I have seen examples of that) and start the next frame with the beginning of the next fragment.
  • There is an offset table. In this case, the caller could even decompress frames in arbitrary order.
  • The number of fragments equals the number of frames (plus the offset table element). In this case an arbitrary selection of frames is possible even if the offset table is empty because each frame must have at least one fragment, so in this case each frame has exactly one fragment).
  • Finally the worst case: the offset table is empty, there are multiple frames and there are more fragments than frames. In this case an arbitrary selection of frames is not possible, one must decode from the first frame to the selected one. In this case the attempt to decode an arbitrary frame could simply fail with an error code, or one could in this case fall back to the current behaviour of the toolkit, which would also work, but be slow of course.
As you see, all of this is possible, it's just a little (well, relatively speaking :wink:) piece of work.

mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

#6 Post by mkostrze »

Thank you very much for the answer, I really appreciate it. It looks I really need it and I'll have to try doing it this week, I wonder what will result from this try... The patch will be relative to version 3.5.4.

thanks again, best regards,

mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

Progress...

#7 Post by mkostrze »

Hi,

I spent some time on learning how dcmtk works and trying to figure out how to do it. Changing codec seems to be very simple, I localized proper fragment. But the 2nd part of it - integrating DicomImage with modified codec is much harder. First of all - even if I followed you advice somehow, it may won't be what I'm looking for. I need something like this (kind of a pseudocode)

1) Open dicomobject (whatever it is) from the input stream
2) do image = dicomobject->read(1st image). During this, 'dicomobject ' reads appropriate number of fragments and remembers the beginning of the 1st frame in its 'table of content' (offset table) (in case that no 'TOC' is stored in original dicom file)
2a) do image->getDIB()
3) do dicomobject->read(2nd image). The 'dicomobject' has its partial TOC calculated before so it can decompress 2nd frame knowing where it begins and updates TOC
3b) do image->getDIB()

Now, returning back to dcmtk, following your advice I imagined that I could do new DicomImage(1st frame) as step #2 and new DicomImage(2nd frame) as step #3. And step #2 would decompress only 1st frame, but unfortunately step #3 would decompress 1st and 2nd, b/c there is no state information (the TOC) between the steps. (Or I missed something important, did I?)

Also if I created new api for decoder to retrieve the decompressed frame instead of putting it in the dataset, I wonder how I can use it in DcmImage (DiImage?) For now it looks to me it's quite tightly coupled to existing datastructures to tweak something w/o breaking too much.

The other approach would be to create DicomImage from all of the frames, but make frame decompression 'lazy' instead of doing it in the constructor. But it seems to be unfeasible b/c DicomImage is coupled with the dataset and requires it to be decompressed and as you truly said you cannot decompress it partially.

BTW using the index when it exists is quite important for video streaming. When you write a video parser/splitter you may want (you should) implement quality control, which means that if renderer is femined (so the splitter is too busy), the splitter may be suggested to skip couple of frames to catch up with the stream time. W/o using the index it's hard to do (impossible w/o actual decompression what you're trying to avoid) (unless you precalculated it, or calculate lazily on the fly)

Please, am I missing something important? I'd like to solve it generically, but as a last sort of rescue I'll just do a hack that applies to my (jpeged with index) images by modifying the dataset prior feeding it to DicomImage. But it will work only with 1st fragment containing the index present and is, well, just ugly... Any more thoughts, please, please?

thank you very much, best regards

mkostrze
Posts: 6
Joined: Thu, 2006-11-09, 15:08

#8 Post by mkostrze »

What about this:

- I'll create an interface StatefulCodecFactory which will produce StatefullCodecs objects. The factory method will have params

Code: Select all

const DcmRepresentationParameter * fromRepParam, DcmPixelSequence * pixSeq, const DcmStack& objStack

- StatefulCodec will have method
decode(DcmPolymorphOBOW& uncompressedPixelData, const DcmCodecParameter * cp, int startFrame, int count)
so given StatefulCodec instance (and so given DcmPixelSequence) there will be possibility to decompress only frames needed into desired uncompressedPixelData and taking advantage of previous decompression (so offset table may be calculated on the fly and retained between invocations


- I'll make DJCodecDecoder (as an example) to implement StatefulCodecFactory, copy the existing 'decode' logic to new DJStatefulCodec class with simple changes and change implementation of existing 'decode' to create DJStatefulCodec and use its decode method, so the logic is in one place, shouldn't affect the performance, or break existing API



- I'll make DicomImageFactory class, which will be able to produce DicomImages objects for given range of frames (in particular one frame) by creating a dataset having only requested frames using new StatefulCodecFactory (or fallback to default method if suitable codec does not implement StatefulCodecFactory). So the user will create DicomImageFactory once (which I suppose will be lightweight) and then asks to create DicomImage from nth frame of underlying dataset. DicomImageFactory will try to decompress only frames needed (and StatefulDecoder will rememeber fragments offsets calculated during evaluating previous calls), creating datasets for each requested frame range and DicomImage on top of them.


Does it sound resonable or is just plain stupid? What are the traps (also performance traps)? I assumed that the most heavyweight operation is frame decompression, but I'll create one dataset and DicomImage per frame - is it something I should be affraid of?


thanks again, best regards,

Marco Eichelberg
OFFIS DICOM Team
OFFIS DICOM Team
Posts: 1437
Joined: Tue, 2004-11-02, 17:22
Location: Oldenburg, Germany
Contact:

#9 Post by Marco Eichelberg »

Your analysis is correct: You need to maintain state information (the partial index table you mention) between calls to the decoder. I would implement a new class for this. Then there is the interesting question of where to store the state information. Currently, the decoding process involves four classes, each of which could be used, at least in theory:
  • The DcmPixelSequence object that maintains the compressed pixel data fragments and is kept inside the DcmPixelData object.
  • The DcmCodec object representing the codec. Not the best choice because all calls to encode() and decode() are const, indicating that the codec should be state-less, which is important for multi-threaded applications.
  • The DcmCodecParameter object, which stores the "configuration" of the codec. Also not a good choice, because this is a singleton that is shared between all instances of the same codec class.
  • The DcmRepresentationParameter object that is passed to the codec. This is mainly used during compression to pass additional information about the desired compression result to the codec, such as the image quality/compression factor).
My favourite would actually be the DcmPixelSequence object, which could receive an additional (optional) member representing the "partial TOC". The decoder always has access to this object, which is non-const during the decompression process, and the object already stores all we know about the compressed image, so it would logically be the right place to also store additional information we have received during the partial decompression process so far. This would also avoid the need for a codec factory. The decoder would simply interact with a "DcmPartialIndexTable" object stored inside the DcmPixelSequence .

Creating different DicomImage instances for each frame is a good idea, because this is currently the only way of making sure that DicomImage does not access all frames during execution of the constructor.

Creating different datasets for each DicomImage is a way to avoid changing the DicomImage class, but is very bad performance wise, because copying a DcmDataset is a very heavy operation. It would be really preferrable to modify class DicomImage such that the pixel data can be "pulled" out of a separate object passed to the constructor instead of the main pixel data element. Hard to say how complex this is - probably not too complex, but will require API extensions in many classes because the architecture of DicomImage is quite "distributed".

Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot], Google [Bot] and 1 guest