Transition to 3.6.x

[Marcel]

Hi,

We're trying to move from 3.5.x to 3.6.x and things seem to be a bit different now:
- 3.6 dcmdump <dcmfile> fails with message
"E: DcmSequenceOfItems: Parse error in sequence, found (414e, 454d) instead of a sequence delimiter"
- 3.5 dcmdump simply produced a dump
This one of course we can solve for 3.6 with dcmdump +E <dcmfile>

But what can we do for the following 3.5 program segment?

Code: Select all

DcmFileFormat fileformat;
OFCondition status = fileformat.loadFile(argv[1]);
if(status.good()){
    ... process ...
}
else CERR << "Error: Cannot load file(" << status.text() << ")" << OFendl;

The 3.6 program failes with the same error as dcmdump does, statust.text() reads "Corrupted data", wheras the 3.5 program used to return true for status.good()...
Of course the dcm file has some problems, but its from a real SR device and with 3.5 we did not need to care about this.
Thanks for your help
Kind regards
Marcel

[J. Riesmeier]

Could you please send the corrupted DICOM file by email to dicom/at/offis/dot/de for further analysis?

[J. Riesmeier]

Thank you for the file. I can confirm that dcmdump from DCMTK 3.6.0 produces this kind of error, but also dcmdump from DCMTK 3.5.4 cannot read it by default. So which version did you use?

[J. Riesmeier]

Of course the dcm file has some problems, but its from a real SR device and with 3.5 we did not need to care about this.

Btw, this DICOM file uses a very old SOP class (SR Detail Storage), which was never intended to be used in products. It is defined in Supplement 23 (Frozen Draft for Trial Implementation) from 1997, but never made it into the DICOM standard!

This is the reason for your problems: Attribute (0040,A170) was defined as Observation Class in Supplement 23 (Frozen Draft), but is defined as Purpose of Reference Code Sequence in the DICOM standard. Since the file uses a transfer syntax with implicit VR, the DCMTK parser assumes that this is a Sequence of Items (SQ). I will ask DICOM WG6 to clarify this in the DICOM standard (if possible).

As a workaround for THIS file, you could add the following line to your data dictionary "dicom.dic" (in case it is loaded at runtime):

Code: Select all

(0040,A170)	CS	RETIRED_ObservationClassTrial	1	DICOM/retired

[Marcel]

The 3.5.x dcm release was 3.5.5_20090818

[J. Riesmeier]

Ok, seems to be an issue with the data dictionary (see above).

[J. Riesmeier]

No, this 3.5.5_20090818 version also did not work. If you look at the textual dump, some data elements are output as "Unknown Tag & Data", including the RETIRED_FindingsSequenceTrial.

This is not an issue with DCMTK but with an inconsistency in the DICOM standard (same tag has been used for different purposes). Let us wait what the DICOM working group recommends.

[Marcel]

Thank you for this info.
Can we simply edit

(0040,A170) CS RETIRED_ObservationClassTrial 1 DICOM/retired

in the installed tree's share/dcmtk/dicom.dic and recompile our app (-dll)
or
do we have to change dicom.dic within the dcmtk source tree, rebuild and install dcmtk and then recompile our app?

[J. Riesmeier]

I would not use this definition as part of the built-in data dictionary. Instead you should add this line to a loadable dictionary file (e.g. "dicom.dic") and specify the DCMDICTPATH environment variable accordingly (see "datadict.txt"). But please, only use this definition for such ancient SR files!

The major concern I have with this change is that "Purpose of Reference Code Sequence" (SQ) uses the same tag and if you ever load a DICOM file (with implicit VR) that contains such a data element (defined in the General Series Module) you'll get another parser error.

[Marcel]

OK, now we come down to some "real-world-ugglyties":
It seems to be a bad habit of the DICOM working group to re-use retired tags for other purposes, simply ignoring the lifetime of fossile devices out in the field...
As David Clunie once stated in another context:

We made a major error in the standard when Sup 23 was finalized, and change the
VR of one of the data elements from CS to SQ and used it for a different purpose
(rather than just retiring the data element). This is (0040,A170) CS Observation
Class in Sup 23 FZ, and (0040,A170) SQ Purpose of Reference Code Sequence in the
final text of Sup 23 and hence the current standard.

Since this is an implicit VR transfer syntax file, the
com.pixelmed.dicom.AttributeList parser doesn't know that this is supposed to be
a CS rather than an SQ as its dictionary tells it.

The dicom3tools parser contains a trick that allows it to skip over the
remainder of a fixed length sequence item, and hence can parse most of the
content.
I will take a look at whether it is worthwhile in the PixelMed toolkit to either
do the same thing, or simply to handle this particularly abused data element as
a special case (since we aren't likely to make the same mistake again in the
standard).

So this seems to be a special case requring some tricks?

[J. Riesmeier]

You are obviously referring to this discussion: http://tech.groups.yahoo.com/group/pixe ... essage/894

So this seems to be a special case requring some tricks?

Yes, I agree. A possible solution could be to add this special case to DcmItem::checkAndUpdateVR() ... but this would require access to the SOP Class UID at the nested item level, which is currently not possible

[Marcel]

Yes, I think checkAndUpdateVR() is not the right place for this. In David's Java App the trick was while parsing parsing (pseudo-code):

Code: Select all

if isSequenceVR {
    try{
         readNewSequenceAttribute
    }catch(supp23){
         if tag == PurposeOfReferenceCodeSequence {
             try{
                 createNewAttribute
             }catch(e){
                 report warning
                 skip this attribute
             }
        }
        else {
               // other kind of real problem nothing to to with supp 23
               throw Exception
        }
    }
}

Can something tike this be done a layer below when parsing?

[J. Riesmeier]

I don't think that we should introduce such a "hack" into the DCMTK. Maybe, you could check the SOP Class UID and Transfer Syntax UID in your code and then replace the VR in the global data dictionary before parsing such a SOP instance.