Question About Sparse WSI DICOM

[wallyyeh]

Hi everyone,

I am currently writing a program to convert image data directly from an instrument into DICOM format.
The instrument does not always return data for all areas, so I need to handle sparse data.

According to the DICOM WSI specification(https://dicom.nema.org/dicom/dicomwsi/):

Some instruments which digitize microscope slides do not capture all areas of the slide at the highest resolution. In this case, the image data within any one level of the conceptual pyramid may be sparse, i.e., lacking some of the tiles.

I’m trying to write a DICOM file where some frames are empty. Below is a snippet of my code simulating this scenario:

Code: Select all

#define FRAME_COUNT (100)

int main() {

    // our goal is to create JPEG-compressed DICOM
    std::string filename = "test.jpg";
    std::ifstream ifs(filename, std::ios::binary);
    std::vector<Uint8> jpegData((std::istreambuf_iterator<char>(ifs)), (std::istreambuf_iterator<char>()));

    // create JPEG-compressed DICOM
    DcmFileFormat file_format;
    DcmDataset* dataset = file_format.getDataset();

    // set necesarry DICOM tags
    dataset->putAndInsertOFStringArray(DCM_PatientName, "Test Patient");
    dataset->putAndInsertOFStringArray(DCM_StudyInstanceUID, "1.2.3.4.5.6.7.8");
    dataset->putAndInsertOFStringArray(DCM_SeriesInstanceUID, "1.2.3.4.5.6.7.9");

    // set DICOM Pixel Data tags for JPEG
    std::string frame_count_str;
    size_t str_len = std::snprintf(nullptr, 0, "%d", FRAME_COUNT + 1);
    std::vector<char> buf(str_len + 1);
    std::snprintf(&buf[0], str_len + 1, "%d", FRAME_COUNT + 1);
    frame_count_str = std::string(&buf[0], &buf[0] + str_len);
    dataset->putAndInsertOFStringArray(DCM_NumberOfFrames, frame_count_str.c_str());
    dataset->putAndInsertUint16(DCM_BitsAllocated, 8);
    dataset->putAndInsertUint16(DCM_BitsStored, 8);
    dataset->putAndInsertUint16(DCM_HighBit, 7);
    dataset->putAndInsertUint16(DCM_SamplesPerPixel, 1);
    dataset->putAndInsertUint16(DCM_Rows, 2048);
    dataset->putAndInsertUint16(DCM_Columns, 2048);
    dataset->putAndInsertOFStringArray(DCM_PhotometricInterpretation, "YBR_FULL_422");
    dataset->putAndInsertOFStringArray(DCM_PixelRepresentation, "0");

    // our DICOM filename
    std::string out_file_name = "LEVEL_0.dcm";
    OFFile out_file;
    out_file.fopen(out_file_name.c_str(), "wb");
   
    DcmOutputFileStream out_file_stream(out_file);
    // DICOM header
    file_format.transferInit();
    file_format.write(out_file_stream, EXS_JPEGProcess1, EET_ExplicitLength, nullptr);
    file_format.transferEnd();

    // Pixel Sequence
    Uint8 pixel_sequence_data[] = { 0xe0, 0x7f, 0x10 ,0x00, 0x4f, 0x42, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff };
    out_file.fwrite(pixel_sequence_data, sizeof(pixel_sequence_data), 1);

    // dummy entry
    DcmPixelItem  dummy_item(DCM_ItemTag);
    dummy_item.write(out_file_stream, EXS_JPEGProcess1, EET_ExplicitLength, nullptr);

    // saving jpeg as frames 
    for (int i = 0; i < (FRAME_COUNT - 10); i++) {
        DcmPixelItem pixelItem(DCM_ItemTag);
        pixelItem.putUint8Array(jpegData.data(), jpegData.size());
        pixelItem.write(out_file_stream, EXS_JPEGProcess1, EET_ExplicitLength, nullptr);
    }

    // end of Sequence
    DcmPixelItem  seq_delimitation(DCM_SequenceDelimitationItemTag);
    seq_delimitation.write(out_file_stream, EXS_JPEGProcess1, EET_ExplicitLength, nullptr);

    out_file.fflush();
    out_file.fclose();
    return 0;
}

In the above code, I specify FRAME_COUNT = 100 but only insert 90 frames in the loop to simulate the presence of 10 empty frames.
However, the resulting DICOM file appears to be corrupted and cannot be opened by third-party DICOM viewers.

My Questions are:
1. Is this approach of simulating empty frames correct?
2. Should the number of frames specified in the DICOM header (NumberOfFrames) match the actual number of frames written?
3. Is there additional metadata or handling required for sparse data in WSI DICOM files?

Any guidance or suggestions would be greatly appreciated. Thanks.

[J. Riesmeier]

It seems that you are still missing many details of the technical specification, i.e. you have to read the DICOM Standard on how WSI objects are encoded, in particular, in TILED_SPARSE mode.

According to the DICOM WSI specification(https://dicom.nema.org/dicom/dicomwsi/):

This is not the specification. It is just a general description. The technical description can be found in the DICOM Standard, e.g. in PS3.3.

My Questions are:
1. Is this approach of simulating empty frames correct?
2. Should the number of frames specified in the DICOM header (NumberOfFrames) match the actual number of frames written?
3. Is there additional metadata or handling required for sparse data in WSI DICOM files?

Re. 1: As far as I can see, your sample code does not really make sense, e.g. you should not write directly to a file that you passed over to the DCMTK, and there is also no need to do this (in your case).

Re. 2: Yes, but this question is not related to the DCMTK, but a general one.

Re. 3: Yes, of course. For example, you need to set the Dimension Organization Type (0020,9311) accordingly. And, you need to fill the Per-frame Functional Groups Sequence (5200,9230)... But again, this is not specific to the DCMTK.

My suggestion would be: Before you start programming, you should read and understand how DICOM sparse WSI objects are to be encoded according to the DICOM Standard.

[wallyyeh]

Hi, Riesmeier:
> My suggestion would be: Before you start programming, you should read and understand how DICOM sparse WSI objects are to be encoded according to the DICOM Standard.
Thanks for pointing out where the technical specifications are located. I'll definitely go through them and post my findings here later.