Memory Leak occurring When Inserting 1000CT Datasets in std::map

All other questions regarding DCMTK

Moderator: Moderator Team

Post Reply
Message
Author
vignesh
Posts: 14
Joined: Mon, 2020-01-20, 11:44

Memory Leak occurring When Inserting 1000CT Datasets in std::map

#1 Post by vignesh »

Hi I'm Using DCMTK 3.6.6 Version.

I have inserted 1000 CT DICOM file datasets into std::map. A CT file size is 48 KB. Initially, the Application size is 3 MB. When I load 1000 CT Files, the Application size is around 50 MB. And in the next iteration, it starts at 3 MB.
I am iterating the while loop after the 80th time, and the memory has not Cleared. It always maintains a minimum of 33+ MB (after the 80th iteration), even at the ideal time.

here the sample code.

Code: Select all

namespace fs = boost::filesystem;
typedef std::shared_ptr<DcmDataset> DcmDatasetPtr_;
typedef std::map<std::string, DcmDatasetPtr_> image_data_map_;

std::mutex c_mutex;

void prepare_image_(std::string& fname, image_data_map_ &img_data_map)
{
	DcmFileFormat file;
	OFCondition status;
	status = file.loadFile(fname.c_str());
	DcmDataset *dataset = file.getDataset();
	OFString sop_instance_uid, studyUid;
	dataset->findAndGetOFString(DCM_SOPInstanceUID, sop_instance_uid);
	DcmDatasetPtr_ data_ptr;
	{
		data_ptr = std::make_shared<DcmDataset>(*dataset);
		{
			std::lock_guard<std::mutex> lock(c_mutex);
			img_data_map.insert(std::make_pair(sop_instance_uid.c_str(), data_ptr));
		}
	}
	dataset->clear();
	data_ptr.reset();
}


void preparelist(std::vector<fs::path> &pathList, image_data_map_& img_map)
{
	for (auto filePath : pathList)
	{
		std::string f_path = filePath.string();
		prepare_image_(f_path, (img_map));
	}
}

int main(int argc, char* argv[])
{
	std::string dir_path = "E:/1000_CT";

	std::vector<fs::path> pathList;
	std::copy(fs::recursive_directory_iterator(dir_path), fs::recursive_directory_iterator(), back_inserter(pathList));
	int i = 1;
	while (true)
	{
		{
			image_data_map_ img_map;
			preparelist(pathList, std::ref(img_map));
		}
		std::cout << "Images are Prepared : " << i << std::endl;
		i++;
		boost::this_thread::sleep_for(boost::chrono::seconds(3));
	}
}

Thanks.

Marco Eichelberg
OFFIS DICOM Team
OFFIS DICOM Team
Posts: 1484
Joined: Tue, 2004-11-02, 17:22
Location: Oldenburg, Germany
Contact:

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#2 Post by Marco Eichelberg »

I would suggest that you use Valgrind to find out where exactly that leak happens.

vignesh
Posts: 14
Joined: Mon, 2020-01-20, 11:44

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#3 Post by vignesh »

thanks for your reply.

Code: Select all

data_ptr = std::make_shared<DcmDataset>(*dataset);
If I have commented out this line, then there are no leaks occurring.

Marco Eichelberg
OFFIS DICOM Team
OFFIS DICOM Team
Posts: 1484
Joined: Tue, 2004-11-02, 17:22
Location: Oldenburg, Germany
Contact:

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#4 Post by Marco Eichelberg »

I have registered this in our issue tracker, see https://support.dcmtk.org/redmine/issues/1082.

Michael Onken
DCMTK Developer
Posts: 2072
Joined: Fri, 2004-11-05, 13:47
Location: Oldenburg, Germany
Contact:

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#5 Post by Michael Onken »

Hi,

I doubt that the leak comes from DCMTK. I used this main(), running 10 iterations only, on a directory with > 1100 DICOM files (up-to-date DCMTK code):

Code: Select all

int main(int argc, char* argv[])
{
	std::string dir_path = "/path/to/images";
	std::vector<fs::path> pathList;
	std::copy(fs::recursive_directory_iterator(dir_path), fs::recursive_directory_iterator(), back_inserter(pathList));
	int i = 1;
	while (i<=10)
	{
		{
			image_data_map_ img_map;
			preparelist(pathList, std::ref(img_map));
		}
		std::cout << "Images are Prepared : " << i << std::endl;
		i++;
		boost::this_thread::sleep_for(boost::chrono::seconds(3));
	}
}
and threw it into valgrind. This is the report:

Code: Select all

❯ valgrind --leak-check=full bin/dcmdump
==465147== Memcheck, a memory error detector
==465147== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==465147== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==465147== Command: bin/dcmdump
==465147== 
Images are Prepared : 1
Images are Prepared : 2
Images are Prepared : 3
Images are Prepared : 4
Images are Prepared : 5
Images are Prepared : 6
Images are Prepared : 7
Images are Prepared : 8
Images are Prepared : 9
Images are Prepared : 10
==465147== 
==465147== HEAP SUMMARY:
==465147==     in use at exit: 112 bytes in 2 blocks
==465147==   total heap usage: 9,280,133 allocs, 9,280,131 frees, 487,824,396 bytes allocated
==465147== 
==465147== LEAK SUMMARY:
==465147==    definitely lost: 0 bytes in 0 blocks
==465147==    indirectly lost: 0 bytes in 0 blocks
==465147==      possibly lost: 0 bytes in 0 blocks
==465147==    still reachable: 112 bytes in 2 blocks
==465147==         suppressed: 0 bytes in 0 blocks
==465147== Reachable blocks (those to which a pointer was found) are not shown.
==465147== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==465147== 
==465147== For lists of detected and suppressed errors, rerun with: -s
==465147== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

~/b/dcmtk_linux>
Thus I would say that a leak in DCMTK is really unlikely, unless your test data triggers something in DCMTK that mine doesn't. As Marco said, try valgrind (Linux) on this and you will find a leak, if there is some, quickly.

Best regards,
Michael

vignesh
Posts: 14
Joined: Mon, 2020-01-20, 11:44

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#6 Post by vignesh »

Hi,

I ran the sample code for 300 iterations using the same 1000 DICOM files

valgrind (Linux) report:

Code: Select all

valgrind --leak-check=full ./leak_check
==76968== Memcheck, a memory error detector
==76968== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==76968== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==76968== Command: ./leak_check
==76968== 
Images are Prepared : 1
Images are Prepared : 2
Images are Prepared : 3
Images are Prepared : 4

Images are Prepared : 298
Images are Prepared : 299
Images are Prepared : 300
==76968== 
==76968== HEAP SUMMARY:
==76968==     in use at exit: 0 bytes in 0 blocks
==76968==   total heap usage: 590,576,324 allocs, 590,576,324 frees, 28,027,059,976 bytes allocated
==76968== 
==76968== All heap blocks were freed -- no leaks are possible
==76968== 
==76968== For lists of detected and suppressed errors, rerun with: -s
==76968== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
In Linux, I don't experience leaks. The leaks might only occur on the Windows platform

Michael Onken
DCMTK Developer
Posts: 2072
Joined: Fri, 2004-11-05, 13:47
Location: Oldenburg, Germany
Contact:

Re: Memory Leak occurring When Inserting 1000CT Datasets in std::map

#7 Post by Michael Onken »

Hi,

I am not aware (at least) of any Windows-specific code in dcmdata's memory management, so I doubt the leak stems from DCMTK. If you can find the reason of the leak, let us know.

Best regards,
Michael

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 1 guest