チケット #40686

std::string custom operator new/delete function

登録: 2020-08-30 18:32 最終更新: 2021-04-16 06:52

報告者:
担当者:
(未割り当て)
チケットの種類:
状況:
完了
コンポーネント:
マイルストーン:
(未割り当て)
優先度:
1 - 最低
重要度:
1 - 最低
解決法:
動いてるけど?
ファイル:
なし
投票
点数: 0
No votes
0.0% (0/0)
0.0% (0/0)

詳細

Hello,

I am writing a custom wrapper for new/delete allocations to get an aligned memory. I noticed that if I use std::string, operator new/delete are not called as it is in a linux version of g++ (operators new/delete are called). With the default optimization level, operator new/delete is not called and with optimization levels O1, O2, O3 only operator delete is called.

Source:

  1. #include <string>
  2. #include <cstdio>
  3. #include <cstdlib>
  4. void* operator new(size_t size)
  5. {
  6. printf("Calling new: %zu", size);
  7. void* address = malloc(size);
  8. printf(" address: 0x%p\n", address);
  9. return address;
  10. }
  11. void* operator new[](size_t size)
  12. {
  13. printf("Calling new[]: %zu", size);
  14. void* address = malloc(size);
  15. printf(" address: 0x%p\n", address);
  16. return address;
  17. }
  18. void operator delete(void* address)
  19. {
  20. printf("Calling delete, address: 0x%p\n", address);
  21. free(address);
  22. }
  23. void operator delete[](void* address)
  24. {
  25. printf("Calling delete[], address: 0x%p\n", address);
  26. free(address);
  27. }
  28. struct Obj
  29. {
  30. std::string m_str;
  31. };
  32. void test()
  33. {
  34. Obj obj;
  35. obj.m_str = "long string ......................";
  36. }
  37. int main()
  38. {
  39. test();
  40. return 0;
  41. }

Compiling with the default optimization level:

  • g++ main.cpp -std=c++11 -o test.exe

Output:

  • NONE

Compiling with the O2 optimization level:

  • g++ main.cpp -std=c++11 -O2 -o test.exe

Output:

  • Calling delete, address: 0x00542098

I am using:

  • OS: Windows 7
  • Mingw: 5.4.1
  • Shell: cmd MINGW32_NT-6.1 MARTIN-LENOVO 1.0.18(0.48/3/2) 2012-11-21 22:34 i686 Msys
  • Uname -a:
  • G++: g++ (MinGW.org GCC Build-2) 9.2.0
  • GCC: gcc (MinGW.org GCC Build-2) 9.2.0
  • ld: GNU ld (GNU Binutils) 2.32

チケットの履歴 (8 件中 3 件表示)

2020-08-30 18:32 更新者: marvol
  • 新しいチケット "std::string custom operator new/delete function" が作成されました
2020-09-02 22:10 更新者: keith
  • 優先度5 - 中 から 1 - 最低 に更新されました
  • 重要度5 - 中 から 1 - 最低 に更新されました
  • 解決法なし から 動いてるけど? に更新されました
コメント

This is an issue which really would have been better to raise on the mailing list, where it would be visible to a broader spectrum of experienced users. My own expertise is predominantly in FORTRAN-77 and C programming; my knowledge of C++ is sketchy, at best. Nonetheless, I will try to answer, to the best of my ability.

Firstly, let me make it perfectly clear: any comparison between behaviour on Windows, and behaviour on Linux, is completely irrelevant; the two platforms are fundamentally different, and there are many pitfalls in migrating from either one to the other. In this case, I believe you are falling foul of one of those pitfalls; there is no bug here, beyond a gap in your understanding.

In your test case, while you define replacements for both new and delete, I see no explicit use of either; any use which does arise is implicit in the instantiation, and subsequent destruction, of the std::string object, to which you refer. Instantiation of that object, apparently, does invoke new, and destruction thus invokes delete, but consider this: where is the implementation of that std::string class object defined? I'll give you some time to think about that, then I'll suggest, in follow-up comments, how the answer may explain the behaviour you have observed.

2020-09-03 15:31 更新者: marvol
コメント

Thank you for your reply.

It is possible that I am missing something. I understand that std::string can have own implementation of allocation of the data. I think that in the current libstdc++ implementation if a length of the string is greater than 15 bytes, then it uses the heap for allocation.

On line 43, there is an assignment operator for the string and this string is long enough to allocate string data on the heap. On the next line the object "obj" ends its lifetime so "obj" destructor is called, also with std::string destructor which releases the string data.

For me it is strange that there is no 1 to 1 relation between calling operator new/delete (how should I release memory that was not allocated in my allocator?). I tried other containers like std::vector and they seem to be consistent.

2020-09-03 21:11 更新者: keith
コメント

You need to think about it some more. You are correct, in your belief that new should be called ... indeed, it is, but it isn't your replacement that's even considered. Think about where the std::string implementation is, and what that implies for where it must find the implementation of new that it does call.

2020-09-04 16:45 更新者: marvol
コメント

So, you think it is because the implementation of the std::string is in an external dll? It may explain that my operator new was not called.

Indeed, If I compile it with the "-static" linker option my operator new/delete is called. But why was my operator delete called? Is it because the compiler optimized the std::string destroy function to be inlined?

I am thinking about the experience that I had with std::vector - maybe it is because the std::vector is a template that is compiled with the source.

2020-09-04 19:57 更新者: keith
コメント

Reply To marvol

So, you think it is because the implementation of the std::string is in an external dll?

Yes ... in fact, I know it to be so.

It may explain that my operator new was not called.

It does. There is a fundamental, and significant, difference between the properties of ELF shared objects, as used on Linux, and the equivalent PE-coff DLLs, as used on Windows. Do you know what it is?

Indeed, If I compile it with the "-static" linker option my operator new/delete is called.

Actually, -static-libstdc++ is sufficient.

But why was my operator delete called? Is it because the compiler optimized the std::string destroy function to be inlined?

Yes. You can study the effect of differing optimization levels, by compiling with -S -o-, and examining the assembly output. At -O1 (and above), your Obj class object is optimized away, and your test function includes exactly one reference to a std::string method — likely a copy constructor — and then your delete operator code is called; at -O2 (and -O3) even the delete call is optimized away, and your code is in-lined. At no optimization level, is your new operator ever called, but it is likely that whatever new operator implementation std::string knows about is called, (by what I assume to be its cryptically named copy constructor). That, in itself, is disturbing, because you have a potentially incompatible new and delete pairing, at any optimization level of -O1 or above.

I am thinking about the experience that I had with std::vector - maybe it is because the std::vector is a template that is compiled with the source.

Maybe. You haven't shown me anything, which I might evaluate.

2020-09-05 17:07 更新者: marvol
コメント

Reply To keith

Thank you for the explanation!

Maybe. You haven't shown me anything, which I might evaluate.

Actually, the code is similar, just include vector

  1. #include <vector>

and replace the "test" function:

  1. void test()
  2. {
  3. std::vector<int> vec;
  4. vec.push_back(42);
  5. printf("vec[0]: %d\n", vec[0]);
  6. }

My operator new/delete is called in all optimization levels.

2021-04-16 06:52 更新者: keith
  • 状況オープン から 完了 に更新されました

添付ファイルリスト

添付ファイルはありません

編集

このチケットにコメントを追加するには、ログインが必要です » ログインする