Improved standards support for aligned memory allocators
keith keith To clarify, some potential issues I've noted in the current mingw-aligned-malloc.c implementation:–
between the c and the p elements, i.e.:–
it should be:–
and more importantly, (because that correction reduces the allocation request by only one byte), instead of:–
the alignment calculation should be:–
in which case, the -1 adjustment is much more critical — it ensures that the alignment boundary is correctly rounded downwards, avoiding a potential over-alignment by one entire alignment frame higher in memory, which would open the door to a buffer overrun, when the buffer is populated from an address which is one such frame address higher in memory than has been allowed for, in the allocation.
Besides the above, there are other issues, for a follow-on comment.
keith)
keith Following on from my preceding comment, (which ran out of space), further issues with the existing mingw-aligned-malloc.c implementation:–
(or corrected, as in the preceding comment):–
is a necessary condition of compliance with the stated constraint, it is not a sufficient condition. Furthermore, since the original alignment and offset parameters are not stored, (nor is any provision made for the possibility of storing them), there simply isn't sufficient information available to __mingw_aligned_offset_realloc(), to formulate a necessary and sufficient conditional test.
keith With regard to the suggested addition of ISO-C11's aligned_alloc(), and/or POSIX.1's optional posix_memalign() APIs, either or both could be trivially implemented in terms of __mingw_aligned_malloc(), were it not for one limitation of the latter's current implementation — a limitation shared with Microsoft's _aligned_malloc() implementation: both of these implementations require accompanying (non-standard) _aligned_realloc() and _aligned_free() counterparts, which must be used when operating on the over-aligned heap, and cannot be used to operate on the regular heap. Neither ISO-C11, nor POSIX.1 require, or specify, any such complementary APIs; it is expected that regular realloc() and free() APIs are sufficient, regardless of heap block alignment.
The __mingw_aligned_free() API could be modified, to permit it to operate on over-aligned heap blocks, (allocated by __mingw_aligned_malloc() or __mingw_aligned_offset_malloc()), and on regularly aligned blocks, (allocated by malloc()); such a modified implementation could then be used in place of free(). As it stands, such an extended implementation of free() would require a "heap walk", to discriminate between regularly aligned and over-aligned blocks; this requirement precludes its deployment on Win9x, (because the _heapwalk() API is documented to be unsupported, in all but name).
A complementary extension of the __mingw_aligned_realloc() API would not be possible, unless the existing implementation is adapted to record alignment and offset attributes of over-aligned heap blocks, within the block prefix ... a modification which I identified, in my preceding comment, as a likely requirement anyway. Such a modification could obviate the need for a heap walk, to discriminate between regular and over-aligned heap blocks; it could also overcome the limitation of missing Win9x support.
keith I've been exploring a possible replacement implementation for the entire aligned heap allocator; in the process, I've just identified a further critical defect in the current implementation, viz. for both _aligned_offset_malloc(), and _aligned_offset_realloc(), Microsoft's documentation states:
_aligned_offset_malloc validates its parameters. If alignment is not a power of 2 or if offset is greater than or equal to size and nonzero, this function invokes the invalid parameter handler
The existing MinGW implementation correctly checks the power of two constraint on alignment, but it seems to completely ignore the (obvious, in spite of Microsoft's awkwardly obscure wording) offset constraint; with my modified implementation, which does honour both constraints, the original test program fails instantly, (where it presumably passed, but should not have, with the original implementation).
keith keith keith I've attached a patch which reimplements the MinGW alternative to Microsoft's _aligned_malloc() (and complementary) API, and also an accompanying test suite. This corrects the deficiencies I've noted, in the previous MinGW implementation; it also offers the following enhancements, (unlikely to be supported by Microsoft's implementation), over the previous MinGW implementation:–
Unless there are significant objections, I propose incorporating this alternative implementation of the __mingw_aligned_malloc() API, into the wsl-5.2 release.
keith)
keith keith keith keith keith keith Following a brief dialogue, on the project contributors (private) forum, I've updated the implementation to provide:–
each of which has been implemented with a capability to detect whether its void * argument refers to memory allocated by a function providing default malloc() alignment, or by any of the functions offering __mingw_aligned_malloc(), (but *not* Microsoft's _aligned_malloc()), extended alignment, and to proceed accordingly. Additionally, for congruence with both Microsoft's and our own earlier implementation, this reimplementation continues to provide (modified versions of):–
Do please note that, while this latter function set is semantically equivalent to its Microsoft counterpart, it is not binary compatible; additionally, void __mingw_aligned_free( void * ); has become redundant — it is now, in fact, implemented as a trivial alias for void __mingw_free( void * );
I have updated the attached patch files, to reflect this revised implementation.
keith Following on from my preceding comment, astute readers may observe that, since
can handle both malloc() alignment, and __mingw_aligned_malloc() alignment, and since their signatures are semantically compatible, they could be mapped as redirection targets for calls to
respectively, thus providing a suitable foundation for implementations of an ISO-C11 compatible
and a POSIX.1 compatible
in terms of simple (potentially in-line) wrappers around
keith keith keith keith I replaced the primary reimplementation patch file, to incorporate some small (but necessary) corrective changes. I've also added a tentative implementation for both ISO-C11's aligned_alloc() function, and POSIX.1-2001's posix_memalign() function.
I am inclined to push the reimplementation of the existing API, together with the accompanying test suite, without further delay, but to defer pushing the new additions, pending peer review and comment.
keith I committed #33a73a6 and #68d0570. The former reimplements the original MinGW aligned heap management API, correcting the identified deficiencies in the original implementation, retaining each of the original API functions, and adding __mingw_realloc() and __mingw_free() as discussed; the latter implements a new test suite for the API, replacing the original test program, (which turned out to be not particularly useful).
This reimplementation of the aligned heap management API will be incorporated into the next WSL release. At this stage, I have not committed any implementation for the proposed ISO-C11 aligned_alloc() and POSIX.1 posix_memalign() functions; I am deferring these, pending peer review, and user demand.
keith)
keith keith I've simplified the previous implementation of both the ISO-C11 aligned_alloc(), and the POSIX.1-2001 posix_memalign() functions, and separated them into two individual patches; please note that the latter is dependent on the former, to the extent that the former substitutes __mingw_free() and __mingw_realloc() for free() and realloc() respectively.
keith As of mingwrt-5.3.1, published on 30-Apr-2020 as a component of wsl-5.3.1, both C11's aligned_alloc(), and C++17's corresponding std::aligned_alloc() are now supported OOTB, as is POSIX.1-2001's posix_memalign().
Fork
Microsoft introduced _aligned_malloc(), and associated functions, with MSVCR70.DLL. Although subsequently supported in MSVCRT.DLL, from WinXP onwards, exposure of these APIs interferes with a clean build of GCC — not only insofar as, having detected presence of the APIs, GCC would become dependent on WinXP and later, thus needlessly breaking legacy support, but furthermore, the GCC sources neglect to include the requisite <malloc.h> header file, and thus do not build cleanly.
Legacy support for similar APIs was added to MinGW, in 2003/2004, under feature request #260; however, it may be ill-advised to make GCC dependent on these MinGW specific APIs, for the following reasons:—
Consequently, I suggest:—