Mirrors/asmjit
1
0
Fork 0
mirror of https://github.com/asmjit/asmjit.git synced 2025-12-17 12:34:35 +03:00
Code Issues Packages Projects Releases Wiki Activity

Use MFD_CLOEXEC when using memfd_create, documentation update

Browse Source
This commit is contained in:
kobalicek
2023-03-04 12:10:41 +01:00
parent 915186f6c5
commit 76520a513d
4 changed files with 99 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
src/asmjit/core.h
View File

@@ -1633,14 +1633,57 @@ namespace asmjit {
//!
//! ### Overview
//!
//! AsmJit's virtual memory management is divided into two main categories:
//! AsmJit's virtual memory management is divided into three main categories:
//!
//! - Low level API that provides cross-platform abstractions for virtual memory allocation. Implemented in
//! \ref VirtMem namespace.
//! - Low level interface that provides cross-platform abstractions for virtual memory allocation. Implemented in
//! \ref VirtMem namespace. This API is a thin wrapper around operating system specific calls such as
//! `VirtualAlloc()` and `mmap()` and it's intended to be used by AsmJit's higher level API. Low-level virtual
//! memory functions can be used to allocate virtual memory, change its permissions, and to release it.
//! Additionally, an API that allows to create dual mapping (to support hardened environments) is provided.
//!
//! - High level API that makes it very easy to store generated code for execution. See \ref JitRuntime, which is
//! used by many examples for its simplicity and easy integration with \ref CodeHolder. There is also \ref
//! JitAllocator, which lays somewhere between RAW memory allocation and \ref JitRuntime.
//! - Middle level API that is provided by \ref JitAllocator, which uses \ref VirtMem internally and offers nicer
//! API that can be used by users to allocate executable memory conveniently. \ref JitAllocator tries to be smart,
//! for example automatically using dual mapping or `MAP_JIT` on hardened environments.
//!
//! - High level API that is provided by \ref JitRuntime, which implements \ref Target interface and uses \ref
//! JitAllocator under the hood. Since \ref JitRuntime inherits from \ref Target it makes it easy to use with
//! \ref CodeHolder. Many AsmJit examples use \ref JitRuntime for its simplicity and easy integration.
//!
//! The main difference between \ref VirtMem and \ref JitAllocator is that \ref VirtMem can only be used to allocate
//! whole pages, whereas \ref JitAllocator has `malloc()` like API that allows to allocate smaller quantities that
//! usually represent the size of an assembled function or a chunk of functions that can represent a module, for
//! example. \ref JitAllocator then tracks used space of each page it maintains. Internally, \ref JitAllocator uses
//! two bit arrays to track occupied regions in each allocated block of pages.
//!
//! ### Hardened Environments
//!
//! In the past, allocating virtual memory with Read+Write+Execute (RWX) access permissions was easy. However, modern
//! operating systems and runtime environments often use hardening, which typically prohibits mapping pages with both
//! Write and Execute permissions (known as the W^X policy). This presents a challenge for JIT compilers because
//! generated code for a single function is unlikely to fit in exactly N pages without leaving some space empty. To
//! accommodate this, the execution environment may need to temporarily change the permissions of existing pages to
//! read+write (RW) to insert new code into them, however, sometimes it's not possible to ensure that no thread is
//! executing code in such affected pages in a multithreaded environment, in which multiple threads may be executing
//! generated code.
//!
//! Such restrictions leave a lot of complexity on the application, so AsmJit implements a dual mapping technique to
//! make the life of AsmJit users easier. In this technique, a region of memory is mapped to two different virtual
//! addresses with different access permissions. One virtual address is mapped with read and write (RW) access, which
//! is used by the JIT compiler to write generated code. The other virtual address is mapped with read and execute (RX)
//! access, which is used by the application to execute the generated code.
//!
//! However, implementing dual mapping can be challenging because it typically requires obtaining an anonymous file
//! descriptor on most Unix-like operating systems. This file descriptor is then passed to mmap() twice to create
//! the two mappings. AsmJit handles this challenge by using system-specific techniques such as `memfd_create()` on
//! Linux, `shm_open(SHM_ANON)` on BSD, and `MAP_REMAPDUP` with `mremap()` on NetBSD. The latter approach does not
//! require a file descriptor. If none of these options are available, AsmJit uses a plain `open()` call followed by
//! `unlink()`.
//!
//! The most challenging part is actually obtaing a file descriptor that can be passed to `mmap()` with `PROT_EXEC`.
//! This is still something that may fail, for example the environment could be hardened in a way that this would
//! not be possible at all, and thus dual mapping would not work.
//!
//! Dual mapping is provided by both \ref VirtMem and \ref JitAllocator.
//! \defgroup asmjit_zone Zone Memory

2
src/asmjit/core/jitallocator.cpp
View File

@@ -544,7 +544,7 @@ static Error JitAllocatorImpl_newBlock(JitAllocatorPrivateImpl* impl, JitAllocat
if (err)
return err;
else
return kErrorOutOfMemory;
return DebugUtils::errored(kErrorOutOfMemory);
}
// Fill the memory if the secure mode is enabled.

20
src/asmjit/core/virtmem.cpp
View File

@@ -53,7 +53,7 @@
// Android NDK doesn't provide `shm_open()` and `shm_unlink()`.
#if !defined(__BIONIC__)
#define ASMJIT_HAS_SHM_OPEN_AND_UNLINK
#define ASMJIT_HAS_SHM_OPEN
#endif
#if defined(__APPLE__) && TARGET_OS_OSX && ASMJIT_ARCH_ARM >= 64
@@ -332,6 +332,11 @@ public:
Error open(bool preferTmpOverDevShm) noexcept {
#if defined(__linux__) && defined(__NR_memfd_create)
#if !defined(MFD_CLOEXEC)
#define MFD_CLOEXEC 0x0001u
#endif
// Linux specific 'memfd_create' - if the syscall returns `ENOSYS` it means
// it's not available and we will never call it again (would be pointless).
//
@@ -344,7 +349,7 @@ public:
static volatile uint32_t memfd_create_not_supported;
if (!memfd_create_not_supported) {
_fd = (int)syscall(__NR_memfd_create, "vmem", 0);
_fd = (int)syscall(__NR_memfd_create, "vmem", MFD_CLOEXEC);
if (ASMJIT_LIKELY(_fd >= 0))
return kErrorOk;
@@ -356,7 +361,7 @@ public:
}
#endif
#if defined(SHM_ANON)
#if defined(ASMJIT_HAS_SHM_OPEN) && defined(SHM_ANON)
// Originally FreeBSD extension, apparently works in other BSDs too.
DebugUtils::unused(preferTmpOverDevShm);
_fd = ::shm_open(SHM_ANON, O_RDWR | O_CREAT | O_EXCL, S_IRUSR | S_IWUSR);
@@ -391,7 +396,7 @@ public:
return kErrorOk;
}
}
#ifdef ASMJIT_HAS_SHM_OPEN_AND_UNLINK
#if defined(ASMJIT_HAS_SHM_OPEN)
else {
_tmpName.assignFormat(kShmFormat, (unsigned long long)bits);
_fd = ::shm_open(_tmpName.data(), O_RDWR | O_CREAT | O_EXCL, S_IRUSR | S_IWUSR);
@@ -415,7 +420,7 @@ public:
FileType type = _fileType;
_fileType = kFileTypeNone;
#ifdef ASMJIT_HAS_SHM_OPEN_AND_UNLINK
#ifdef ASMJIT_HAS_SHM_OPEN
if (type == kFileTypeShm) {
::shm_unlink(_tmpName.data());
return;
@@ -511,9 +516,9 @@ static bool hasHardenedRuntime() noexcept {
uint32_t flag = globalHardenedFlag.load();
if (flag == kHardenedFlagUnknown) {
uint32_t pageSize = uint32_t(::getpagesize());
size_t pageSize = size_t(::getpagesize());
void* ptr = mmap(nullptr, pageSize, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (ptr == MAP_FAILED) {
flag = kHardenedFlagEnabled;
}
@@ -521,6 +526,7 @@ static bool hasHardenedRuntime() noexcept {
flag = kHardenedFlagDisabled;
munmap(ptr, pageSize);
}
globalHardenedFlag.store(flag);
}

58
src/asmjit/core/virtmem.h
View File

@@ -50,57 +50,71 @@ enum class MemoryFlags : uint32_t {
//! Memory is executable.
kAccessExecute = 0x00000004u,
//! A combination of \ref MemoryFlags::kAccessRead and \ref MemoryFlags::kAccessWrite.
//! A combination of \ref kAccessRead and \ref kAccessWrite.
kAccessReadWrite = kAccessRead | kAccessWrite,
//! A combination of \ref MemoryFlags::kAccessRead, \ref MemoryFlags::kAccessWrite.
//! A combination of \ref kAccessRead, \ref kAccessWrite.
kAccessRW = kAccessRead | kAccessWrite,
//! A combination of \ref MemoryFlags::kAccessRead and \ref MemoryFlags::kAccessExecute.
//! A combination of \ref kAccessRead and \ref kAccessExecute.
kAccessRX = kAccessRead | kAccessExecute,
//! A combination of \ref MemoryFlags::kAccessRead, \ref MemoryFlags::kAccessWrite, and
//! \ref MemoryFlags::kAccessExecute.
//! A combination of \ref kAccessRead, \ref kAccessWrite, and \ref kAccessExecute.
kAccessRWX = kAccessRead | kAccessWrite | kAccessExecute,
//! Use a `MAP_JIT` flag available on Apple platforms (introduced by Mojave), which allows JIT code to be executed
//! in MAC bundles. This flag is not turned on by default, because when a process uses `fork()` the child process
//! has no access to the pages mapped with `MAP_JIT`, which could break code that doesn't expect this behavior.
//! Use a `MAP_JIT` flag available on Apple platforms (introduced by Mojave), which allows JIT code to be
//! executed in a MAC bundle.
//!
//! This flag may be turned on by the allocator if there is no other way of allocating executable memory.
//!
//! \note This flag can only be used with \ref VirtMem::alloc(), `MAP_JIT` only works on OSX and not on iOS.
//! When a process uses `fork()` the child process has no access to the pages mapped with `MAP_JIT`.
kMMapEnableMapJit = 0x00000010u,
//! Pass `PROT_MAX(PROT_READ)` to mmap() on platforms that support `PROT_MAX`.
//! Pass `PROT_MAX(PROT_READ)` or `PROT_MPROTECT(PROT_READ)` to `mmap()` on platforms that support it.
//!
//! \note This flag can only be used with \ref VirtMem::alloc().
//! This flag allows to set a "maximum access" that the memory page can get during its lifetime. Use
//! \ref VirtMem::protect() to change the access flags.
//!
//! \note This flag can only be used with \ref VirtMem::alloc() and \ref VirtMem::allocDualMapping().
//! However \ref VirtMem::allocDualMapping() may automatically use this if \ref kAccessRead is used.
kMMapMaxAccessRead = 0x00000020u,
//! Pass `PROT_MAX(PROT_WRITE)` to mmap() on platforms that support `PROT_MAX`.
//! Pass `PROT_MAX(PROT_WRITE)` or `PROT_MPROTECT(PROT_WRITE)` to `mmap()` on platforms that support it.
//!
//! \note This flag can only be used with \ref VirtMem::alloc().
//! This flag allows to set a "maximum access" that the memory page can get during its lifetime. Use
//! \ref VirtMem::protect() to change the access flags.
//!
//! \note This flag can only be used with \ref VirtMem::alloc() and \ref VirtMem::allocDualMapping().
//! However \ref VirtMem::allocDualMapping() may automatically use this if \ref kAccessWrite is used.
kMMapMaxAccessWrite = 0x00000040u,
//! Pass `PROT_MAX(PROT_EXEC)` to mmap() on platforms that support `PROT_MAX`.
//! Pass `PROT_MAX(PROT_EXEC)` or `PROT_MPROTECT(PROT_EXEC)` to `mmap()` on platforms that support it.
//!
//! \note This flag can only be used with \ref VirtMem::alloc().
//! This flag allows to set a "maximum access" that the memory page can get during its lifetime. Use
//! \ref VirtMem::protect() to change the access flags.
//!
//! \note This flag can only be used with \ref VirtMem::alloc() and \ref VirtMem::allocDualMapping().
//! However \ref VirtMem::allocDualMapping() may automatically use this if \ref kAccessExecute is used.
kMMapMaxAccessExecute = 0x00000080u,
//! A combination of \ref MemoryFlags::kMMapMaxAccessRead and \ref MemoryFlags::kMMapMaxAccessWrite.
//! A combination of \ref kMMapMaxAccessRead and \ref kMMapMaxAccessWrite.
kMMapMaxAccessReadWrite = kMMapMaxAccessRead | kMMapMaxAccessWrite,
//! A combination of \ref MemoryFlags::kMMapMaxAccessRead and \ref MemoryFlags::kMMapMaxAccessWrite.
//! A combination of \ref kMMapMaxAccessRead and \ref kMMapMaxAccessWrite.
kMMapMaxAccessRW = kMMapMaxAccessRead | kMMapMaxAccessWrite,
//! A combination of \ref MemoryFlags::kMMapMaxAccessRead and \ref MemoryFlags::kMMapMaxAccessExecute.
//! A combination of \ref kMMapMaxAccessRead and \ref kMMapMaxAccessExecute.
kMMapMaxAccessRX = kMMapMaxAccessRead | kMMapMaxAccessExecute,
//! A combination of \ref MemoryFlags::kMMapMaxAccessRead, \ref MemoryFlags::kMMapMaxAccessWrite, \ref
//! MemoryFlags::kMMapMaxAccessExecute.
//! A combination of \ref kMMapMaxAccessRead, \ref kMMapMaxAccessWrite, \ref kMMapMaxAccessExecute.
kMMapMaxAccessRWX = kMMapMaxAccessRead | kMMapMaxAccessWrite | kMMapMaxAccessExecute,
//! Use `MAP_SHARED` when calling mmap().
//!
//! \note In some cases `MAP_SHARED` may be set automatically. For example when using dual mapping it's important to
//! to use `MAP_SHARED` instead of `MAP_PRIVATE` to ensure that the OS would not copy pages on write (that would mean
//! updating only the RW mapped region and not RX mapped one).
//! \note In some cases `MAP_SHARED` may be set automatically. For example, some dual mapping implementations must
//! use `MAP_SHARED` instead of `MAP_PRIVATE` to ensure that the OS would not apply copy on write on RW page, which
//! would cause RX page not having the updated content.
kMapShared = 0x00000100u,
//! Not an access flag, only used by `allocDualMapping()` to override the default allocation strategy to always use