Release Notes for Version 2.0.0

This section is divided into the following subsections:

• Distribution Contents

• Changes Since Prior Releases

• Limitations

• Plans for Future Releases

Distribution Contents

The following are part of the DynamoRIO release distribution:

• Four different DynamoRIO libraries: debug and release for each of 32-bit and 64-bit. The debug library enables assertion messages to more easily diagnose API usage errors.
• The DynamoRIO configuration and execution libraries and command-line utilities drconfiglib.dll, drinjectlib.dll, drconfig.exe, drrun.exe, and drinject.exe. Windows package only.
• A utility drview.exe for viewing which processes are running under DynamoRIO control. Windows package only.

• The drconfig, drrun, and drinject scripts for configuring and running applications under DynamoRIO on Linux (Linux package only).

• Header files for the DynamoRIO APIs.
• This documentation.
• Sample clients.

• A graphical statistics viewer DRgui.exe that displays internal DynamoRIO statistics as well as custom statistics defined by a client (see Use of Custom Client Statistics with the Windows GUI). Windows package only. DynamoRIO exports a large number of statistics in its debug build, and a handful in release builds.

• A binary tracedump reader, which also functions as a sample client using DynamoRIO as a standalone library (see Using DynamoRIO as a Standalone Library).

Changes Since Prior Releases

The current version is 2.0.0. Its client API is backward compatible with releases from 1.0.0 (originally called 0.9.6: see below) onward. However, the configuration and deployment API and tools have changed and are not backward compatible: see below for details.

The changes between version 2.0.0 and 1.5.0 are:

• Changed the configuration and deployment model for both Linux and Windows to use a configuration file based approach on both platforms, which adds control over child processes on Linux and supports local configuration on Windows for un-privileged and parallel execution (Issue 265). The registry is no longer used for individual application configuration on Windows other than to point at the location for global configuration files, when used.
  IMPORTANT: On Windows the following non-backward-compatible changes have been made:
  • drdeploy.exe no longer exists (replaced by drconfig.exe and drrun.exe)
  • drconfig.dll is now drconfiglib.dll
  • drconfiglib.dll's API routines now take in a process id to support one-time targeted-process configuration (to support parallel execution)
  • configuration is either per-user or global, with per-user taking precedence when both exist
  • configuration does NOT enable systemwide injection by default: use the -syswide_on parameter to drconfig or drrun for that (it requires administrative privileges)
  IMPORTANT: On Linux, if you're using custom injection via raw environment variables rather than using the drdeploy script, you must also set DYNAMORIO_RUNUNDER to 1 for injection to work with this release.
• Added drinjectlib.dll and dr_inject.h, making custom injection tools easier to build (Issue 246)
• Added DynamoRIO Extension support for auxiliary libraries that extend the DynamoRIO API (Issue 277)
• Added symbol lookup support via Extension (Windows only for now) (Issue 44)
• Added a "drcontainers" Extension that initially contains a hashtable
• Added thread creation support: dr_create_client_thread() (Issue 41)
• Added dr_sleep()
• Added dr_set_itimer() and dr_get_itimer() (Linux-only) (Issue 283)
• Added dr_app_pc_for_decoding()
• Added -synch_at_exit option and dr_request_synchronized_exit() to provide guarantees over thread exit event timing in release build
• Added instr_cmovcc_triggered() and instr_cmovcc_to_jcc()
• Renamed OP_fcmovene to OP_fcmovne
• Implemented instr_invert_cbr() for OP_jcc_short
• Added the full path to modules in module_data_t
• Added dr_get_proc_address_ex() to support indirect code objects
• Added dr_get_milliseconds() and dr_get_time() impl for Linux
• Added instr_is_undefined()

The changes between version 1.5.0 and 1.4.0 are:

• Added a private loader on Windows for better support for library usage by clients
• Added nudge support on Linux
• Added dr_suspend_all_other_threads() and dr_resume_all_other_threads()
• Made it easier for clients to use faults to push rare events out of instrumentation paths:
  • Added access to the pre-translated context and the code fragment information for both dr_register_signal_event() and dr_register_exception_event() This changed the return type for exception event callbacks.
  • Added a signal/exception event on a fault in non-code-cache DynamoRIO code, such as client-generated code.
  • Added the "meta-instruction that can fault" property via instr_is_meta_may_fault(), instr_set_meta_may_fault(), instrlist_meta_fault_preinsert(), instrlist_meta_fault_postinsert(), and instrlist_meta_fault_append().
  • Added a new event dr_register_restore_state_ex_event() that provides the pre-translated context and code fragment information, and allows for translation failure for non-fault translations.
• Added dr_dup_file_handle()
• Added dr_memory_is_dr_internal() and dr_memory_is_in_client()
• Added dr_get_parent_id()
• Added decode_opcode_name()
• Removed the deprecated snprintf() as it causes symbol pre-emption problems on Linux. Older clients should switch to dr_snprintf().
• Fixed bug in cross-architecture execve (Issue 146)
• Clone record is now passed via dstack instead of ebp (Issue 149)
• Fixed close() syscall handling, !HAVE_TLS assert & minor issues (Issue 151)

The changes between version 1.4.0 and 1.3.2 are:

• Added directly-addressable thread-local storage slots for exclusive client use:
  • dr_raw_tls_calloc()
  • dr_raw_tls_free()
• Provide 64-bit versions of the drdeploy.exe and drview.exe tools
• Provide dr_get_proc_address() on Linux
• Added dr_query_memory_ex() to allow address space walking on Linux
• Added -msgbox_mask on Linux: waits for a keypress
  • Added STDIN and dr_get_stdin_file()
• Added shared library versioning on Linux
• Support calling dr_get_mcontext() from bb and trace callbacks
• Provide support for building clients using CMake (see cmake/*, and for an example of usage see samples/CMakeLists.txt)
• Provide support for clients to use -fvisibility by setting the define USE_VISIBILITY_ATTRIBUTES for dr_defines.h
• Added instr_compute_address_ex() for instrs with multiple memory operands
• Provide dr_snprintf() (identical to snprintf()) for consistent naming and to avoid gcc warnings about using pointers with x (which we're using because there is no portable way to precisely control p)
• The statistics viewer DRgui.exe is no longer supported on Windows NT. Statistics still work, but the graphical application itself will not run on NT.
• Changed the top-level registry key to "DynamoRIO"
• Re-arranged layout of bin and samples directories
• Symbols for all binaries are now included

The changes between version 1.3.2 and 1.3.1 are:

• Added support for Linux execve of cross-architectural executables (e.g., 32-bit process performing execve of 64-bit executable)
  • Also, libdrpreload.so is now kept in the same libXX/{debug,release} directory as libdynamorio.so
• instr_convert_short_meta_jmp_to_long() now returns the longer version of the taken jump, to use when setting the target of a jecxz or loop* branch.
• Various bug fixes including in these areas:
  • dr_syscall_set_result() and dr_syscall_invoke_another()
  • 64-bit drinject stack alignment
  • 64-bit erroneous assert in dr_get_process_id()
  • 64-bit dr_file_{tell,seek} worked but returned failure
  • -opt_memory bugs resulting in asserts
  • sigprocmask() corner case bug
  • signal handler sharing for NPTL threads
  • decoding across page boundaries on Linux

Version 1.3.1 is identical to 1.3.0 but is under a BSD license (see License).

We re-numbered the previous DynamoRIO versions as follows:

• 1.0.0 = 0.9.6 build 9600
• 1.1.0 = 0.9.6 build 9601
• 1.2.0 = 0.9.6 build 9602

The changes between version 1.3.0 and version 1.2.0 (0.9.6 9602) are:

• Version numbering shift to 1.x.y instead of 0.9.6 960x
• New system call pre, post, and filter events, and new system call parameter and result access, along with a feature to chain system calls:
  • dr_register_pre_syscall_event()
  • dr_register_post_syscall_event()
  • dr_register_filter_syscall_event()
  • dr_syscall_get_param()
  • dr_syscall_set_param()
  • dr_syscall_set_sysnum()
  • dr_syscall_get_result()
  • dr_syscall_set_result()
  • dr_syscall_invoke_another()
  • dr_is_wow64()
• New signal event for Linux
  • dr_register_signal_event()
• New option -pause_on_error, and error messages to stdout by default for release builds, to improve Linux debugging
• New routines for memory allocation and memory manipulation:
  • dr_nonheap_alloc()
  • dr_nonheap_free()
  • dr_memory_protect()
• New option -syntax_intel for Intel-style disassembly
• New option -sysenter_is_int80
• The parameter to an application's system call (normally kept in the eax register) can now be freely changed in basic blocks on all platforms
• Added support for 64-bit -thread_private
• Added corner-case undocumented IA-32 opcode support
• Fixed bug running multi-threaded 64-bit Linux apps
• Fixed bugs in 64-bit Linux signal handling
• Fixed bug running -thread_private debug build
• Fixed bug running 32-bit apps on 64-bit Linux on AMD processors
• Fixed bug where OS_OPEN_APPEND overwrote instead of appending on Windows

The changes between the 0.9.6 release builds 9602 and 9601 are:

• Performance improvements for both the base DynamoRIO system and for client instrumentation when running on Pentium M, Core, and Core 2 processors.
• 64-bit bug fixes
• Added several convenience routines:
  • get_register_name()
  • reg_to_pointer_sized()
  • reg_is_gpr()
  • reg_is_segment()
  • reg_32_to_8()
• Disassembly now expands immed sizes to match operands
• Fixed bug in instr_is_nop()

The changes between the 0.9.6 release builds 9601 and 9600 are:

• The Windows registry key used is now "VMware, Inc." instead of "VMware"
• Added large file support (see DR_FILE_ALLOW_LARGE)
• Added support for decoding from a copy of code: decode_from_copy() and disassemble_from_copy().
• Changed the default options to favor performance, and added the -opt_memory runtime option to prioritize memory instead.

Release 0.9.6 is not backward compatible with prior releases 0.9.1-0.9.5.

The major changes between the 0.9.6 and 0.9.5 releases include 64-bit support, multiple clients, state translation, trace contents, and Linux module events and fast system calls:

• 64-bit applications and clients are now supported. This changed several function signatures:
  • instr_encode()
  • decode_next_pc()
  • decode_sizeof()
  • decode_eflags_usage()
  • instr_init()
  • The binary trace dump format changed.
  Several new functions were added:
  • set_x86_mode()
  • get_x86_mode()
  • instr_set_x86_mode()
  • instr_get_x86_mode()
  • opnd_create_rel_addr()
  • opnd_create_far_rel_addr()
  • opnd_is_rel_addr()
  • opnd_is_near_rel_addr()
  • opnd_is_far_rel_addr()
  • instr_has_rel_addr_reference()
  • instr_get_rel_addr_target()
  • instr_get_rel_addr_dst_idx()
  • instr_get_rel_addr_src_idx()
  • instr_shrink_to_32_bits()
  • opnd_shrink_to_32_bits()
  • reg_32_to_64()
  • reg_64_to_32()
  • reg_is_extended()
  • reg_parameter_num()
  To build a 64-bit client, set the X86_64 preprocessor define before including the DynamoRIO header files, and link with the 64-bit build of DynamoRIO (for a 32-bit client, set X86_32).
• Multiple clients are now supported. This changed the signatures of dr_init(), dr_get_options(), and dr_get_client_path(). It also changed how clients are deployed and nudged, and how events are unregistered: explicit unregistration routines are now used.
• State translation in the presence of clients is now fully supported. This changed the signature for the basic block and trace event callbacks (see dr_register_bb_event() and dr_register_trace_event()), added a new event dr_register_restore_state_event(), and added new functions instr_set_translation(), instr_set_meta_no_translation(), and INSTR_XL8().
• The trace callback (dr_register_trace_event()) now presents original application code to the client, rather than code that has already been modified for execution in the code cache. The client also has flexibility in which instrumentation is included from constituent basic blocks added to a trace (the for_trace parameter: see dr_register_bb_event()).
• Fast system calls (syscall and sysenter) are now supported on Linux.
• Module load/unload events and module iteration are now supported on Linux.
• System calls for 32-bit applications on 64-bit kernels are no longer hidden by vsyscall; related functions were removed: instr_is_lol64_syscall(), instr_is_32on64_syscall().
• Due to both 64-bit support and full WOW64 (32-bit applications on 64-bit Windows) support, xmm registers were added to dr_mcontext_t, and a new function dr_mcontext_xmm_fields_valid() was added.
• Far instr-type operands are now supported: opnd_create_far_instr(), opnd_is_near_instr(), opnd_is_far_instr().
• Miscellaneous new functions were added:
  • instr_convert_short_meta_jmp_to_long()
  • instr_reads_from_reg()
  • LOCK()
  • OPND_CREATE_INT_32OR8()
  • OPND_CREATE_INT_16OR8()
  • instrlist_meta_append()
  • dr_using_all_private_caches()
• The type of nudge arguments was changed from (void *) to uint64.
• The signature of dr_lookup_module() changed. It no longer has an IMAGE_SECTION_HEADER out argument. See dr_lookup_module_section() for that functionality.
• The disassemble-from-address routines now return NULL when pointed at invalid instructions (matching the decode routines).
• The routines to access DynamoRIO tls slots from the cache were changed. dr_insert_write_temp_tls was eliminated in favor of a generalized dr_save_reg with more flexibility on which slot can be used. dr_save_arith_flags was similarly generalized. Slots are now guaranteed to remain valid until the next non-meta instruction allowing access to saved registers during clean calls via dr_read_saved_reg and dr_write_saved_reg. dr_insert_mbr_instrumentation also now requires caller to specify the spill slot to be clobbered which must be less than dr_max_opnd_accessible_spill_slot().

The major changes between the 0.9.5 and 0.9.4 releases are:

• The basic block hook (Basic Block Creation) passes completely unmodified application code to the client (no mangling or elision).
• The old client hook exports have been replaced with an explicit event registration model.
• Instruction representation simplification: the client only sees fully decoded instructions.
• Easier-to-use clean calls (see dr_insert_clean_call).
• Library support (-wrap on linux, ntdll on windows: see Using External Libraries and Common Utilities).
• Some features have been removed (these are easily implemented by a client): there is no more edge-counting profile build, no more custom exit stubs, and no more prefixes.
• Infrastructure improvements:
  • Thread-shared caches (can still request thread-private: -thread_private option). Note that there are some subtle changes stemming from using thread-shared: in particular, note that the context passed to the deletion event may be NULL (see dr_register_delete_event).
  • Direct access to TLS slots (dr_save_reg, dr_insert_write_temp_tls, dr_insert_write_tls_field).
• Module events (dr_register_module_load_event), module iteration (dr_module_iterator_start, dr_lookup_module, etc.), and memory querying (dr_query_memory, dr_virtual_query).
• The full API is now documented in html and pdf for easy browsing.
• Numerous type and routine name changes.

Limitations

Client Limitations

The most important limitation on a client is that it remain transparent. This is described fully in Client Transparency. Here we summarize the key points to remain transparent:

• For full transparency, the client should be a self-contained library with linkage to nothing other than DynamoRIO libraries. On Windows we provide private loading that makes some use of system libraries safe, but global resource conflicts can still occur and clients are cautioned from using system-interacting library routines. On Linux we recommend using only static linking for now. See Using External Libraries for further details.
• Currently communication API provided by DynamoRIO is limited to file I/O and nudges.

Platform Limitations

• DynamoRIO currently supports the following NT-based 32-bit Windows systems : NT (all service packs), 2000 (all service packs), XP (32-bit, service packs 0-3), 2003 (32-bit, service packs 0-2), and Vista (32-bit, service packs 0-1). It does not support Windows 95, 98, ME or Server 2008.
• This release of DynamoRIO has limited support for running 32-bit Windows applications on the following 64-bit Windows operating systems : Windows XP Professional x64 (service pack 2), Windows 2003 x64 (service pack 2), Vista x64 (service packs 0-1). Only the 32-bit code will be seen, and children processes created will not be injected into. On 64-bit Windows 32-bit applications are automatically run through the Windows-On-Windows or WOW64 emulator so system call and indirect call processing clients must be aware of instr_is_wow64_syscall().
• This release of DynamoRIO supports running 64-bit Windows applications, using the 64-bit DynamoRIO build, on the following 64-bit Windows systems: Windows XP Professional x64 (service pack 2), Windows 2003 x64 (service pack 2), Vista x64 (service packs 0-1). It does not support any Itanium based Windows systems. For 64-bit applications clients must be compiled for 64-bit with a fixed preferred address in the lower 2GB of the address space (e.g., with the link options /base:"0x72000000" /FIXED), and multiple clients with conflicting base addresses aren't supported. See ../../cmake/DynamoRIOConfig.cmake for an example. The target application may fail to initialize properly or load DynamoRIO if there is an address conflict with the preferred bases of the client or DynamoRIO dlls, but given the choice of bases and the fact that DynamoRIO is loaded quite early we expect this to only very rarely be a problem. This limitation will be addressed in future releases.
• This release does not fully support applications that mix 32-bit and 64-bit code. Future releases will support such mixtures.
• When running a cygwin application under control of DynamoRIO, stderr and stdout output from DynamoRIO or its clients may not be visible.
• The statistics viewer DRgui.exe is no longer supported on Windows NT. Statistics still work, but the graphical application itself will not run on NT.

• This release of DynamoRIO has support for most 32-bit Linux distributions running on Intel-compatible hardware, including Ubuntu 7.04, Ubuntu 8.04, and Fedora 9. It may not support applications using segment registers beyond the standard threading library usage. Future releases will match the Windows version in comprehensive support.
• This release of DynamoRIO has support for running 32-bit Linux applications on 64-bit Linux operating systems on AMD64-compatible hardware.
• This release of DynamoRIO has support for running 64-bit Linux applications, using the 64-bit DynamoRIO build. Similarly to 64-bit Windows, clients must set a preferred base address in the lower 2GB of the address space, and multiple clients with conflicting base addresses aren't supported. See Building a Client. This limitation will be addressed in future releases.
• Cross-architecture execve (e.g., a 32-bit process performing execve of a 64-bit executable) may stop working if the paths to the libdynamrio.so and libdrpreload.so libraries are renamed.

Performance Limitations

• In order to present a more straightforward code stream to clients, DynamoRIO has several optimizations disabled in this release. System-call-heavy applications are the ones most likely to be affected. Future releases may allow clients to choose performance versus visibility. This release does provide the -opt_memory option to enable prioritizing memory usage.
• The performance when starting up large desktop applications may be noticeably worse than native. Upcoming releases will address this issue.

• The performance when running Java, .Net, Flash or similar managed execution applications can be noticeably worse then native. This can include applications that load these components as in-process plugins (such as Internet Explorer). Upcoming releases will address this issue.

• When using DynamoRIO on all or many processes on a system simultaneously, memory usage may become a factor. Upcoming releases will address this issue.

Deployment Limitations

• The dr_config.lib library is not multi-thread safe. Users of the library should ensure that no more then one thread accesses the library at a time.
• Other installed software that uses hooks may not always be interoperable with DynamoRIO.
• Other installed software may conflict with DynamoRIO's use of the \HKLM\SOFTWARE\Microsoft\Windows\WindowsNT\CurrentVersion\AppInit_DLLs registry key (only very rarely an issue).
• Cygwin processes do not work with DynamoRIO due to cygwin's implementation of fork being incompatible with DynamoRIO.
• Though DynamoRIO supports running on Windows NT, the provided installer will only run on Windows 2000 and higher. After installation on a 2000 or higher machine the Distribution Contents may be copied to the NT machine for use there.
• On Windows NT, a reboot is required after the initial dr_register_process() in order for DynamoRIO to take control of any applications. The reboot will not change or undo the registration.
• A Windows application that does not statically link with user32.dll will not be run under control of DynamoRIO unless its parent process (typically cmd.exe or explorer.exe, for manually launched applications) is already under DynamoRIO control or the drinject.exe utility is used to launch the application. Only some small non-graphical applications do not link with user32.dll.
• When invoking an application in any way that is not from a parent process under DynamoRIO control, DynamoRIO takes control a little later and in some rare cases the application has already created a new thread. This is in violation of the Windows specficiations, but cygwin processes do this. This new thread will not be under DynamoRIO control and can cause a variety of problems. In such cases it is best to invoke from a parent process under DynamoRIO control. However, for 32-bit applications on 64-bit Windows operating systems, DynamoRIO takes over later even from the parent. Future releases will address this limitation.
• This release does not support running some Windows services under control of DynamoRIO: System, smss.exe, csrss.exe, and protected processes on Windows Vista.
• This release does not support nudging 64-bit processes.
• On Windows Vista most services are run in a separate session as a security feature. This means that neither DynamoRIO nor its client will be able to display a messagebox when running in said services (they have no associated visible window station). See dr_messagebox().
• On Windows Vista the code from certain dlls (found mostly in services) and the code subsequently called from those dlls is run natively and is not visible to the instrumentation APIs. This only applies to dlls that have a .pexe section (only 13 dlls have such a section is a standard Vista install) which seems to be associated with a likely obfuscation method involving kernel side components that this release has no other workaround for.

• On Linux, the only provided deployment method in this release (aside from the app_start() app_stop() interface, which requires soure code modification) is LD_PRELOAD. This means that static binaries cannot be injected into. Also, suid-root binaries disallow absolute paths in LD_PRELOAD and ignore LD_LIBRARY_PATH, so alternate methods of specifying the path are needed there. One method is to place libdrpreload.so's full path in /etc/ld.so.preload and copy libdynamorio.so to /usr/lib or some other system search directory.

Plans for Future Releases

We hope to include the following major features in future releases:

• Persistent and process-shared code caches
• Auto-inlining of instrumentation callouts
• Client threads

We also plan to add a number of secondary features, including:

• More flexible (earlier, or later via attach) Linux injection
• Full control over trace building
• Client exception event for client fault recovery/remediation

To discuss current and future features, join the DynamoRIO Users group.