Abstract: An anti-malware system monitors the emulation of a suspicious program in a sandbox environment. The anti-malware system determines that the suspicious program is attempting to access a restricted area of memory (e.g., an executable instruction in the restricted area). Rather than stop the emulation, the anti-malware system can temporarily pause the emulation of the suspicious program. During this pause, the anti-malware system can determine whether the suspicious program is containable within the sandbox environment. If the anti-malware system determines that the emulation of the executable instruction is safe (e.g., that the program is containable), the anti-malware system will resume emulation. If the anti-malware system determines that the emulation of the executable instruction is not safe, the anti-malware system may shut down emulation.

Abstract: An anti-malware application can emulate a suspicious program in a sandbox environment and retrieve any exception handlers the suspicious program attempts to register with the operation system. When the suspicious program triggers an exception, the anti-malware application can save a current context of the suspicious program being emulated. To emulate the handling of the exception, the anti-malware application can validate an exception handler chain including one or more exception handlers added by the suspicious program. The anti-malware application can then select and emulate an exception handler based on the saved context of the suspicious program at the time the exception was triggered. If the first exception handler is successful at resolving the exception, the anti-malware application can then save an updated post-exception context and continue emulation of the suspicious program based on the result of the first exception handler.

Abstract: An anti-malware application can emulate a suspicious program in a sandbox environment and retrieve any exception handlers the suspicious program attempts to register with the operation system. When the suspicious program triggers an exception, the anti-malware application can save a current context of the suspicious program being emulated. To emulate the handling of the exception, the anti-malware application can validate an exception handler chain including one or more exception handlers added by the suspicious program. The anti-malware application can then select and emulate an exception handler based on the saved context of the suspicious program at the time the exception was triggered. If the first exception handler is successful at resolving the exception, the anti-malware application can then save an updated post-exception context and continue emulation of the suspicious program based on the result of the first exception handler.

Abstract: A method that automatically generates blacklists for a sandbox application. The method first obtains a set of disassembled operating system (OS) dynamic-link libraries (DLLs) and then identifies application programming interfaces (API) functions that have respective kernel interruptions. The identified API functions that have kernel instructions are saved to an interrupt list. Based on the interrupt list, a processor generates a blacklist that includes for each of the DLLs, the identified API functions in the interrupt list, all API functions that directly or indirectly invoke one of the identified API functions in the interrupt list via one or more nested API functions. The method outputs the blacklist to the sandbox application that operates on a sample file to emulate API functions of the sample file that match the blacklist. All other APIs not identified as being blacklisted, are then considered whitelisted and are allowed to run natively.

Abstract: A method that automatically generates blacklists for a sandbox application. The method first obtains a set of disassembled operating system (OS) dynamic-link libraries (DLLs) and then identifies application programming interfaces (API) functions that have respective kernel interruptions. The identified API functions that have kernel instructions are saved to an interrupt list. Based on the interrupt list, a processor generates a blacklist that includes for each of the DLLs, the identified API functions in the interrupt list, all API functions that directly or indirectly invoke one of the identified API functions in the interrupt list via one or more nested API functions. The method outputs the blacklist to the sandbox application that operates on a sample file to emulate API functions of the sample file that match the blacklist. All other APIs not identified as being blacklisted, are then considered whitelisted and are allowed to run natively.

Abstract: An anti-malware application can emulate a suspicious program in a sandbox environment and retrieve any exception handlers the suspicious program attempts to register with the operation system. When the suspicious program triggers an exception, the anti-malware application can save a current context of the suspicious program being emulated. To emulate the handling of the exception, the anti-malware application can validate an exception handler chain including one or more exception handlers added by the suspicious program. The anti-malware application can then select and emulate an exception handler based on the saved context of the suspicious program at the time the exception was triggered. If the first exception handler is successful at resolving the exception, the anti-malware application can then save an updated post-exception context and continue emulation of the suspicious program based on the result of the first exception handler.

Abstract: A stealthy internal function (IF) debugger that leverages control flow detours can escape detection by traditional anti-debugging methods. Software that attempts to impede reverse engineering via dynamic analysis, by using anti-debugging or packing measures can be thwarted by using a stealthy IF debugger. Data mining through an IF utility can aid reverse engineering by constructing a data and code flow analysis after an execution of a program.

Abstract: A method of stealthily debugging software comprises dynamically injecting a jump into an executing target program; performing a debugging operation on the target program; and dynamically removing the injected jump from the target program. Dynamically injecting a jump comprises copying memory contents from a selected breakpoint location to a second memory location, and writing a jump instruction and location at the breakpoint location. Dynamically removing the injected jump comprises copying memory contents back to the breakpoint location from the second memory location. The method may further comprise replacing a pointer to a system function with a pointer to a debug module in a syscall table, and placing the debug module in a slack space of the target program. A debugging system is also disclosed.

Abstract: A system and method are disclosed that enable automated deobfuscation of software. A method may include identifying at least one section of target software matching trigger criteria, either by using pattern matching or behavior analysis; emulating at least a portion of the identified section; and generating deobfuscated software by substituting a simplified section for the identified section. The method may further be iterated. Emulation includes simulating the effect of certain instructions on control flow and/or memory locations, such as the program stack, a register, cache memory, heap memory, or other memory. The simplified section may comprise a number of no operation (NOP) instructions replacing, which may then be jumped for further simplification.