Gray Hat Python: Python Programming for Hackers and Reverse Engineers

Gray Hat Python

Table of Contents

FOREWORDACKNOWLEDGMENTSINTRODUCTION1. SETTING UP YOUR DEVELOPMENT ENVIRONMENTOperating System RequirementsObtaining and Installing Python 2.5Installing Python on WindowsInstalling Python for LinuxSetting Up Eclipse and PyDevThe Hacker's Best Friend: ctypesUsing Dynamic LibrariesConstructing C DatatypesPassing Parameters by ReferenceDefining Structures and Unions2. DEBUGGERS AND DEBUGGER DESIGNGeneral-Purpose CPU RegistersThe StackFunction Call in CDebug EventsBreakpointsSoft BreakpointsHardware BreakpointsMemory Breakpoints3. BUILDING A WINDOWS DEBUGGERDebuggee, Where Art Thou?my_debugger_defines.pyObtaining CPU Register StateThread EnumerationPutting It All TogetherImplementing Debug Event Handlersmy_debugger.pyThe Almighty BreakpointSoft BreakpointsHardware BreakpointsMemory BreakpointsConclusion4. PYDBG—A PURE PYTHON WINDOWS DEBUGGERExtending Breakpoint Handlersprintf_random.pyAccess Violation HandlersProcess SnapshotsObtaining Process SnapshotsPutting It All Together5. IMMUNITY DEBUGGER—THE BEST OF BOTH WORLDSInstalling Immunity DebuggerImmunity Debugger 101PyCommandsPyHooksExploit DevelopmentFinding Exploit-Friendly InstructionsBad-Character FilteringBypassing DEP on WindowsDefeating Anti-Debugging Routines in MalwareIsDebuggerPresentDefeating Process Iteration6. HOOKINGSoft Hooking with PyDbgfirefox_hook.pyHard Hooking with Immunity Debuggerhippie_easy.py7. DLL AND CODE INJECTIONRemote Thread CreationDLL InjectionCode InjectionGetting EvilFile HidingCoding the BackdoorCompiling with py2exe8. FUZZINGBug ClassesBuffer OverflowsInteger OverflowsFormat String AttacksFile Fuzzerfile_fuzzer.pyFuture ConsiderationsCode CoverageAutomated Static Analysis9. SULLEYSulley InstallationSulley PrimitivesStringsDelimitersStatic and Random PrimitivesBinary DataIntegersBlocks and GroupsSlaying WarFTPD with SulleyFTP 101Creating the FTP Protocol SkeletonSulley SessionsNetwork and Process MonitoringFuzzing and the Sulley Web Interface10. FUZZING WINDOWS DRIVERSDriver CommunicationDriver Fuzzing with Immunity Debuggerioctl_fuzzer.pyDriverlib—The Static Analysis Tool for DriversDiscovering Device NamesFinding the IOCTL Dispatch RoutineDetermining Supported IOCTL CodesBuilding a Driver Fuzzerioctl_dump.py11. IDAPYTHON—SCRIPTING IDA PROIDAPython InstallationIDAPython FunctionsUtility FunctionsSegmentsFunctionsCross-ReferencesDebugger HooksExample ScriptsFinding Dangerous Function Cross-ReferencesFunction Code CoverageCalculating Stack Size12. PYEMU—THE SCRIPTABLE EMULATORInstalling PyEmuPyEmu OverviewPyCPUPyMemoryPyEmuExecutionMemory and Register ModifiersHandlersRegister HandlersLibrary HandlersException HandlersInstruction HandlersOpcode HandlersMemory HandlersHigh-Level Memory HandlersProgram Counter HandlerIDAPyEmuaddnum.cppFunction EmulationPEPyEmuExecutable PackersUPX PackerUnpacking UPX with PEPyEmu

Gray Hat Python

Justin Seitz

Copyright © 2009

For information on book distributors or translations, please contact No Starch Press, Inc. directly:

No Starch Press, Inc.

555 De Haro Street, Suite 250, San Francisco, CA 94107

phone: 415.863.9900; fax: 415.863.9950; [email protected]; www.nostarch.com

Library of Congress Cataloging-in-Publication Data:

Seitz, Justin.

  Gray hat Python : Python programming for hackers and reverse engineers / Justin Seitz.

      p. cm.

  ISBN-13: 978-1-59327-192-3

  ISBN-10: 1-59327-192-1

1.  Computer security. 2.  Python (Computer program language)  I. Title.

  QA76.9.A25S457 2009

  005.8--dc22

                                                            2009009107

No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

The information in this book is distributed on an As Is basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it.

No Starch Press

Dedication

Mom,

If there's one thing I wish for you to remember, it's that I love you very much.

Alzheimer Society of Canada—www.alzheimers.ca

FOREWORD

The phrase most often heard at Immunity is probably, Is it done yet? Common parlance usually goes something like this: I'm starting work on the new ELF importer for Immunity Debugger. Slight pause. Is it done yet? or I just found a bug in Internet Explorer! And then, Is the exploit done yet? It's this rapid pace of development, modification, and creation that makes Python the perfect choice for your next security project, be it building a special decompiler or an entire debugger.

I find it dizzying sometimes to walk into Ace Hardware here in South Beach and walk down the hammer aisle. There are around 50 different kinds on display, arranged in neat rows in the tiny store. Each one has some minor but extremely important difference from the next. I'm not enough of a handyman to know what the ideal use for each device is, but the same principle holds when creating security tools. Especially when working on web or custom-built apps, each assessment is going to require some kind of specialized hammer. Being able to throw together something that hooks the SQL API has saved an Immunity team on more than one occasion. But of course, this doesn't just apply to assessments. Once you can hook the SQL API, you can easily write a tool to do anomaly detection against SQL queries, providing your organization with a quick fix against a persistent attacker.

Everyone knows that it's pretty hard to get your security researchers to work as part of a team. Most security researchers, when faced with any sort of problem, would like to first rebuild the library they are going to use to attack the problem. Let's say it's a vulnerability in an SSL daemon of some kind. It's very likely that your researcher is going to want to start by building an SSL client, from scratch, because "the SSL library I found was ugly."

You need to avoid this at all costs. The reality is that the SSL library is not ugly—it just wasn't written in that particular researcher's particular style. Being able to dive into a big block of code, find a problem, and fix it is the key to having a working SSL library in time for you to write an exploit while it still has some meaning. And being able to have your security researchers work as a team is the key to making the kinds of progress you require. One Python-enabled security researcher is a powerful thing, much as one Ruby-enabled one is. The difference is the ability of the Pythonistas to work together, use old source code without rewriting it, and otherwise operate as a functioning superorganism. That ant colony in your kitchen has about the same mass as an octopus, but it's much more annoying to try to kill!

And here, of course, is where this book helps you. You probably already have tools to do some of what you want to do. You say, I've got Visual Studio. It has a debugger. I don't need to write my own specialized debugger. Or, Doesn't WinDbg have a plug-in interface? And the answer is yes, of course WinDbg has a plug-in interface, and you can use that API to slowly put together something useful. But then one day you'll say, Heck, this would be a lot better if I could connect it to 5,000 other people using WinDbg and we could correlate our results. And if you're using Python, it takes about 100 lines of code for both an XML-RPC client and a server, and now everyone is synchronized and working off the same page.

Because hacking is not reverse engineering—your goal is not to come up with the original source code for the application. Your goal is to have a greater understanding of the program or system than the people who built it. Once you have that understanding, no matter what the form, you will be able to penetrate the program and get to the juicy exploits inside. This means that you're going to become an expert at visualization, remote synchronization, graph theory, linear equation solving, statistical analysis techniques, and a whole host of other things. Immunity's decision regarding this has been to standardize entirely on Python, so every time we write a graph algorithm, it can be used across all of our tools.

In Chapter 6, Justin shows you how to write a quick hook for Firefox to grab usernames and passwords. On one hand, this is something a malware writer would do—and previous reports have shown that malware writers do use high-level languages for exactly this sort of thing (http://philosecurity.org/2009/01/12/interview-with-an-adware-author). On the other hand, this is precisely the sort of thing you can whip up in 15 minutes to demonstrate to developers exactly which of the assumptions they are making about their software are clearly untrue. Software companies invest a lot in protecting their internal memory for what they claim are security reasons but are really copy protection and digital rights management (DRM) related.

So here's what you get with this book: the ability to rapidly create software tools that manipulate other applications. And you get to do this in a way that allows you to build on your success either by yourself or with a team. This is the future of security tools: quickly implemented, quickly modified, quickly connected. I guess the only question left is, Is it done yet?

Dave Aitel

Miami Beach, Florida

February 2009

ACKNOWLEDGMENTS

I would like to thank my family for tolerating me throughout the whole process of writing this book. My four beautiful children, Emily, Carter, Cohen, and Brady, you helped give Dad a reason to keep writing this book, and I love you very much for being the great kids you are. My brothers and sister, thanks for encouraging me through the process. You guys have written some tomes yourselves, and it was always helpful to have someone who understands the rigor needed to put out any kind of technical work—I love you guys. To my Dad, your sense of humor helped me through a lot of the days when I didn't feel like writing—I love ya Harold; don't stop making everyone around you laugh.

For all those who helped this fledgling security researcher along the way—Jared DeMott, Pedram Amini, Cody Pierce, Thomas Heller (the uber Python man), Charlie Miller—I owe all you guys a big thanks. Team Immunity, without question you've been incredibly supportive of me writing this book, and you have helped me tremendously in growing not only as a Python dude but as a developer and researcher as well. A big thanks to Nico and Dami for the extra time you spent helping me out. Dave Aitel, my technical editor, helped drive this thing to completion and made sure that it makes sense and is readable; a huge thanks to Dave. To another Dave, Dave Falloon, thanks so much for reviewing the book, making me laugh at my own mistakes, saving my laptop at CanSecWest, and just being the oracle of network knowledge that you are.

Finally, and I know they always get listed last, the team at No Starch Press. Tyler for putting up with me through the whole book (trust me, Tyler is the most patient guy you'll ever meet), Bill for the great Perl mug and the words of encouragement, Megan for helping wrap up this book as painlessly as possible, and the rest of the crew who I know works behind the scenes to help put out all their great titles. A huge thanks to all you guys; I appreciate everything you have done for me. Now that the acknowledgments have taken as long as a Grammy acceptance speech, I'll wrap it up by saying thanks to all the rest of the folks who helped me and who I probably forgot to add to the list—you know who you are.

INTRODUCTION

I learned Python specifically for hacking—and I'd venture to say that's a true statement for a lot of other folks, too. I spent a great deal of time hunting around for a language that was well suited for hacking and reverse engineering, and a few years ago it became very apparent that Python was becoming the natural leader in the hacking-programming-language department. The tricky part was the fact that there was no real manual on how to use Python for a variety of hacking tasks. You had to dig through forum posts and man pages and typically spend quite a bit of time stepping through code to get it to work right. This book aims to fill that gap by giving you a whirlwind tour of how to use Python for hacking and reverse engineering in a variety of ways.

The book is designed to allow you to learn some theory behind most hacking tools and techniques, including debuggers, backdoors, fuzzers, emulators, and code injection, while providing you some insight into how prebuilt Python tools can be harnessed when a custom solution isn't needed. You'll learn not only how to use Python-based tools but how to build tools in Python. But be forewarned, this is not an exhaustive reference! There are many, many infosec (information security) tools written in Python that I did not cover. However, this book will allow you to translate a lot of the same skills across applications so that you can use, debug, extend, and customize any Python tool of your choice.

There are a couple of ways you can progress through this book. If you are new to Python or to building hacking tools, then you should read the book front to back, in order. You'll learn some necessary theory, program oodles of Python code, and have a solid grasp of how to tackle a myriad of hacking and reversing tasks by the time you get to the end. If you are familiar with Python already and have a good grasp on the Python library ctypes, then jump straight to Chapter 2. For those of you who have been around the block, it's easy enough to jump around in the book and use code snippets or certain sections as you need them in your day-to-day tasks.

I spend a great deal of time on debuggers, beginning with debugger theory in Chapter 2, and progressing straight through to Immunity Debugger in Chapter 5. Debuggers are a crucial tool for any hacker, and I make no bones about covering them extensively. Moving forward, you'll learn some hooking and injection techniques in Chapters Chapter 6 and Chapter 7, which you can add to some of the debugging concepts of program control and memory manipulation.

The next section of the book is aimed at breaking applications using fuzzers. In Chapter 8, you'll begin learning about fuzzing, and we'll construct our own basic file fuzzer. In Chapter 9, we'll harness the powerful Sulley fuzzing framework to break a real-world FTP daemon, and in Chapter 10 you'll learn how to build a fuzzer to destroy Windows drivers.

In Chapter 11, you'll see how to automate static analysis tasks in IDA Pro, the popular binary static analysis tool. We'll wrap up the book by covering PyEmu, the Python-based emulator, in Chapter 12.

I have tried to keep the code listings somewhat short, with detailed explanations of how the code works inserted at specific points. Part of learning a new language or mastering new libraries is spending the necessary sweat time to actually write out the code and debug your mistakes. I encourage you to type in the code! All source will be posted to http://www.nostarch.com/ghpython.htm for your downloading pleasure.

Now let's get coding!

Chapter 1. SETTING UP YOUR DEVELOPMENT ENVIRONMENT

Before you can experience the art of gray hat Python programming, you must work through the least exciting portion of this book, setting up your development environment. It is essential that you have a solid development environment, which allows you to spend time absorbing the interesting information in this book rather than stumbling around trying to get your code to execute.

This chapter quickly covers the installation of Python 2.5, configuring your Eclipse development environment, and the basics of writing C-compatible code with Python. Once you have set up the environment and understand the basics, the world is your oyster; this book will show you how to crack it open.

Operating System Requirements

I assume that you are using a 32-bit Windows-based platform to do most of your coding. Windows has the widest array of tools and lends itself well to Python development. All of the chapters in this book are Windows-specific, and most examples will work only with a Windows operating system.

However, there are some examples that you can run from a Linux distribution. For Linux development, I recommend you download a 32-bit Linux distro as a VMware appliance. VMware's appliance player is free, and it enables you to quickly move files from your development machine to your virtualized Linux machine. If you have an extra machine lying around, feel free to install a complete distribution on it. For the purpose of this book, use a Red Hat–based distribution like Fedora Core 7 or Centos 5. Of course, alternatively, you can run Linux and emulate Windows. It's really up to you.

FREE VMWARE IMAGES

VMware provides a directory of free appliances on its website. These appliances enable a reverse engineer or vulnerability researcher to deploy malware or applications inside a virtual machine for analysis, which limits the risk to any physical infrastructure and provides an isolated scratchpad to work with. You can visit the virtual appliance marketplace at http://www.vmware.com/appliances/ and download the player at http://www.vmware.com/products/player/.

Obtaining and Installing Python 2.5

The Python installation is quick and painless on both Linux and Windows. Windows users are blessed with an installer that takes care of all of the setup for you; however, on Linux you will be building the installation from source code.

Installing Python on Windows

Windows users can obtain the installer from the main Python site: http://python.org/ftp/python/2.5.1/python-2.5.1.msi. Just double-click the installer, and follow the steps to install it. It should create a directory at C:/Python25/; this directory will have the python.exe interpreter as well as all of the default libraries installed.

Note

You can optionally install Immunity Debugger, which contains not only the debugger itself but also an installer for Python 2.5. In later chapters you will be using Immunity Debugger for many tasks, so you are welcome to kill two birds with one installer here. To download and install Immunity Debugger, visit http://debugger.immunityinc.com/.

Installing Python for Linux

To install Python 2.5 for Linux, you will be downloading and compiling from source. This gives you full control over the installation while preserving the existing Python installation that is present on a Red Hat–based system. The installation assumes that you will be executing all of the following commands as the root user.

The first step is to download and unzip the Python 2.5 source code. In a command-line terminal session, enter the following:

# cd /usr/local/

#

wget http://python.org/ftp/python/2.5.1/Python-2.5.1.tgz

#

tar -zxvf Python-2.5.1.tgz

#

mv Python-2.5.1 Python25

#

cd Python25

You have now downloaded and unzipped the source code into /usr/local/Python25. The