Why program when scanners are available?
There are commercial vulnerability scanners available in the market which can be used for vulnerability discovery. However, such vulnerability scanners have their own limitations and even the most advanced scanners sometimes are not able to provide full coverage. This makes the job of a penetration tester a little more difficult. This is where custom scripts/tools come into the picture. They help in filling the gaps created by the scanner since they’re customized to fit the target application.
It should be noted here that custom tools written for specialized purpose using languages like Python should not be a replacement for vulnerability scanners, and ideally should be used in addition to these scanners to get the best results.
The aim of this article is to introduce web application penetration testers with Python and explain how Python can be used for making customized HTTP requests – which in turn can be further expanded for development of custom scripts/tools that can be developed for special conditions where scanners fail. Readers will be introduced on libraries that can help a penetration tester in making custom HTTP requests using Python.
Setting up the environment
This article will not get into the details of setting up the environment – which is straight forward. Installers are available for Python and can be downloaded here.
If you are a Linux or Mac user, chances are high that you don’t have to install Python, since it usually comes pre-installed. To check if Python is installed on your system, launch the command prompt and type “python”, if Python is pre-installed, the interpreter will launch immediately.
Windows users can download the installer from above mentioned URL and install Python. To further make the use of Python easier, Windows users can add Python to the system path by editing the environment variable. Once done, users can just fire up Python from the command prompt – irrespective of the current working directory and still be able to invoke Python interpreter.
Python Modules for crafting HTTP requests
Python has multiple modules that can be used for generating custom HTTP Requests. We’ll cover 2 such modules that can be used for developing customized scripts, and can fire up our payloads along with performing the same actions that a penetration tester performs manually – the only difference being, this is done by a script instead of a manual attempt.
This module has been renamed to httplib.client in Python 3, however since in this article I am using version 2.7., I am going to stick with httplib. Normally this module is not directly used but instead urllib module uses it internally to make HTTP Requests. However, interested users can always use it directly.
In order for us to send custom requests, we need to do the following steps:
1. Import the library - Before using a library, we need to import it. Since in this case, we are going to use httplib library to send HTTP Requests and receive the responses back, we need to import it.
2. Create a connection - Once imported, we can start using it straight away. We need to create a connection object first. This can be achieved using HTTPConnection() method.