Web Content Scan or fuzz with dirb tool

Web Content Scan or fuzz with dirb tool

One of the tools that you may use during the first activities of web application security assessment is the dirb tool. Its usage is very simple, it is very lightweight so it doesn’t require any GUI and it is all done through command line.

The way that it works is to brute force the target URL that you pass to it as an argument, with a big dictionary of words that typically can be found on all the websites around the world. Thus, if dirb receives from the tested website an HTTP 200 response then, it is likely a browsable URL path. Otherwise, an HTTP 404 will not be shown as a finding on the tool output. Other HTTP response codes might be interpreted as browsable paths like 403.

Such a tool is used for black box or dynamic testing since it needs the web application to be running and you might not fully know how it is implemented. Similar tools to dirb are dirbuster, OWASP ZAP, Burp Suite or wfuzz. 

The findings from this tool could be helpful to enumerate the website, for further testing the detected URL paths and to find any potential information disclosure that could have. Nevertheless, don’t expect dirb to be a vulnerability scanner since it doesn’t do anything regarding.

To get familiar with the tool, continue reading this post so you can check or even reproduce some use cases described with dirb.

Install dirb in your Linux system

The dirb tool normally comes with the Kali Linux distribution so, if you have an environment with this OS then check if it is already installed. If so, you may jump to the next section of this post.

If you want to install dirb, proceed to execute the following commands in Ubuntu:

$ sudo apt-get install dirb

On the other hand, to install dirb in CentOS you may do it from source code:

$ sudo yum install libcurl-devel -y
$ wget -q https://github.com/v0re/dirb/archive/refs/heads/master.zip
$ unzip master.zip
$ cd dirb-master/
$ ./configure
$ make
$ sudo make install
$ export PATH=$PATH:/usr/local/bin/

After installing the tool, the built-in wordlists used for brute force should be located in /usr/share/dirb/wordlists/:

$ ls -l /usr/share/dirb/wordlists/
dirb wordlist files and their path

Preparing the test bed for dirb

Let’s prepare a scanning target for running dirb and get some findings to know how they look. For this purpose, we are going to use docker to deploy the vulnerables/web-dvwa container which is used for testing only. If you don’t have docker installed on your testing environment, you may find the installation guide for Ubuntu or CentOS.

To deploy the docker container image, run the following command:

$ docker run -p 80:80 -d -it vulnerables/web-dvwa

Once it is deployed, you may run the following to check the container status:

$ docker ps

If there is no unhealthy status then it is correctly up and running:

checking if the deployed container image vulnerables/web-dvwa is correctly up and running

Scan the website with dirb

By default, dirb will do the brute force with the file common.txt which contains more than 4000 of commonly used URL paths, unless you specify as an argument another wordlist file.

$ dirb http://localhost/

The outcome should be:

fse@fse2:~$ dirb http://localhost/ 

-----------------
DIRB v2.22    
By The Dark Raver
-----------------

START_TIME: Thu Aug 18 01:53:28 2022
URL_BASE: http://localhost/
WORDLIST_FILES: /usr/share/dirb/wordlists/common.txt

-----------------

GENERATED WORDS: 4612                                                          

---- Scanning URL: http://localhost/ ----
==> DIRECTORY: http://localhost/config/                                                                                                                                                                                                                                                                                   
==> DIRECTORY: http://localhost/docs/                                                                                                                                                                                                                                                                                     
==> DIRECTORY: http://localhost/external/                                                                                                                                                                                                                                                                                 
+ http://localhost/favicon.ico (CODE:200|SIZE:1406)                                                                                                                                                                                                                                                                       
+ http://localhost/index.php (CODE:302|SIZE:0)                                                                                                                                                                                                                                                                            
+ http://localhost/php.ini (CODE:200|SIZE:148)                                                                                                                                                                                                                                                                            
+ http://localhost/phpinfo.php (CODE:302|SIZE:0)                                                                                                                                                                                                                                                                          
+ http://localhost/robots.txt (CODE:200|SIZE:26)                                                                                                                                                                                                                                                                          
+ http://localhost/server-status (CODE:403|SIZE:297)                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/config/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/docs/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/external/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                               
-----------------
END_TIME: Thu Aug 18 01:53:29 2022
DOWNLOADED: 4612 - FOUND: 6

If you want to scan with a different wordlist other than the default, specify the path as the second argument after the target URL:

$ dirb http://localhost:1234 /usr/share/dirb/wordlists/big.txt

Then, the output obtained is:

fse@fse2:~$ dirb http://localhost/ /usr/share/dirb/wordlists/big.txt

-----------------
DIRB v2.22    
By The Dark Raver
-----------------

START_TIME: Thu Aug 18 01:54:34 2022
URL_BASE: http://localhost/
WORDLIST_FILES: /usr/share/dirb/wordlists/big.txt

-----------------

GENERATED WORDS: 20458                                                         

---- Scanning URL: http://localhost/ ----
==> DIRECTORY: http://localhost/config/                                                                                                                                                                                                                                                                                   
==> DIRECTORY: http://localhost/docs/                                                                                                                                                                                                                                                                                     
==> DIRECTORY: http://localhost/external/                                                                                                                                                                                                                                                                                 
+ http://localhost/favicon.ico (CODE:200|SIZE:1406)                                                                                                                                                                                                                                                                       
+ http://localhost/robots.txt (CODE:200|SIZE:26)                                                                                                                                                                                                                                                                          
+ http://localhost/server-status (CODE:403|SIZE:297)                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/config/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/docs/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                                                                                                                                                                                                                                                                          
---- Entering directory: http://localhost/external/ ----
(!) WARNING: Directory IS LISTABLE. No need to scan it.                        
    (Use mode '-w' if you want to scan it anyway)
                                                                               
-----------------
END_TIME: Thu Aug 18 01:54:43 2022
DOWNLOADED: 20458 - FOUND: 3

As you may appreciate from the previous 2 outputs, the common.txt wordlist has found more results than the big.txt wordlist. In some cases, you might want to combine both to create a more complete wordlist or even generate a customised one.

There are some options from dirb that might be interesting to know they exist like:

  • -H option to send a header in the dirb requests which could be useful depending on how the web application is implemented. 
  • -a option to set the HTTP request user agent
  • -c to send a custom cookie for scanning content that requires authentication 

To conclude

As you may appreciate, using the dirb tool it’s pretty easy, fast which might help during the initial steps of website security assessment. Nevertheless, if you want to perform other web security testing or more advanced features then you might want to check OWASP ZAP or Burp Suite.

Finally, do not use this tool to target any other website on the internet since it is not legit. These kinds of actions should be done in your offline test environment or in a previously formally agreed corporate network for the penetration testing or scanning activities.