OSINT Tools

It is very important to learn how to use the different tools, frameworks, and eventually how to automate them. It will make everything run a lot faster and smoother. It’s also a good idea to learn how to research the tools. Learn more about the tools you are using while realizing that the tools may change over time and that there may be other tools out there. Some tools may stop working so you’ll need to find something to replace those.

Subsections of OSINT Tools

Image and Location

More tools can be found if you do some research.

google.com

  • exif tool site:github.com

exiftool - Might have to install it.

apt search exiftool                           
Sorting... Done
Full Text Search... Done

libimage-exiftool-perl/kali-rolling 12.67+dfsg-1 all
  library and program to read and write meta information in multimedia files
sudo apt install libimage-exiftool-perl    
[sudo] password for kali:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libcompress-raw-lzma-perl libmime-charset-perl libsombok3 libunicode-linebreak-perl
Suggested packages:
  libposix-strptime-perl libencode-eucjpascii-perl libencode-hanextra-perl libpod2-base-perl
The following NEW packages will be installed:
  libcompress-raw-lzma-perl libimage-exiftool-perl libmime-charset-perl libsombok3 libunicode-linebreak-perl
0 upgraded, 5 newly installed, 0 to remove and 1284 not upgraded.
Need to get 4,114 kB of archives.
After this operation, 24.7 MB of additional disk space will be used.
Do you want to continue? [Y/n]
exiftool dog.JPG             
ExifTool Version Number         : 12.67
File Name                       : dog.JPG
Directory                       : .
File Size                       : 3.9 MB
File Modification Date/Time     : 2023:12:28 10:59:18-05:00
File Access Date/Time           : 2023:12:28 11:01:19-05:00
File Inode Change Date/Time     : 2023:12:28 11:01:19-05:00
File Permissions                : -rwxrwx---
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Rotate 90 CW
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 5.0.1
Modify Date                     : 2012:03:11 12:01:53
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/1842
F Number                        : 2.4
Exposure Program                : Program AE
ISO                             : 64
Exif Version                    : 0221
Date/Time Original              : 2012:03:11 12:01:53
Create Date                     : 2012:03:11 12:01:53
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/1842
Aperture Value                  : 2.4
Brightness Value                : 10.39054726
Metering Mode                   : Multi-segment
Flash                           : Off, Did not fire
Focal Length                    : 4.3 mm
Subject Area                    : 1631 1223 881 881
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 3264
Exif Image Height               : 2448
Sensing Method                  : One-chip color area
Exposure Mode                   : Auto
White Balance                   : Auto
Focal Length In 35mm Format     : 35 mm
Scene Capture Type              : Standard
Sharpness                       : Normal
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 17:30:26
GPS Img Direction Ref           : True North
GPS Img Direction               : 191.2603175
Compression                     : JPEG (old-style)
Thumbnail Offset                : 914
Thumbnail Length                : 9959
Image Width                     : 3264
Image Height                    : 2448
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.4
Image Size                      : 3264x2448
Megapixels                      : 8.0
Scale Factor To 35 mm Equivalent: 8.2
Shutter Speed                   : 1/1842
Thumbnail Image                 : (Binary data 9959 bytes, use -b option to extract)
GPS Altitude                    : 182 m Above Sea Level
GPS Latitude                    : 41 deg 40' 43.20" N
GPS Longitude                   : 83 deg 39' 21.00" W
Circle Of Confusion             : 0.004 mm
Field Of View                   : 54.4 deg
Focal Length                    : 4.3 mm (35 mm equivalent: 35.0 mm)
GPS Position                    : 41 deg 40' 43.20" N, 83 deg 39' 21.00" W
Hyperfocal Distance             : 2.08 m
Light Value                     : 14.0

Emails and Breached Data

Hunting Emails and Breached Data

theHarvester: github.com/laramies/theHarvester

It performs open source intelligence (OSINT) gathering to help determine a domain’s external threat landscape. The tool gathers names, emails, IPs, subdomains, and URLs by using multiple public resources

Note: - Some websites may start to block you if you are hitting them too much, such as google.com might block you if you are searching too much with theHarvester or other tools like this.

 theHarvester -h              
*******************************************************************
*  _   _                                            _             *
* | |_| |__   ___    /\  /\__ _ _ ____   _____  ___| |_ ___ _ __  *
* | __|  _ \ / _ \  / /_/ / _` | '__\ \ / / _ \/ __| __/ _ \ '__| *
* | |_| | | |  __/ / __  / (_| | |   \ V /  __/\__ \ ||  __/ |    *
*  \__|_| |_|\___| \/ /_/ \__,_|_|    \_/ \___||___/\__\___|_|    *
*                                                                 *
* theHarvester 4.3.0                                              *
* Coded by Christian Martorella                                   *
* Edge-Security Research                                          *
* cmartorella@edge-security.com                                   *
*                                                                 *
*******************************************************************
usage: theHarvester [-h] -d DOMAIN [-l LIMIT] [-S START] [-p] [-s] [--screenshot SCREENSHOT] [-v] [-e DNS_SERVER] [-t]
                    [-r [DNS_RESOLVE]] [-n] [-c] [-f FILENAME] [-b SOURCE]

theHarvester is used to gather open source intelligence (OSINT) on a company or domain.

options:
  -h, --help            show this help message and exit
  -d DOMAIN, --domain DOMAIN
                        Company name or domain to search.
  -l LIMIT, --limit LIMIT
                        Limit the number of search results, default=500.
  -S START, --start START
                        Start with result number X, default=0.
  -p, --proxies         Use proxies for requests, enter proxies in proxies.yaml.
  -s, --shodan          Use Shodan to query discovered hosts.
  --screenshot SCREENSHOT
                        Take screenshots of resolved domains specify output directory: --screenshot output_directory
  -v, --virtual-host    Verify host name via DNS resolution and search for virtual hosts.
  -e DNS_SERVER, --dns-server DNS_SERVER
                        DNS server to use for lookup.
  -t, --take-over       Check for takeovers.
  -r [DNS_RESOLVE], --dns-resolve [DNS_RESOLVE]
                        Perform DNS resolution on subdomains with a resolver list or passed in resolvers, default False.
  -n, --dns-lookup      Enable DNS server lookup, default False.
  -c, --dns-brute       Perform a DNS brute force on the domain.
  -f FILENAME, --filename FILENAME
                        Save the results to an XML and JSON file.
  -b SOURCE, --source SOURCE
                        anubis, baidu, bevigil, binaryedge, bing, bingapi, bufferoverun, brave, censys, certspotter, criminalip, crtsh,
                        dnsdumpster, duckduckgo, fullhunt, github-code, hackertarget, hunter, hunterhow, intelx, otx, pentesttools,
                        projectdiscovery, rapiddns, rocketreach, securityTrails, sitedossier, subdomainfinderc99, threatminer, urlscan,
                        virustotal, yahoo, zoomeye
theHarvester -d tesla.com -b all -l 50

theHarvester -d tesla.com -b yahoo -l 50

breach-parse: github.com/hmaverickadams/breach-parse

A tool for parsing breached passwords

Install: `sudo ./install.sh`

Download breached password list from magnet located here: `magnet:?xt=urn:btih:7ffbcd8cee06aba2ce6561688cf68ce2addca0a3&dn=BreachCompilation&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Fglotorrents.pw%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337`

If you don't store the password list (BreachCompilation) in `/opt/breach-parse`, specify the location like:

`breach-parse @gmail.com gmail.txt "~/Downloads/BreachCompilation/data"`

Run `breach-parse` for instructions

The breach compilation database is around 44GB. You have to download it and place it in the breach-parse directory.

h8mail github.com/khast3x/h8mail

h8mail is an email OSINT and breach hunting tool using different breach and reconnaissance services, or local breaches such as Troy Hunt’s “Collection1” and the infamous “Breach Compilation” torrent.

Requires API-keys and a lot of them are paid. You also can use that same breach compilation database with h8mail. One problem is that it can’t run a search that’s based on just a domain.

 pip3 install h8mail              
Defaulting to user installation because normal site-packages is not writeable
Collecting h8mail
  Downloading h8mail-2.5.6-py3-none-any.whl (34 kB)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from h8mail) (2.31.0)
Installing collected packages: h8mail
Successfully installed h8mail-2.5.6
h8mail -t target@example.com

h8mail -t targets.txt -bc ../Downloads/BreachCompilation/ -sk

h8mail -t targets.txt -gz /tmp/Collection1/ -sk

EmailHarvester: github.com/maldevel/EmailHarvester

Usernames and Accounts

WhatsMyName - Not the same tool now.
github.com/WebBreacher/WhatsMyName

WhatsMyName (WMN) consists of a JSON file with detections in it. Submissions from people all over the world are included. When a request is made to one of those sites from a tool like the ones in the next section, the server replies with data that will match one of our detections.

Website/Apps using WhatsMyName:

sherlock - Not to be confused with Sherlock.ps1
github.com/sherlock-project/sherlock

 sherlock thecybermentor 

  [*] Checking username thecybermentor on:

[+] AllMyLinks: https://allmylinks.com/thecybermentor
[+] BuyMeACoffee: https://buymeacoff.ee/thecybermentor
[+] Clubhouse: https://www.clubhouse.com/@thecybermentor
[+] GitHub: https://www.github.com/thecybermentor
[+] HackerOne: https://hackerone.com/thecybermentor
[+] Keybase: https://keybase.io/thecybermentor
[+] Linktree: https://linktr.ee/thecybermentor
[+] Nightbot: https://nightbot.tv/t/thecybermentor/commands
[+] Patreon: https://www.patreon.com/thecybermentor
[+] Reddit: https://www.reddit.com/user/thecybermentor
[+] TryHackMe: https://tryhackme.com/p/thecybermentor
[+] Twitch: https://www.twitch.tv/thecybermentor
[+] Twitter: https://twitter.com/thecybermentor
[+] mastodon.cloud: https://mastodon.cloud/@thecybermentor
[+] metacritic: https://www.metacritic.com/user/thecybermentor

blackbird

 python blackbird.py -u thecybermentor   

    ▄▄▄▄    ██▓    ▄▄▄       ▄████▄   ██ ▄█▀ ▄▄▄▄    ██▓ ██▀███  ▓█████▄
    ▓█████▄ ▓██▒   ▒████▄    ▒██▀ ▀█   ██▄█▒ ▓█████▄ ▓██▒▓██ ▒ ██▒▒██▀ ██▌
    ▒██▒ ▄██▒██░   ▒██  ▀█▄  ▒▓█    ▄ ▓███▄░ ▒██▒ ▄██▒██▒▓██ ░▄█ ▒░██   █▌
    ▒██░█▀  ▒██░   ░██▄▄▄▄██ ▒▓▓▄ ▄██▒▓██ █▄ ▒██░█▀  ░██░▒██▀▀█▄  ░▓█▄   ▌
    ░▓█  ▀█▓░██████▒▓█   ▓██▒▒ ▓███▀ ░▒██▒ █▄░▓█  ▀█▓░██░░██▓ ▒██▒░▒████▓
    ░▒▓███▀▒░ ▒░▓  ░▒▒   ▓▒█░░ ░▒ ▒  ░▒ ▒▒ ▓▒░▒▓███▀▒░▓  ░ ▒▓ ░▒▓░ ▒▒▓  ▒
    ▒░▒   ░ ░ ░ ▒  ░ ▒   ▒▒ ░  ░  ▒   ░ ░▒ ▒░▒░▒   ░  ▒ ░  ░▒ ░ ▒░ ░ ▒  ▒
    ░    ░   ░ ░    ░   ▒   ░        ░ ░░ ░  ░    ░  ▒ ░  ░░   ░  ░ ░  ░
    ░          ░  ░     ░  ░░ ░      ░  ░    ░       ░     ░        ░
        ░                  ░                     ░               ░

                                        Made with ❤️️ by p1ngul1n0

[!] Searching 'thecybermentor' across 582 social networks
[+] - #16 Twitter Archived account found - http://archive.org/wayback/available?url=https://twitter.com/thecybermentor [200 OK]
[+] - #19 Xhamster account found - https://xhamster.com/users/thecybermentor [200 OK]
[+] - #12 Github account found - https://github.com/thecybermentor [200 OK]
   |--Name:
   |--Nickname:           thecybermentor


   |--picture: https://avatars.githubusercontent.com/u/75207118?v=4?s=400
[+] - #41 Ebay account found - https://www.ebay.com/usr/thecybermentor [200 OK]
[+] - #80 HackerOne account found - https://hackerone.com/thecybermentor?type=user [200 OK]
[+] - #1 Facebook account found - https://www.facebook.com/thecybermentor [200 OK]
[+] - #7 Instagram account found - https://www.picuki.com/profile/thecybermentor [200 OK]
   |--Name: Heath Adams
   |--Bio:             Living my best life.
   |--Followers: 70,496
   |--Following: 38
   |--picture: https://cdn1.picuki.com/hosted-by-instagram/q/yep6IPkO1EBGZyPbcMUVwONSiqxxRQlN.jpeg
[+] - #15 Xbox Gamertag account found - https://www.xboxgamertag.com/search/thecybermentor [200 OK]
   |--Name: TheCyberMentor
   |--picture: https://images.weserv.nl/?url=https://images-eds-ssl.xboxlive.com/image?url=8Oaj9Ryq1G1_p3lLnXlsaZgGzAie6Mnu24_PawYuDYIoH77pJ.X5Z.MqQPibUVTcS9jr0n8i7LY1tL3U7AiafSCYxKvpXj31MkoG3bO_PRAaZnxiCXXhJwXdM2GFciVMQ65NJp_gJqzkZPkN4cbjqg--&format=png&maxage=1d&output=webp&w=90&h=90
[+] - #25 WordPress Site account found - https://thecybermentor.wordpress.com/ [200 OK]
[+] - #14 Linktree account found - https://linktr.ee/thecybermentor [200 OK]
   |--Name: @thecybermentor
   |--Description: Linktree. Make your link do more.
   |--picture: https://assets.production.linktr.ee/profiles/_next/static/logo-assets/default-meta-image.png
[+] - #117 BugBounty account found - https://bugbounty.gg/members/thecybermentor/ [200 OK]
[+] - #45 Armor Games account found - https://armorgames.com/user/thecybermentor [200 OK]
[+] - #312 GitHub account found - https://github.com/thecybermentor [200 OK]
[+] - #147 Fark account found - https://www.fark.com/users/thecybermentor [200 OK]
Traceback (most recent call last):
  File "/opt/blackbird/blackbird.py", line 298, in <module>
    asyncio.run(findUsername(arguments.username, interfaceType, arguments.csv))
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/blackbird/blackbird.py", line 58, in findUsername
    userFile = open(pathSave, "w")
               ^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/opt/blackbird/results/thecybermentor.json'

Phone Numbers

phoneinfoga: github.com/sundowndev/phoneinfoga
PhoneInfoga is one of the most advanced tools to scan international phone numbers. It allows you to first gather basic information such as country, area, carrier and line type, then use various techniques to try to find the VoIP provider or identify the owner. It works with a collection of scanners that must be configured in order for the tool to be effective. PhoneInfoga doesn’t automate everything, it’s just there to help investigating on phone numbers.

Needs country code before number. This would be a US number:

phoneinfoga scan -n 15555551212

Start localhost server:

phoneinfoga serve -p 8080 

Load http://localhost:8080/#/ in a web browser.

Social Media

Note: Most of these tools are currently either broken or don’t work at all due to the changes in social media services. For example Twint is dead.

All in one

snscrape: github.com/JustAnotherArchivist/snscrape

choose from 'facebook-community', 'facebook-group', 'facebook-user', 'instagram-hashtag', 'instagram-location', 'instagram-user', 'mastodon-profile', 'mastodon-toot', 'reddit-search', 'reddit-submission', 'reddit-subreddit', 'reddit-user', 'telegram-channel', 'twitter-cashtag', 'twitter-community', 'twitter-hashtag', 'twitter-list-posts', 'twitter-profile', 'twitter-search', 'twitter-trends', 'twitter-tweet', 'twitter-user', 'twitter-users', 'vkontakte-user', 'weibo-user'

A few tests with this tool were unsuccessful but it might possibly work for some of the services.
It scrapes things like user profiles, hashtags, or searches and returns the discovered items, e.g. the relevant posts from these social networking services.

  • Facebook: user profiles, groups, and communities (aka visitor posts)
  • Instagram: user profiles, hashtags, and locations
  • Mastodon: user profiles and toots (single or thread)
  • Reddit: users, subreddits, and searches (via Pushshift)
  • Telegram: channels
  • Twitter: users, user profiles, hashtags, searches (live tweets, top tweets, and users), tweets (single or surrounding thread), list posts, communities, and trends
  • VKontakte: user profiles
  • Weibo (Sina Weibo): user profiles

Twitter

Twint is no longer maintained due to changes in twitter.
twint: github.com/twintproject/twint

Another tool but it probably doesn’t work now, after changes in June 2023.
github.com/markowanga/stweet

Another tool but haven’t tested it yet.
github.com/Altimis/Scweet

Instagram

Doesn’t appear to work now.
InstagramOSINT: github.com/sc1341/InstagramOSINT

List of Social Media Tools

github.com/osintambition/Social-Media-OSINT-Tools-Collection

Websites

Firefox extension:
Wappalyzer

whatweb - command line.
whatweb https://domain.com

whois - command line.
whois domain.com

httprobe: github.com/tomnomnom/httprobe
amass: github.com/owasp-amass/amass

Subdomains

More subdomain tools:
github.com/topics/subdomain-enumeration

Command Examples:

subfinder -d domain.com

sublist3r -d domain.com

bbot -t z3r0r3z.com -f subdomain-enum -rf passive
bbot -s -t z3r0r3z.com -f subdomain-enum -rf passive

assetfinder --subs-only domain.com >> domain-com-subdom.txt
assetfinder domain.com | grep domain.com | sort -u
assetfinder domain.com | grep domain.com > domain.txt
cat domain.txt | grep dev
cat domain.txt | grep sta
cat domain.txt | grep admin

amass enum -d domain.com

cat domain.txt | sort -u | httprobe -s -p https:443

gowitness single https://domain.com
gowitness file -f ./alive_gowitness.txt -P captures_gowitness/ --no-http

bbot example - partial output:

 bbot -t z3r0r3z.com -f subdomain-enum -rf passive
[INFO] Loaded defaults from /home/kali/.local/pipx/venvs/bbot/lib/python3.11/site-packages/bbot/defaults.yml
[INFO] Creating BBOT config at /home/kali/.config/bbot/bbot.yml
[INFO] Creating BBOT secrets at /home/kali/.config/bbot/secrets.yml
[INFO] 
[INFO] ### MODULES ###
 
[....] (Too much to paste here)

[INFO] Finishing scan
[INFO] asn: +---------+------------------+--------------+----------------+-------------------------+-----------+
[INFO] asn: | ASN     | Subnet           | Host Count   | Name           | Description             | Country   |
[INFO] asn: +=========+==================+==============+================+=========================+===========+
[INFO] asn: | AS63410 | 109.150.165.0/22 | 6            | PRIVATECOSYSTEMS | PrivateEcoSystems Petworks | US        |
[INFO] asn: +---------+------------------+--------------+----------------+-------------------------+-----------+
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | Module          | Produced           | Consumed                     |
[INFO] aggregate: +=================+====================+==============================+
[INFO] aggregate: | certspotter     | 4 (4 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | leakix          | 4 (4 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | columbus        | 3 (3 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | crt             | 3 (3 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | massdns         | 3 (3 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | dnsdumpster     | 2 (2 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | hackertarget    | 2 (2 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | otx             | 2 (2 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | wayback         | 2 (2 DNS_NAME)     | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | CNAME           | 2 (2 DNS_NAME)     | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | azure_tenant    | 1 (1 AZURE_TENANT) | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | speculate       | 1 (1 DNS_NAME)     | 4 (3 DNS_NAME, 1 IP_ADDRESS) |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | A               | 1 (1 IP_ADDRESS)   | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | anubisdb        | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | azure_realm     | 0                  | 5 (5 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | digitorus       | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | dnscommonsrv    | 0                  | 5 (5 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | myssl           | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | nsec            | 0                  | 3 (3 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | rapiddns        | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | riddler         | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | sitedossier     | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | subdomaincenter | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | threatminer     | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | urlscan         | 0                  | 1 (1 DNS_NAME)               |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | ipneighbor      | 0                  | 1 (1 IP_ADDRESS)             |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | PTR             | 0                  | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | SOA             | 0                  | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | NS              | 0                  | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | AAAA            | 0                  | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] aggregate: | host            | 0                  | 0                            |
[INFO] aggregate: +-----------------+--------------------+------------------------------+
[INFO] output.csv: Saved CSV output to /home/kali/.bbot/scans/scheming_kyle/output.csv
[INFO] output.human: Saved TXT output to /home/kali/.bbot/scans/scheming_kyle/output.txt
[INFO] output.json: Saved JSON output to /home/kali/.bbot/scans/scheming_kyle/output.ndjson
[INFO] output.subdomains: Saved subdomains to /home/kali/.bbot/scans/scheming_kyle/subdomains.txt
[SUCC] Scan scheming_kyle completed in 40 seconds with status FINISHED
[INFO] Saved word cloud (9 words) to /home/kali/.bbot/scans/scheming_kyle/wordcloud.tsv

Set up go to run in bash/zsh:

nano ~/.bashrc
nano ~/.zshrc

export GOPATH=$HOME/go 
export GOROOT=/usr/lib/go 
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin 

source ~/.bashrc
source ~/.zshrc

Set up go to run in fish:

vim ~/.config/fish/config.fish

set -x GOPATH $HOME/go
set -x PATH $PATH $GOPATH/bin

source ~/.config/fish/config.fish

Frameworks

Tools

  • recon-ng
  • Spiderfoot
  • sn0int
  • etc. Check in Kali and online for more.

recon-ng

hackertarget.py

[recon-ng][default] > marketplace search

[recon-ng][default] > marketplace install hackertarget

[recon-ng][default] > modules load hackertarget

[recon-ng][default][hackertarget] > info

[recon-ng][default][hackertarget] > options set SOURCE domain.com

[recon-ng][default][hackertarget] > run

[recon-ng][default][hackertarget] > show hosts

[recon-ng][default][hackertarget] > back
[recon-ng][default] >

profiler.py

Dec 12, 2023: If your profiler.py is printing errors in red here is probably how you can fix it:
github.com/lanmaster53/recon-ng-marketplace/pull/246

To make it easier, I put all the file locations and modifications here:

recon-ng-marketplace Edit #1
Edit: /home/kali/.recon-ng/modules/recon/profiles-profiles/profiler.py at line 30.

   def module_thread(self, site, user):
        d = dict(site)
        # if d['valid'] == True:
        if d.get('valid', True) == True:
            self.verbose(f"Checking: {d['name']}")

recon-ng-marketplace Edit #2
Edit: /home/kali/.recon-ng/modules/recon/profiles-profiles/profiler.py at line 30.

meta = {
  'name': 'OSINT HUMINT Profile Collector',
  'author': 'Micah Hoffman (@WebBreacher), Brendan Burke (@gbinv)',
  # 'version': '1.1',
  'version': '1.2',
  'description': 'Takes each username from the profiles table and searches a variety of web sites for those users. The list of valid sites comes from the parent project at https://github.com/WebBreacher/WhatsMyName',
  'comments': (

recon-ng-marketplace Edit #3
Edit: /home/kali/.recon-ng/modules.yml on line 1101 and 1105.

  files: []
  last_updated: '2023-12-30'
  name: OSINT HUMINT Profile Collector
  path: recon/profiles-profiles/profiler
  required_keys: []
  version: '1.2'
- author: Robert Frost (@frosty_1313, frosty[at]unluckyfrosty.net)
  dependencies: []

After updating those files you should be able to run it.

[recon-ng][default] > marketplace install profiler

[recon-ng][default] > modules load profiler

[recon-ng][default][profiler] > info

[recon-ng][default][profiler] > options set SOURCE thecybermentor

[recon-ng][default][profiler] > run

[recon-ng][default][profiler] > show profiles

Maltego

A great tool, especially if you have access to API keys.

  • Create an account, install maltego on Kali, launch it and log in.
  • API keys are needed for most of the Hub Partners, but it some free recon.
  • Click the + at the top right to open a new graph.
  • Type domain in the search and drag it over to the graph area.
  • Rename the domain to the target.
  • Right click on the Domain Entity and select from the list of Transforms.
  • Testing All Transforms here.
    • Click on the “double” arrow to begin.
    • Click on the + area to see what it does and the arrow to return.
  • After setting the Required Inputs, click Run.
  • In the results, click on anything to see more, add notes, etc.
  • You can select an item from the results and run another Transform on it.
    • Emails, domains, companies, etc.
    • Then select another item from here and run a Transform again.
  • If you run it on a company that’s been around for a while and has a decent presence, you should gather a lot of data like emails, DNS entries, subdomains, IP addresses, open ports/services, locations, people, etc.

Other Tools

tracelabs.org
Trace Labs is a nonprofit organization whose mission is to accelerate the family reunification of missing persons while training members in the tradecraft of open source intelligence (OSINT).

hunch.ly

  • Pricing: 30 day free trial, $129.99 per year, Team (quote).
  • Setup Example
    • Create a new case.
    • Fill out “Selectors” for things you want to find.
      • username, name, email address, etc.
      • Keep adding new items that you find.
      • Filter results in History for each Selector.
    • Fill out “Tags” for grouping findings.
      • Breached Passwords
      • Social Media
      • Videos, Images
      • Phone Numbers
      • People
      • Set up default tags.
      • Filter results in History for each Tag.
    • Create a To Do list.
    • Check Settings that you may want.
      • Highlight Selectors
    • Webpages will be listed in history.
    • Go to google and turn on with “Capture” toggle.
    • Assign captures to the new case.
    • Start searching on google.
    • Start opening websites that you find.
    • You may start finding selectors that you created.
    • Right click on page for some options.
    • Assign tags to websites.
    • Add notes, captures images.
    • In the History you can check google cache or wayback machine.
    • When you’re done you can export all the data as a report if necessary.