Technologies: A Complete Guide to Web Technology Detection and Fingerprinting Tools
Technologies: A Complete Guide to Web Technology Detection and Fingerprinting Tools
Technology detection and fingerprinting are essential phases in reconnaissance. Knowing what frameworks, servers, libraries, and services a target uses helps you identify potential vulnerabilities, choose the right exploits, and map the attack surface efficiently. This guide covers eleven powerful tools for detecting web technologies, JavaScript libraries, WAFs, CDNs, GraphQL engines, and more.
⚠️ Legal and ethical notice: These tools interact with target systems, often making HTTP requests or querying public APIs. Only use them against systems you own or have explicit written authorization to test. Unauthorized scanning may violate laws and terms of service.
Table of Contents
- Why Technology Detection Matters
- Tool Overview
- webanalyze
- python-builtwith
- whatweb
- retire.js
- httpx
- fingerprintx
- graphw00f
- wafw00f
- cdncheck
- tlsx
- MurMurHash
- Comparison Table
- Best Practices & Workflow
- Final Tips
Why Technology Detection Matters
Every web application is built on a stack of technologies: web servers (Nginx, Apache), programming languages (PHP, Python, Node.js), frameworks (Laravel, Django, React), CMSs (WordPress, Drupal), JavaScript libraries (jQuery, Angular), and security layers (WAFs, CDNs). Identifying these components provides:
- Attack surface mapping – Know what software is running to focus your testing.
- Vulnerability correlation – Match detected versions against known CVEs.
- Bypass strategies – Understand WAFs and CDNs to craft effective payloads.
- Efficiency – Avoid wasting time on irrelevant tests.
The tools in this guide automate technology detection at scale, from single-host analysis to mass scanning.
Tool Overview
| Tool | Language | Primary Purpose | Best For |
|---|---|---|---|
| webanalyze | Go | Port of Wappalyzer for mass scanning | Bulk technology detection |
| python-builtwith | Python | BuiltWith API client | API-based tech profiling |
| whatweb | Ruby | Next-generation web scanner | Detailed fingerprinting |
| retire.js | JavaScript | Detect vulnerable JS libraries | Client-side library auditing |
| httpx | Go | Fast multi-purpose HTTP toolkit | Probing and tech detection |
| fingerprintx | Go | Service discovery on open ports | Port-based service fingerprinting |
| graphw00f | Python | GraphQL engine fingerprinting | GraphQL endpoint analysis |
| wafw00f | Python | WAF detection and fingerprinting | Identifying WAF products |
| cdncheck | Go | Identify CDN/WAF from DNS/IP | CDN and WAF mapping |
| tlsx | Go | TLS data collection and analysis | TLS certificate analysis |
| MurMurHash | Python | Calculate favicon MurmurHash | Finding similar sites via favicon |
webanalyze
webanalyze is a Go port of Wappalyzer that uncovers technologies used on websites, designed for mass scanning.
Installation
go install github.com/rverton/webanalyze/cmd/webanalyze@latest
Basic Usage
# Single URLwebanalyze -host https://example.com# Multiple URLs from a filewebanalyze -hosts urls.txt -output results.json# Update the technology databasewebanalyze -update
Key Features
- Fast, concurrent scanning
- Uses the Wappalyzer technology database
- JSON and CSV output
- Supports custom technology definitions
- Can read from stdin
When to Use
Use webanalyze when you need to quickly identify technologies across a large list of URLs. It is ideal for initial recon and asset inventory.
python-builtwith
python-builtwith is a Python client for the BuiltWith API, allowing you to query technology profiles for domains.
Installation
pip install builtwith
Basic Usage
import builtwithresult = builtwith.parse('https://example.com')print(result)
Key Features
- Access to BuiltWith's extensive technology database
- Simple Python API
- Returns detailed technology categories
- Requires a BuiltWith API key for full access
When to Use
Use python-builtwith when you have a BuiltWith API subscription and need programmatic access to technology data. It is suitable for integration into Python-based recon pipelines.
whatweb
whatweb is a next-generation web scanner that identifies technologies, version numbers, and other details.
Installation
# Kali Linuxsudo apt install whatweb# From sourcegit clone https://github.com/urbanadventurer/WhatWebcd WhatWebbundle install
Basic Usage
# Single URLwhatweb example.com# Aggressive modewhatweb -a 3 example.com# Scan multiple URLs from a filewhatweb -i urls.txt --log-json=results.json
Key Features
- Over 1,800 plugins for technology detection
- Aggression levels from stealthy to heavy
- JSON, XML, and human-readable output
- Plugin development support
- Can perform version detection
When to Use
Use whatweb when you need detailed, plugin-driven fingerprinting of individual sites or small lists. It is more thorough than webanalyze but slower.
retire.js
retire.js is a scanner that detects the use of JavaScript libraries with known vulnerabilities.
Installation
npm install -g retire
Basic Usage
# Scan a URLretire --url https://example.com# Scan a local directoryretire --path ./js# Output JSONretire --url https://example.com --outputformat json
Key Features
- Detects vulnerable JavaScript libraries
- Uses a database of known CVEs
- Can scan URLs, local files, and directories
- Integrates with CI/CD pipelines
- Supports custom vulnerability databases
When to Use
Use retire.js when auditing client-side JavaScript dependencies. It is essential for finding outdated libraries like jQuery, Angular, or Bootstrap with known vulnerabilities.
httpx
httpx is a fast and multi-purpose HTTP toolkit that allows you to run multiple probers using a retryable HTTP library. It is designed for reliability with increased threads.
Installation
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest
Basic Usage
# Probe a list of hostscat hosts.txt | httpx -silent -o live.txt# Technology detectioncat hosts.txt | httpx -tech-detect -o tech.txt# Full probe with status, title, and techcat hosts.txt | httpx -status-code -title -tech-detect -o results.txt
Key Features
- Fast, concurrent HTTP probing
- Technology detection (
-tech-detect) - Status code, title, content length, and more
- Supports custom headers, proxies, and rate limiting
- Pipeline-friendly
When to Use
Use httpx as a Swiss Army knife for HTTP probing. It is often the first step after subdomain enumeration to identify live web servers and their technologies.
fingerprintx
fingerprintx is a standalone utility for service discovery on open ports that works well with other popular bug bounty command-line tools.
Installation
go install github.com/praetorian-inc/fingerprintx/cmd/fingerprintx@latest
Basic Usage
# Scan a single host:portfingerprintx -t example.com:80# Scan a list of targetsfingerprintx -f targets.txt -o results.json# Scan a CIDR rangefingerprintx -t 192.168.1.0/24
Key Features
- Identifies services on open ports
- Supports many protocols (HTTP, SSH, FTP, SMB, etc.)
- JSON output
- Fast and lightweight
- Integrates with other tools
When to Use
Use fingerprintx after port scanning to identify what services are running on open ports. It complements Nmap's service detection with a focus on bug bounty workflows.
graphw00f
graphw00f is a GraphQL Server Engine Fingerprinting utility for software security professionals looking to learn more about what technology is behind a given GraphQL endpoint.
Installation
git clone https://github.com/dolevf/graphw00fcd graphw00fpip install -r requirements.txt
Basic Usage
python graphw00f.py -d https://example.com/graphql
Key Features
- Fingerprints GraphQL engines (Apollo, Hasura, etc.)
- Detects GraphQL endpoints
- Provides detailed engine information
- Supports custom headers and authentication
When to Use
Use graphw00f when you encounter a GraphQL endpoint and need to identify the underlying engine. This helps tailor your GraphQL-specific attacks.
wafw00f
wafw00f allows one to identify and fingerprint Web Application Firewall (WAF) products protecting a website.
Installation
pip install wafw00f
Basic Usage
# Single URLwafw00f https://example.com# Multiple URLs from a filewafw00f -i urls.txt -o results.txt# Verbose outputwafw00f -v https://example.com
Key Features
- Detects over 150 WAF products
- Fingerprints WAF type and version
- Supports HTTP/HTTPS
- Can use a proxy
- Outputs to file
When to Use
Use wafw00f before launching attacks to understand what WAF is in place. This informs your evasion strategy.
cdncheck
cdncheck is a tool for identifying the technology associated with DNS / IP network addresses.
Installation
go install github.com/projectdiscovery/cdncheck/cmd/cdncheck@latest
Basic Usage
# Check a single IPcdncheck -i 1.1.1.1# Check a list of IPscdncheck -iL ips.txt -o results.json# Check a domaincdncheck -d example.com
Key Features
- Identifies CDN, WAF, and cloud providers
- Works with IPs, CIDRs, and domains
- JSON output
- Fast and lightweight
- Integrates with ProjectDiscovery tools
When to Use
Use cdncheck to determine if a target is behind a CDN or WAF, which affects how you approach scanning and exploitation.
tlsx
tlsx is a fast and configurable TLS grabber focused on TLS-based data collection and analysis.
Installation
go install github.com/projectdiscovery/tlsx/cmd/tlsx@latest
Basic Usage
# Grab TLS certificate from a hosttlsx -u example.com:443# Scan a list of hoststlsx -l hosts.txt -o results.json# Extract certificate detailstlsx -u example.com -cert -json
Key Features
- Fast TLS handshake and certificate grabbing
- Extracts certificate details (SANs, issuer, expiry)
- Supports SNI, custom ports, and concurrency
- JSON output
- Integrates with other tools
When to Use
Use tlsx to collect TLS certificates from hosts. This can reveal additional subdomains (via SANs), expired certificates, and misconfigurations.
MurMurHash
MurMurHash is a small tool to calculate a MurmurHash value of a favicon. This favicon hash can be used to look for similar websites on various search engines.
Installation
git clone https://github.com/m4ll0k/MurmurHashcd MurmurHashpip install -r requirements.txt
Basic Usage
python murmurhash.py -u https://example.com/favicon.ico
Key Features
- Calculates MurmurHash of a favicon
- Output can be used in Shodan searches (
http.favicon.hash:<hash>) - Helps find related infrastructure
- Simple and fast
When to Use
Use MurMurHash when you want to find other websites using the same favicon, which often indicates shared infrastructure or the same organization.
Comparison Table
| Tool | Language | Primary Focus | Output | Speed |
|---|---|---|---|---|
| webanalyze | Go | Technology detection | JSON/CSV | Fast |
| python-builtwith | Python | BuiltWith API | Python dict | API-dependent |
| whatweb | Ruby | Detailed fingerprinting | JSON/XML | Medium |
| retire.js | JavaScript | Vulnerable JS libraries | JSON | Fast |
| httpx | Go | HTTP probing + tech detect | Text/JSON | Very fast |
| fingerprintx | Go | Service discovery | JSON | Fast |
| graphw00f | Python | GraphQL engine fingerprinting | Text | Medium |
| wafw00f | Python | WAF detection | Text | Medium |
| cdncheck | Go | CDN/WAF identification | JSON | Fast |
| tlsx | Go | TLS certificate analysis | JSON | Fast |
| MurMurHash | Python | Favicon hash | Text | Fast |
Best Practices & Workflow
- Start with httpx. Probe live hosts and detect technologies with
-tech-detect. - Use webanalyze for bulk scanning. When you have thousands of URLs, webanalyze is faster.
- Follow up with whatweb. For critical targets, run whatweb in aggressive mode for detailed fingerprinting.
- Check client-side libraries. Run retire.js on any web application to find vulnerable JavaScript.
- Identify WAFs and CDNs. Use wafw00f and cdncheck to understand defenses.
- Analyze TLS. Use tlsx to extract certificate data and discover additional subdomains.
- Fingerprint GraphQL. If you find a GraphQL endpoint, run graphw00f.
- Favicon hashing. Use MurMurHash to find related sites via Shodan.
- Combine results. Merge findings into a comprehensive technology profile for each target.
Example Workflow
# 1. Find live hostscat subdomains.txt | httpx -silent -o live.txt# 2. Bulk technology detectionwebanalyze -hosts live.txt -output tech.json# 3. Detailed fingerprinting on interesting hostswhatweb -i interesting.txt -a 3 --log-json=whatweb.json# 4. Check JavaScript librariesretire --path ./js --outputformat json# 5. WAF detectionwafw00f -i live.txt -o waf.txt# 6. CDN checkcdncheck -iL ips.txt -o cdn.json# 7. TLS analysistlsx -l live.txt -cert -json -o tls.json# 8. Favicon hash for Shodanpython murmurhash.py -u https://example.com/favicon.ico
Final Tips
- Automate with pipelines. These tools are designed to work together; pipe output from one to another.
- Keep databases updated. webanalyze, retire.js, and wafw00f rely on up-to-date signatures.
- Respect rate limits. When scanning many hosts, use concurrency controls to avoid overwhelming targets.
- Combine passive and active. Use API-based tools (python-builtwith) alongside active scanners.
- Document findings. Technology profiles are valuable for reporting and future assessments.
- Stay authorized. Always obtain written permission before scanning any system.
Official Resources:
- webanalyze: github.com/rverton/webanalyze
- python-builtwith: github.com/richardpenman/builtwith
- whatweb: github.com/urbanadventurer/WhatWeb
- retire.js: github.com/RetireJS/retire.js
- httpx: github.com/projectdiscovery/httpx
- fingerprintx: github.com/praetorian-inc/fingerprintx
- graphw00f: github.com/dolevf/graphw00f
- wafw00f: github.com/EnableSecurity/wafw00f
- cdncheck: github.com/projectdiscovery/cdncheck
- tlsx: github.com/projectdiscovery/tlsx
- MurMurHash: github.com/m4ll0k/MurmurHash
Remember: Technology detection is a powerful reconnaissance technique. Use it responsibly, stay within scope, and always obtain proper authorization before testing any system.
Author
Discussion · 0
No comments yet — start the discussion.
Sign in to join the discussion.