shodan
A small Haskell client for the Shodan API.
Licensed under the BSD 3-Clause License.
Current version: 0.3.0.0. See the changelog
for additions and the SearchMatch constructor change requiring a PVP major bump.
This project supports GET /api-info (ApiInfo and UsageLimits),
GET /shodan/host/count (CountResult), and one-page
GET /shodan/host/search (SearchResult). The API-info example fields and
the count response's total field are required; unknown fields are ignored.
Numeric values use Integer; count totals must be nonnegative.
Usage
The executable example in examples/api-info/Main.hs reads the API key from
SHODAN_API_KEY, calls /api-info, and prints the plan and remaining query/scan
credits. In Bash, enter the key without displaying it or putting it in a command:
read -r -s -p "Shodan API key: " SHODAN_API_KEY
printf '\n'
export SHODAN_API_KEY
cabal run shodan-api-info
Missing or blank keys are rejected before making a request. Errors go to stderr
with a nonzero exit status; the key is not printed. This command calls the live
Shodan API when a key is present.
To use the library in your own program:
import qualified Data.Text as Text
import Shodan
import System.Environment (getEnv)
main :: IO ()
main = do
key <- Text.pack <$> getEnv "SHODAN_API_KEY"
client <- newClient key
result <- getApiInfo client
case result of
Left err -> print err
Right info -> print (apiInfoPlan info, apiInfoQueryCredits info)
Reuse the client for multiple calls. newClient creates a TLS manager with
certificate verification. newClientWithManager lets applications share their
own TLS-capable http-client manager.
getApiInfo returns IO (Either ShodanError ApiInfo). Errors distinguish
transport failures (NetworkError), non-200 HTTP statuses (HttpError status),
and invalid successful responses (InvalidResponse). Raw requests, response
bodies, and exception messages are omitted from errors to avoid exposing the
key. Showing a Client also hides its key. Custom manager hooks can access the
request and must not log the key.
Each request has a 30-second response timeout. Redirects are not followed and
there are no automatic retries. Thread cancellation is allowed to propagate.
Count matching results
With SHODAN_API_KEY set, run:
cabal run shodan-count -- 'port:443 country:KR'
Pass the query as one argument, quoting it when it contains spaces. The example
prints only the total count. Missing/blank queries or keys are rejected; API
errors are printed to stderr with a nonzero exit status.
The library function is:
countHosts :: Client -> Text -> IO (Either ShodanError CountResult)
Read the total with countTotal. For individual search results, use
searchHosts as described below. Optional facet requests are not supported yet.
According to the official count endpoint
documentation, this call uses no query credits.
Facet response type
Facet represents a single statistics bucket such as
{"value":"Amazon.com","count":3012386}. Its FromJSON instance is available
for standalone parsing and is used by CountResult:
data CountResult = CountResult
{ countTotal :: Integer
, countFacets :: Map Text [Facet]
}
countFacets groups buckets by name, such as country or port. Use
Data.Map.Strict.lookup "country" (countFacets result) to read a breakdown.
An absent facets field or an empty object produces an empty map. A present
facets field must be an object of lists: nulls, incorrect shapes, and invalid
buckets are rejected. Bucket order in each list is preserved.
countHosts still sends only the query and key; requesting facets over HTTP
will be added separately.
facetValue :: Value preserves the JSON value, including strings, numbers,
and explicit nulls, without coercing it to text.
facetCount :: Integer stores a nonnegative result count.
Both fields are required. Unknown fields are ignored, while missing fields,
invalid counts, and non-object buckets are rejected.
Search
Version 0.2.0.0 adds one-page searches:
searchHosts :: Client -> Text -> Int -> IO (Either ShodanError SearchResult)
The page number starts at 1. Values below 1 return Left InvalidPage before
any request. Each call retrieves exactly the requested page; additional pages
are not fetched automatically. Shodan requires pages past 100 to be requested
in order.
With SHODAN_API_KEY set, the executable accepts a query and an optional page
(default 1):
cabal run shodan-search -- 'product:nginx country:KR'
cabal run shodan-search -- 'product:nginx country:KR' 2
These commands call Shodan and can use query credits: a filtered query or any
page after the first uses a credit under the official API
policy. The example prints the total across
all pages, the selected page number, the returned service count, and a table of
IP, port, product, version, and ASN. Missing product/version/ASN values are displayed as -.
SearchMatch describes one matching service with matchIp, matchPort,
matchProduct, matchVersion, and matchAsn. IP text and an integer port from 0 through 65535
are required. Product, version, and ASN are optional: missing or null values become
Nothing, while other non-text values are rejected. IP syntax is not validated.
Unknown banner fields are ignored.
SearchResult contains searchTotal, searchMatches, and searchFacets.
The total covers all pages, while the matches describe one page and preserve
service order (multiple services may share an IP). Total and matches are
required; the total is a nonnegative Integer. Facets follow the same parsing
rules as CountResult.
Optional facet requests and field selection are not supported yet. Search
tests use local JSON fixtures and a mock HTTP server, and do not consume Shodan
query credits. A one-page live search has also succeeded manually.
Build
The initial development compiler is GHC 9.8.4.
cabal build
cabal test
Approach
- Use ordinary records, sum types, and explicit IO functions.
- Add dependencies and language extensions only when needed.
- Keep API keys out of logs and error messages.
- Implement one endpoint at a time, beginning with
GET /api-info.
Use stylish-haskell 0.14.6.0 with the checked-in .stylish-haskell.yaml.
To install it with GHC 9.8:
cabal install stylish-haskell-0.14.6.0 --flags=-ghc-lib --constraint='ghc-lib-parser-ex +no-ghc-lib'
With the Cabal executable directory on PATH, run from the project directory:
stylish-haskell -i src/Shodan.hs test/*.hs examples/api-info/Main.hs
stylish-haskell -i examples/count/Main.hs
stylish-haskell -i examples/search/Main.hs
cabal test
Files
shodan.cabal: package and library settings.
cabal.project: local Cabal project settings.
CHANGELOG.md: release history and compatibility notes.
src/Shodan.hs: client, errors, response records, and JSON parsing.
examples/api-info/Main.hs: executable example using an environment variable.
examples/count/Main.hs: count-query executable using the same environment variable.
examples/search/Main.hs: one-page search executable with an optional page argument.
test/Main.hs: response parsing and HTTP tests against a local mock server.
test/SearchTests.hs: search-response JSON parsing tests.
No API key is needed to build or test. Automated tests do not call Shodan.
They check request encoding, successful responses, HTTP errors, invalid JSON,
redirect handling, a simulated transport failure, and count responses (including
zero, large totals, malformed responses, and query encoding). Live authenticated
HTTPS calls to /api-info and /shodan/host/count have also succeeded manually;
live calls are not part of the automated test suite.
Standalone facet parsing tests also cover the documented organization example,
numeric and null values, zero and large counts, lists of buckets, unknown fields,
and malformed or invalid buckets.