JupiterOne & Project Discovery: Automating Nuclei with JupiterOne

At JupiterOne, we empower security practitioners to get deeper insights faster with contextual information about your cyber assets through a security Knowledge Graph. This context is accessible for developers (via API) to create or enhance workflows and augment their datasets. JupiterOne can enhance any off-the-shelf tools and capitalize on what they do best for deeper insights into all your cyber assets.

To show what is possible, we are releasing j1nuclei, an open-source tool to automate vulnerability scanning. The tool is available to anyone using the free version of JupiterOne (start here) by downloading the security scanner from ProjectDiscovery. j1nuclei combines the superpowers of JupiterOne and Nuclei to offer complete coverage of your endpoints, no matter how complex or ephemeral your environment is.

In this article, we’ll discuss the basics of leveraging j1nuclei. We’ll follow up in a later blog about using JupiterOne for continuous scanning.

j1nuclei overview

j1nuclei is a Python module that relays information between JupiterOne and Nuclei. It automates the extraction of vulnerability scanning targets, runs Nuclei, and brings back the findings to JupiterOne. You can analyze the results using our query language (J1QL) or JupiterOne Insights dashboards.

‍

Installation

Installation is simple through python pip.

1. Install nuclei

pip install nuclei

2. Install j1nuclei

pip install j1nuclei

3. Launch j1nuclei from a console or terminal

usage: j1nuclei [-h] -a ACCOUNT [-c CONCURRENT] [-n NUCLEI_PARAMS] [-q QUERY_FILE] [-apikey-env APIKEY_ENV] [-r NUCLEI_REPORT_PATH]

Step 1 - Target discovery

As each environment becomes more hybrid, geo-distributed, and ephemeral, getting an accurate list of assets can take time and effort. JupiterOne provides a comprehensive view of all your assets. You can identify hosts to scan with JupiterOne Query Language (J1QL).

For example, we can ask, “What are all my domains?” 

FIND Domain

A complex question like “What endpoints are exposed to everyone on the internet?” can be answered via a query:

Find (Everyone|Internet) that ALLOWS * as target 

RETURN target.url 

Because getting a comprehensive view may require several queries, j1nuclei automatically runs all questions in the target_query.json file. By default, the file is populated with common queries but can be extended with your queries or those found in the JupiterOne Questions library. Each extension query must retrieve the _key,  _integrationInstanceId, and _source properties used to maintain the state between JupiterOne and Nuclei.

Give me all endpoints exposed to everyone on the internet.

Find (Everyone|Internet) that ALLOWS * as d  

RETURN

d._key as key,

coalesce(d.baseUrl, d.url) as target,

d._integrationInstanceId as scope,

d._source as source

Give me all domains.

FIND Domain as d 

RETURN

d._key as key,

d.displayName as target,

d._integrationInstanceId as scope,

d._source as source

Give me all my data store endpoints.

FIND DataStore as d 

RETURN

d._key as key,

d.baseUrl as target,

d._integrationInstanceId as scope,

d._source as source

Give me all my public IP addresses.

FIND IpAddress 

WITH tag.Production = true and publicIp != "" as ip

RETURN

ip._key as key,

ip.publicIp as target,

ip._integrationDefinitionId as scope,

ip._source as source

You can experiment with queries through our web console or using our command-line utility jupiterone-client-nodejs.

Step 2 - Scanning

This step is quite simple. It passes the target to Nuclei directly through the -u or -target command-line arguments. The tool uses Python concurrency to launch multiple instances. You can control the number of concurrent Nuclei to launch with the j1nuclei -c argument (5 by default).

Step 3 - Bring learnings back

After the scan completes, the tool parses the Nuclei findings and brings the learnings back through our bulk synchronization API (documentation).

The Nuclei findings are connected back to their original target in this schema.

‍

The data is available using our standard console, through J1QL, and our Insights dashboard. Below is a FAQ about the findings identified using J1QL. Converting the FAQ into an automated dashboard is easy using our J1 Insights dashboard. You can use the dashboard provided as part of j1nuclei (dashboard_nuclei_port.json) or create your own.

‍

Exploring results with J1QL

Q. How many Nuclei issues do I have?

FIND nuclei_finding as f 

RETURN count(f) as value

Q. How many of my production critical assets are affected?

FIND *

WITH tag.Production = true AND classification = 'critical' AS asset

THAT HAS >> nuclei_finding

RETURN COUNT(asset)

Q. How many endpoints are affected?

FIND UNIQUE * as asset

THAT HAS >> nuclei_finding

RETURN count(asset) as value

Q. What is the criticality of the issues?

FIND nuclei_finding as f

RETURN f.severity as x, count(f) as y

Q. Show me my issues as a graph.

FIND *

THAT HAS >> nuclei_finding

THAT IS >> nuclei_vulnerability

RETURN TREE

JupiterOne Insight Dashboard