How to Add a Newly Registered Domains Feed to MISP

Quick answer. A newly registered domain (NRD) feed in MISP flags how old a domain is at the moment an analyst looks at it. WhoisFreaks publishes NRD data as a local MISP feed directory, so MISP pulls it on its own schedule with no API key between the two systems. Run it in cache-only mode and you get the freshness signal and the exact registration date, with zero events, zero attributes, and zero correlation rows in the database.

Cache or fetch? Start here

MISP has two independent ways to consume a feed, and picking the wrong one is the mistake that costs people a weekend. The answer for an NRD feed is cache-only, unless you specifically need attributes as searchable objects.

Cache-only gives you the freshness signal at triage time, the exact registration date, a window that genuinely rolls, and no database growth. What you give up is attributes as first-class objects: you cannot search, filter, tag or pivot on them, and /attributes/restSearch will not find them. If a SOAR playbook needs to query this programmatically rather than an analyst reading it on screen, verify MISP's feed-cache search covers your case before assuming cache-only is enough.

Fetch mode remains supported and is right if you need real attributes. Go in knowing it does not respect the retention window, and plan a purge.

The rest of this post is why. Both of those answers cost us a reversal to find.

Why domain age is context, not a verdict

Domain age is the only useful signal you can have about a domain before anyone has decided it's malicious.

Everything else in threat intelligence is retrospective. A domain lands on a blocklist because somebody got phished by it. It gets a reputation score because traffic to it was observed and classified. All of that is evidence of harm that has already happened. Registration date isn't like that. It comes from the registry, the moment the domain exists, before it has done anything at all.

That matters because of how quickly the attack cycle runs. Research published in the Journal of Cybersecurity (Agarwal and Vasek, 2026) found that criminals deploy phishing sites on domains "as soon as they are registered." Palo Alto's Unit 42 documented the same pattern in malware infrastructure, including a domain observed serving C2 traffic on the day it was registered, and notes that malicious new domains are often alive for only a few hours before any vendor detects them.

By the time a domain shows up in a reputation feed, the campaign that used it has usually finished. The domain was, however, verifiably four hours old when the email arrived, and that fact was available at the time.

But a domain being three days old is not evidence of anything. The overwhelming majority of newly registered domains are entirely legitimate: businesses launching, people starting blogs, campaign microsites, companies rebranding. Treat "new" as "bad" and you'll block your customer's rebrand and a conference microsite in the same afternoon.

Some of them are malicious, and there is a separate feed for exactly that population. We'll come back to it, because the two need opposite handling and the reason why is the most transferable idea here.

The framing that works: when an analyst is triaging an alert and one of the domains in it was registered on Tuesday, that changes what they do next. It doesn't decide anything for them. It's the sort of thing that should already be on screen rather than something they go and look up in another tab.

That distinction between context and verdict drives nearly every technical decision that follows.

MISP feed vs the WhoisFreaks expansion module

WhoisFreaks has shipped a MISP expansion module for a while. An analyst with a domain in an event clicks enrich, the module calls our API, and back come the WHOIS and DNS details for that one domain.

It's useful and it is not this.

An expansion module is invoked per attribute, so it structurally cannot tell you about a domain nobody has looked at yet. But that's exactly the case that matters: freshness is most valuable on the domain sitting in an alert that hasn't been triaged.

MISP handles bulk data through an entirely separate subsystem called feeds. Different code path, different configuration surface, not a module at all. So the two integrations complement rather than overlap. The feed flags freshness broadly and ahead of time; the module goes deep on the one domain you decide to investigate.

Why this writes a feed directory instead of calling the MISP API

The obvious implementation is to use PyMISP to create events and push attributes through the API. That's the approach we started with, and it's wrong. Call it the first reversal.

A single update_event call carrying tens of thousands of attributes fails outright. The workable pattern is to build the event client-side and upload it in batches of about a thousand. For a 374,000-domain day that means nearly 400 sequential API calls, each of which can fail independently, and you own all the retry and partial-state logic that follows. MISP's feed subsystem exists precisely because the API is the wrong tool at this volume.

So the tool doesn't talk to MISP at all. It reads the WhoisFreaks NRD API and writes a MISP feed directory to disk:

WhoisFreaks NRD API  ->  per-day gzip cache  ->  MISP feed directory  ->  MISP pulls

Three files, which is the entire interface:

• manifest.json: a time-indexed list of events
• <event-uuid>.json: one file per feed-day
• hashes.csv: md5(value),event_uuid per line

MISP is pointed at the directory and does the ingesting itself, on its own schedule, with its own error handling. There's no API key between the two systems and no authentication to manage. If MISP is down, the feed just sits there until it isn't.

This also made the integration structurally identical to our other seven NRD tools, including BIND 9 RPZ, Suricata and Zeek. Write a file in the format the tool already understands, then let the tool consume it.

Two details worth stealing if you're implementing something similar.

Deterministic UUIDs. Event UUIDs are uuid5(namespace, "wf-nrd-event|<date>") and attribute UUIDs are uuid5(namespace, "wf-nrd-attr|<date>|<domain>"). Re-running for a day updates that event rather than creating a duplicate, which makes the whole thing idempotent and safe to run from cron with no state tracking at all. This paid off unexpectedly during testing. We accidentally triggered two concurrent fetch jobs on the same feed, and because both were inserting the same UUIDs, MISP upserted instead of duplicating. With random UUIDs that would have been 800,000 attributes and a cleanup job.

Timestamps that track content, not runs. A day's event timestamp changes only when that day's domain set actually changes. Get this wrong and every nightly run looks to MISP like a new version of every attribute in the window, and it re-ingests millions of rows for nothing.

Why NRD feeds need correlation disabled in MISP

This is the part that decides whether the integration is usable.

MISP correlates attributes: every incoming value is compared against everything already in the database, and matches are written to a correlation table. The cost scales with new attributes multiplied by the size of the existing corpus.

Now do the arithmetic. All figures below are gTLD and ccTLD combined, measured at 373,759 domains per day across a seven-day window:

Eleven million correlating domains on a modest instance means a correlation table larger than the rest of the database combined, ingest measured in hours, and event views that time out because rendering them requires reading that table.

And the correlations you'd get are close to worthless. Set aside performance for a moment and ask what they'd actually tell you. The bulk of them are NRD attributes matching each other, for the sole reason that all of these domains were registered recently. That's a property of the feed, not a relationship between the domains. Every event view fills with matches meaning "this is also a new domain," which you knew, because that's why it's in the feed.

So the feed ships with disable_correlation: true and to_ids: false on every attribute, plus the feed-level Disable correlation setting in MISP as a second layer. That setting is documented in the MISP feed management guide: "If this is checked, correlations will be disabled for all events coming from this Feed."

Measured on a live instance after ingesting a full seven-day window:

attributes:    2,616,312
correlations:            0

disable_correlation | to_ids | count
                  1 |      0 | 2,616,312

Two and a half million domains, not one correlation row, and every single attribute carrying the flags we intended.

Critically, this costs you nothing you wanted. Turning off correlation does not turn off feed lookups. Those run through MISP's Redis feed cache, which is a separate mechanism: a matching domain shows up as a "Feed hit" on the event you're viewing, and it deliberately doesn't enter the correlation graph. You keep the lookup and pay none of the correlation cost.

Why the rolling window is fictional in fetch mode

Here's the bug we shipped in our own documentation, and how we found it. The second reversal.

The tool maintains a rolling window. Each day's event is pruned from the manifest once it falls outside the retention period. Our README said a domain ages out because its event drops out of the manifest, "so MISP's own feed handling retires it."

That was an assumption, not a finding. So we tested it: narrowed the window until two event files remained on disk, then looked at MISP.

MISP still listed all seven previously ingested events.

Feed ingestion is additive. Nothing about removing an event from a manifest causes MISP to delete an event it has already imported. Which means that in fetch mode, a seven-day window doesn't give you seven days in MISP. It gives you seven days arriving per week and nothing ever leaving. Two months in, you're holding roughly two months of events, on the order of 22 million attributes. Precisely the unbounded database growth the rest of the design works to avoid.

The rolling window was real on disk and fictional in MISP.

Caching behaves differently. Narrow the window, re-cache, search for a domain from a pruned day, and it's gone. The cache genuinely rolls, because it's rebuilt from the current hashes.csv rather than accumulated.

How cache-only keeps the registration date with nothing ingested

Cache-only looked attractive but seemed to carry a real cost: with no events in the database, surely you lose the registration date and are left with only "somewhere in the last seven days"? The date is the valuable part.

So we deleted every event and attribute, down to zero, left the Redis cache in place, and searched again.

The cache hit still reported the matching event's ID, its date, and its domain count. On an instance holding no events and no attributes whatsoever.

The mechanism is that MISP resolves event metadata from manifest.json on demand, using the same machinery that powers feed preview. hashes.csv supplies the event UUID for the matched domain; the manifest supplies that event's info string, which reads WhoisFreaks NRD - 2026-08-17 (374000 domains).

Which means a design decision made for an unrelated reason turned out to be the one that matters. One event per feed-day with the date in its info field was chosen for idempotent regeneration. It's also what makes registration-date attribution survive with nothing ingested.

One operational note if you do choose fetch: Caching enabled on a feed only marks it as eligible. Nothing is cached until you explicitly trigger Cache all feeds, a fact that costs people an afternoon, because a fetched-but-uncached feed looks half-broken.

Adding the WhoisFreaks threat feeds alongside NRD

Everything above treats domain age as context. That's the right call for NRD data, and it's also a limitation. The feed tells you a domain is young, never that it's dangerous. Most young domains aren't.

WhoisFreaks publishes the other half separately, as the Domain Threat Intelligence Feeds: curated lists of domains observed doing something specific. This integration publishes three of them today.

Each list begins from confirmed seed domains, extracts the infrastructure those seeds share (registrant email, phone, organization, NS, MX, CNAME), and matches those pivots across the WhoisFreaks domain database to surface related domains, often before they reach public blocklists. Every row carries a confidence score, first-seen and last-seen timestamps, and the number of pivots that linked it. All three share one CSV schema, documented in the threat feed documentation.

The same tool publishes them, as a second MISP feed:

sudo -u misp-nrd misp-nrd-feed --threat

Every setting inverts, and that's the interesting part

to_ids: true because you genuinely want these in your IDS rulesets. Pushing every newly registered domain into Suricata would be absurd at 374,000 a day. Exporting a curated list of domains seen serving malware is the entire point.

Correlation enabled, for the same reason it's disabled for NRD. Above, correlations were noise: millions of domains matching each other solely because they were all registered recently. Here a match means an indicator you already hold has been independently flagged as malicious. That is the single most valuable thing MISP can tell you, and the volume makes it affordable.

One practical consequence: Disable correlation is a per-feed setting in MISP. One feed needs it ticked and the other needs it unticked, so they cannot be the same feed, which means they cannot share a directory. The tool refuses a configuration that points both at one path.

Withdrawal: making a retracted domain actually go away

This is the part to settle before enabling threat feeds. Threat attributes carry to_ids: true, so a domain that lingers after the provider withdraws it keeps firing IDS rules and correlating against your events. False positives do get retracted, so this is not hypothetical.

Each threat response is a full snapshot of current state, dated or not, which is a different delivery model from NRD. One request per feed per day is all it takes: no baseline to bootstrap, nothing to replay. That is also what makes withdrawal work at the feed level. Because the newest snapshot is authoritative, a domain the provider drops simply stops appearing, and the tool prunes its superseded event file:

day 1 published: ['false-positive.com', 'real-threat.com']
day 2 published: ['real-threat.com']

Attribute UUIDs for threat domains are deliberately not date-scoped, unlike NRD attributes. A flagged domain is one indicator regardless of which snapshot carried it, so a reappearance refreshes last_seen on the existing attribute instead of creating a second one.

Inside MISP is the part that needs a decision from you, because feed ingestion is additive there too: removing a domain from the feed does not remove it from MISP. Cache-only sidesteps this entirely, since the Redis cache is rebuilt from the current hashes.csv on every run. If you fetch, pair it with a scheduled purge. The deterministic attribute UUIDs, uuid5(namespace, "wf-threat-attr|<type>|<domain>"), mean you can compute the UUID of any withdrawn domain without querying MISP first.

If you fetch and do neither, withdrawn domains persist indefinitely with to_ids set. That is the failure mode to avoid, and it is why we suggest starting cache-only here too, then moving to fetch once a purge job exists. Full setup is in docs/THREAT-FEEDS.md.

Run both feeds and an analyst gets two independent signals on one screen: this domain is four days old, and it appears in the phishing feed. Either alone is weaker than the pair.

Benchmarks: feed generation and MISP ingest

Volume figures come from the WhoisFreaks Newly Registered Domains feed, measured across a seven-day gTLD and ccTLD window ending August 17, 2026. Performance figures are measured on a laptop-class machine, not projected.

Note the asymmetry: generating the feed is trivial, and essentially all elapsed time is MISP ingesting. That's the strongest practical argument for cache-only. It skips the expensive half entirely.

The ingest cost is also front-loaded. A backfill processes the whole window once; after that a daily run adds a single day, regardless of how wide the window is.

Install it

git clone https://github.com/WhoisFreaks/wf-misp-nrd-feed.git
cd wf-misp-nrd-feed
sudo bash scripts/install.sh --dry-run

The installer preflights everything it needs (Python version, venv support, useradd, SELinux state, egress, disk) and refuses with a specific fix rather than failing halfway through. docs/SETUP-LINUX.md walks the whole thing from an empty machine, including standing up MISP in Docker and the container volume mount a local feed needs. You need an account with the WhoisFreaks Newly Registered Domains product. MISP 2.5 is recommended, since 2.4 is in its security-fix-only end of life phase.

Same feed, other surfaces: Pi-hole and AdGuard Home for DNS blocking, BIND 9 RPZ for DNS firewalling, Rspamd and SpamAssassin for mail scoring, Suricata for IDS alerting (linked above), and Zeek for passive network analysis.

MISP is the analyst-facing one. It doesn't block anything. It just makes sure that when someone is deciding whether a domain matters, they already know how old it is.

Frequently asked questions

Should I use cache or fetch mode for an NRD feed in MISP?

Cache-only, unless you need attributes as searchable objects. Caching gives you the freshness signal, the registration date, and a retention window that actually rolls, in seconds rather than hours. Fetch mode creates real attributes but does not respect the window, so it grows without bound until you purge.

Does disabling correlation stop MISP from matching domains in the NRD feed?

No. Feed lookups run through MISP's Redis feed cache, which is separate from the correlation engine. A matching domain still appears as a "Feed hit" on the event you are viewing. Only the correlation table entries are suppressed.

Do I need a MISP API key to run this feed?

Not in the recommended setup. The tool writes a feed directory to disk and MISP pulls from it, so no credentials pass between the two systems. An API key is only needed if you run fetch mode and want to script a purge of aged-out events.

Can the NRD feed and the threat feeds share one MISP feed entry?

No. Disable correlation is a per-feed setting, and the two need opposite values: ticked for NRD, unticked for the threat feeds. They must be registered as two feeds pointing at two directories. The tool rejects a configuration that points both at the same path.

What happens when a domain is withdrawn from the threat feed?

Each threat response is a full snapshot, so a withdrawn domain stops appearing in the published feed immediately. Inside MISP it persists unless you are running cache-only, because feed ingestion is additive. If you fetch, schedule a purge, or those retracted domains keep firing IDS rules.