App-Netdisco
view release on metacpan or search on metacpan
stages, as found in the configured worker plugins and python worklets
* #1725 drop redundant indexes and add four that existing queries lack
(database schema 101: run netdisco-deploy)
* #1726 faster IP Inventory, and device search by complete MAC or IP address
* #1734 job log by device and device delete use their indexes
* #1730 warn of the PostgreSQL 14 floor, and show the version in red below
14 in Statistics
[BUG FIXES]
* #1717 let device_auth_tag_hint win over cached credentials again
* #1725 Subnet Utilization reports the size of IPv6 subnets correctly
* #1726 IP Inventory no longer lists a used address as never seen, and shows the most recently seen MAC for an IP with several
* #1730 Statistics shows PostgreSQL 10 and newer as 18.6, not 18.00.6
* #1731 discover no longer stops when one VRF's SNMP context does not answer
* #1731 SNMPv3 VRF contexts are queried by their exact name, such as
prod-red or 100, never with the VLAN context prefix
* API object last_node for a port no longer fails with a server error
2.111000 - 2026-10-08
* fix arbitrary module loading in the SNMP browser's munge parameter
* fix a privilege escalation in the queue API and the admin job controls,
where a job action naming a backend command worker could be queued
* fix an open redirect in the login form's return_url
* warn at startup when metrics_path is set with neither metrics_allow nor
metrics_token, so the endpoint is not published unrestricted unnoticed
* raise the bcrypt work factor for locally stored passwords. an existing
password is rewritten at the new factor on its owner's next successful
login, and safe_password_store still governs whether that happens
* bound the port count accepted when creating a pseudo device
* fix a command injection in the external credentials command, where a
device's DNS name was substituted into it unchecked
2.109000 - 2026-09-13
[ENHANCEMENTS]
* this release needs Safari 16.4, Firefox 101, Chrome 74 or Edge 79, or
newer. it is the first that will not work on an older browser
* pages are now drawn in the display mode the current framework is written
for. rows in a table stand a little taller as a result
* #1478 fixes vendor not known by netdisco
2.097000 - 2025-12-16
[NEW FEATURES]
* IOS-XE: add SSH ARP (multi-VRF) and MAC collectors (replaces IOSXEMac collector)
[ENHANCEMENTS]
* #1473 get_credentials can now also open a plain file
2.096001 - 2025-12-13
[ENHANCEMENTS]
* #1458 add login_banner_message setting to put text on the login page
[BUG FIXES]
* #1440 promote bare empty string to empty array if found on list custom field
* allow ACL property match to contain ":"
* fix skip long SNMP timeout for initial discover
2.063002 - 2023-07-14
[ENHANCEMENTS]
* #1059 NETDISCO_SNMP_BULKWALK_OFF environment to disable bulkwalk
* #1060 portctl_topology setting to enable manual topology for port control users
* #1066 faster device credentials discovery and snmp_try_slow_connect setting
* #1067 netdisco-do --force to enqueue more than 512 jobs
* deleting a device is now always a job, never done in the web engine
* update netdisco-do documentation
[BUG FIXES]
* #1058 error when the log string is empty
2.063001 - 2023-06-28
* #539 fix an issue with VLAN reindixing for VRFs (earendilfr)
* fix device port change check
2.041000 - 2019-03-12
[NEW FEATURES]
* netdisco-sshcollector no longer required - the functionality (arpnip via
ssh) will be run within netdisco's core schedule (rc9000 and ollyg)
* get_credentials replaces get_community and accepts any device_auth stanza
in JSON format (ollyg)
2.040007 - 2019-03-06
[BUG FIXES]
* #521-redux Search Node Date Range not working (ollyg)
2.040006 - 2019-03-04
[ENHANCEMENTS]
* Device Port filter by vlan includes both tagging and untagged (closes #37)
* Change colour and name of Port Control toggle option in Device Port View
* Incorporate latest DB schema changes from upstream
* Add some doc clarifcations
[BUG FIXES]
* No longer prompt for demo/demo credentials (closes #38) (J. Binks)
* Display IP in Vlan search if device has no DNS (closes #36) (J. Binks)
2.004000 - 2013-01-14
[BUG FIXES]
* Very naughty typo in netmap code, which I forgot to merge a fix for.
* Typo in installation docs (S. Gupta)
2.002000 - 2013-01-06
xt/81-token-entropy.t
xt/82-trusted-client-address.t
xt/83-csv-formula-neutralization.t
xt/84-datatables-raw-columns.t
xt/85-postfix-conditional-escaping.t
xt/86-snapshot-regenerator.t
xt/87-snmp-munger-allowlist.t
xt/88-job-action-refusal.t
xt/89-login-return-url.t
xt/90-pseudodevice-ports-and-hash-cost.t
xt/91-external-credentials-host.t
xt/92-password-rehash-on-login.t
xt/93-device-scope-acl.t
xt/94-metrics-allow-empty.t
xt/95-csv-routes-render-csv-templates.t
xt/96-worker-action-is-literal.t
xt/97-api-auth-failure-status.t
xt/98-api-delete-count.t
xt/99-api-json-errors.t
xt/bin/cpan-audit-coverage
xt/bin/cpan-audit-fix-hints
lib/App/Netdisco/Util/DeviceAuth.pm view on Meta::CPAN
use Dancer qw/:syntax :script/;
use App::Netdisco::Util::DNS 'hostname_from_ip';
use Storable 'dclone';
use Try::Tiny;
use base 'Exporter';
our @EXPORT = ();
our @EXPORT_OK = qw/
fixup_device_auth get_external_credentials
/;
our %EXPORT_TAGS = (all => \@EXPORT_OK);
=head1 NAME
App::Netdisco::Util::DeviceAuth
=head1 DESCRIPTION
Helper functions for device authentication.
lib/App/Netdisco/Util/DeviceAuth.pm view on Meta::CPAN
foreach my $stanza (@new_stanzas) {
$stanza->{driver} ||= 'snmp'
if exists $stanza->{community}
or exists $stanza->{user};
}
return @new_stanzas;
}
=head2 get_external_credentials( $device, $mode )
Runs a command to gather SNMP credentials or a C<device_auth> stanza.
Mode can be C<read> or C<write> and defaults to 'read'.
=cut
sub get_external_credentials {
my ($device, $mode) = @_;
my $cmd = (setting('get_credentials') || setting('get_community'));
my $ip = $device->ip;
my $host = ($device->dns || hostname_from_ip($ip) || $ip);
$mode ||= 'read';
# the command below reaches a shell, and this name comes from DNS, so fall
# back to the address unless it is shaped like a hostname
$host = $ip unless $host =~ m/^[A-Za-z0-9](?:[A-Za-z0-9.-]*[A-Za-z0-9])?$/;
if (defined $cmd and length $cmd) {
# replace variables
lib/App/Netdisco/Util/DeviceAuth.pm view on Meta::CPAN
$cmd =~ s/\%HOST\%/$host/egi;
$cmd =~ s/\%IP\%/$ip/egi;
my $result = (-r $cmd && ! -x $cmd) ? do { local (@ARGV, $/) = $cmd; <> } : `$cmd`; # BACKTICKS
return () unless defined $result and length $result;
my @lines = split (m/\n/, $result);
foreach my $line (@lines) {
if ($line =~ m/^community\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'read') {
debug sprintf '[%s] external read credentials added',
$device->ip;
return map {{
read => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
elsif ($line =~ m/^setCommunity\s*=\s*(.*)\s*$/i) {
if (length $1 and $mode eq 'write') {
debug sprintf '[%s] external write credentials added',
$device->ip;
return map {{
write => 1,
only => [$device->ip],
community => $_,
}} split(m/\s*,\s*/,$1);
}
}
else {
my $stanza = undef;
try {
$stanza = from_json( $line );
debug sprintf '[%s] external credentials stanza added',
$device->ip;
}
catch {
info sprintf '[%s] error! failed to parse external credentials stanza',
$device->ip;
};
return $stanza if ref $stanza;
}
}
}
return ();
}
lib/App/Netdisco/Util/SNMP.pm view on Meta::CPAN
package App::Netdisco::Util::SNMP;
use Dancer qw/:syntax :script/;
use App::Netdisco::Util::DeviceAuth 'get_external_credentials';
use Path::Class 'dir';
use File::Spec::Functions qw/splitdir catdir catfile/;
use MIME::Base64 'decode_base64';
use SNMP::Info;
use JSON::PP ();
use Scalar::Util 'refaddr';
use base 'Exporter';
our @EXPORT = ();
lib/App/Netdisco/Util/SNMP.pm view on Meta::CPAN
sub get_communities {
my ($device, $mode) = @_;
$mode ||= 'read';
my $seen_tags = {}; # for cleaning community table
my $config = (setting('device_auth') || []);
my @communities = ();
# first of all, use external command if configured
push @communities, get_external_credentials($device, $mode);
# last known-good by tag
my $tag_name = 'snmp_auth_tag_'. $mode;
my $stored_tag = eval { $device->community->$tag_name };
# A hint for a configured tag other than the stored one goes ahead of the
# last known-good credentials, so that a device answering to both can be
# moved to the hinted ones, as device_auth_tag_hint is documented to do.
# The known-good stanza stays in the list behind it, as does every other.
my $hint = setting('device_auth_tag_hint');
my $hint_wins = ($hint and $hint ne ($stored_tag // '')
and scalar grep { $_->{tag} and $_->{tag} eq $hint } @$config);
if ($device->in_storage and $stored_tag) {
foreach my $stanza (@$config) {
if ($stanza->{tag} and $stored_tag eq $stanza->{tag}) {
push @communities, {%$stanza, only => [$device->ip]}
lib/App/Netdisco/Util/SNMPText.pm view on Meta::CPAN
my $bytes = try { Encode::encode('iso-8859-1', $string, $CHECK) };
return undef unless defined $bytes;
return scalar decode_text_with($bytes, $charsets || []);
}
=head2 scan_config_encoding( $config )
Walks hashes (keys and values) and arrays in C<$config> and returns a list of
C<< { path => 'discover_no[2]', value => ..., suggestion => ... } >> for every
string which C<looks_garbled>. Other references are skipped. Reads only the
data it is given. Under the settings which hold device credentials
(C<device_auth>, C<snmp_auth>, C<community>, C<community_rw>) the value and
suggestion are C<(hidden)>.
=cut
my $CREDENTIAL_SETTINGS = qr/^(?:device_auth|snmp_auth|community|community_rw)\b/;
sub scan_config_encoding {
my ($config) = @_;
my @findings;
lib/App/Netdisco/Web/AuthN.pm view on Meta::CPAN
or return _api_unauthorized('invalid or expired token');
session(logged_in_user => $user->username);
session(logged_in_user_realm => 'users');
}
elsif ($delegated) {
session(logged_in_user => $delegated->username);
session(logged_in_user_realm => 'users');
}
else {
# An API request with no credentials is answered here. The handler for
# '/' tells the API apart by the Accept header alone, so a client that
# sent none got the login page as HTML, with a 200 and a JSON
# Content-Type, and took that for success.
return _api_unauthorized('not authorized') if request_is_api;
# user has no AuthN - force to handler for '/'
request->path_info('/');
}
};
share/config.yml view on Meta::CPAN
use_legacy_sshcollector: false
custom_fields:
device: []
device_port: []
tags:
device: {}
device_port: {}
hide_tags:
device: []
device_port: []
get_credentials: ""
bulkwalk_off: false
bulkwalk_no: []
bulkwalk_repeaters: 20
nonincreasing: false
snmpver: 3
snmptimeout: 3000000
snmp_fast_connect_timeout: 200000
snmpretries: 2
net_snmp_options: {}
snmp_try_slow_connect: true
share/environments/deployment.yml view on Meta::CPAN
name: 'netdisco'
user: 'changeme'
pass: 'changeme'
#host: 'localhost'
# --------------------
# RECOMMENDED SETTINGS
# --------------------
# Device authentication settings
# define snmp communities and ssh credentials here
# ````````````````````````````````````````````````
device_auth:
- tag: 'default_v2_readonly'
community: 'public'
read: true
write: false
# - tag: 'default_v2_for_write'
# community: 'private'
# read: false
# write: true
share/public/javascripts/htmx.min.js view on Meta::CPAN
var htmx=(()=>{const e={parse(t){if(!t)return{};if(t.startsWith("{"))return JSON.parse(t);let r=/(?:"([^"]+)"|'([^']+)'|([^\s,:]+))(?:\s*:\s*(?:"([^"]*)"|'([^']*)'|<((?:[^/]|\/(?!>))+)\/>|([^\s,]+)))?(?=\s|,|$)/g,i={};for(let s of t.matchAll(r)){let[...
share/public/javascripts/netdisco-request.js view on Meta::CPAN
/**
* Posts form-encoded fields to one of netdisco's ajax routes.
* @param {string} url the route, already carrying uri_base
* @param {URLSearchParams} body the fields to send
* @returns {Promise<Response>} resolves for every status, including 4xx and 5xx
*/
function post(url, body) {
return fetch(url, {
method: 'POST',
credentials: 'same-origin',
headers: {
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'
},
body: body
});
}
/**
* Normalizes a submitted value's line endings to CRLF, which is what a form
share/public/javascripts/netdisco-request.js view on Meta::CPAN
return fields(form, 'input, select, textarea').toString();
}
/**
* Fetches from one of netdisco's ajax routes.
* @param {string} url the route, already carrying uri_base
* @returns {Promise<Response>} resolves for every status, including 4xx and 5xx
*/
function get(url) {
return fetch(url, {
credentials: 'same-origin',
headers: { 'X-Requested-With': 'XMLHttpRequest' }
});
}
/**
* Fetches and parses JSON: a failed response is an error rather than a body
* to parse.
* @param {string} url the route, already carrying uri_base
* @returns {Promise<any>} the parsed body
*/
share/public/swagger-ui/swagger-ui-bundle.js view on Meta::CPAN
/*! For license information please see swagger-ui-bundle.js.LICENSE.txt */
!function webpackUniversalModuleDefinition(s,o){"object"==typeof exports&&"object"==typeof module?module.exports=o():"function"==typeof define&&define.amd?define([],o):"object"==typeof exports?exports.SwaggerUIBundle=o():s.SwaggerUIBundle=o()}(this,(...
share/public/swagger-ui/swagger-ui-bundle.js.map view on Meta::CPAN
{"version":3,"file":"swagger-ui-bundle.js","mappings":";CAAA,SAAUA,iCAAiCC,EAAMC,GAC1B,iBAAZC,SAA0C,iBAAXC,OACxCA,OAAOD,QAAUD,IACQ,mBAAXG,QAAyBA,OAAOC,IAC9CD,OAAO,GAAIH,GACe,iBAAZC,QACdA,QAAyB,gBAAID,IAE7BD,EAAsB,gBAAIC,GAC3B,CATD,CASGK,MAAM,cCRLC,EA...
share/python/netdisco/netdisco/util/ssh.py view on Meta::CPAN
"""
netdisco.util.ssh
~~~~~~~~~~~~~~~~~
This module provides a netmiko connection handler using
the credentials in device_auth.
"""
import os
from netmiko import ConnectHandler
from netdisco.util.config import setting
from netdisco.util.job import job
if 'ND2_FSM_TEMPLATES' in os.environ:
xt/120-hint-over-stored-tag.t view on Meta::CPAN
#!/usr/bin/env perl
use strict;
use warnings;
# device_auth_tag_hint is documented to "override, refresh, or reset cached
# authentication for a device", and #1549 introduced it for exactly that: move
# a device answering to two sets of credentials onto the new ones without
# deleting it. Since #1592 the hint only promotes its stanza within the
# configured list, and get_communities puts the last known-good tag and
# community in front of that list, so a device that still answers to the old
# credentials kept them and the hint was never tried.
#
# Asserted on the order get_communities returns, which is the order Transport
# tries. A wrong hint must still fall back to everything else.
use Test::More 0.88;
use App::Netdisco;
use App::Netdisco::Util::SNMP 'get_communities';
use Dancer qw/:script !pass/;
package FakeCommunity;
xt/120-hint-over-stored-tag.t view on Meta::CPAN
package FakeDevice;
sub new { my ($class, %v) = @_; return bless {%v}, $class }
sub ip { return '192.0.2.1' }
sub dns { return 'switch1.example.com' }
sub in_storage { return 1 }
sub snmp_comm { return $_[0]->{snmp_comm} }
sub community { return $_[0]->{community} }
package main;
config->{'get_credentials'} = undef;
config->{'get_community'} = undef;
config->{'device_auth'} = [
{ tag => 'old', community => 'oldcomm', read => 1 },
{ tag => 'new', community => 'newcomm', read => 1 },
{ tag => 'other', community => 'othercomm', read => 1 },
];
# Worker::Runner promotes the hinted stanza before get_communities sees the
# list, so do the same here to get the order a real job tries
sub order {
xt/91-external-credentials-host.t view on Meta::CPAN
#!/usr/bin/env perl
# get_credentials is a command template run through a shell, and %HOST% is
# filled from the device's DNS name, which the device's owner does not control.
# A name that is not shaped like a hostname is replaced by the address.
#
# The command below writes a file. Its absence after a run is what proves the
# injected text never reached the shell.
use strict;
use warnings;
BEGIN { $ENV{DANCER_ENVIRONMENT} = 'testing' }
use Test::More 0.88;
use File::Temp ();
use File::Spec::Functions 'catfile';
use App::Netdisco;
use Dancer qw/setting/;
use App::Netdisco::Util::DeviceAuth 'get_external_credentials';
my $dir = File::Temp->newdir();
{ package Test::FakeDevice;
sub new { my ($c, %a) = @_; return bless {%a}, $c }
sub ip { $_[0]->{ip} }
sub dns { $_[0]->{dns} }
}
subtest 'getExternalCredentials__an_ordinary_name__is_used' => sub {
setting('get_credentials' => 'echo community=%HOST%');
my @got = get_external_credentials(
Test::FakeDevice->new(ip => '192.0.2.10', dns => 'switch.example.com'), 'read');
is scalar @got, 1, 'one credential returned';
is $got[0]->{community}, 'switch.example.com', 'the name reached the command';
};
subtest 'getExternalCredentials__a_name_carrying_shell_syntax__never_reaches_the_shell' => sub {
my $marker = catfile($dir, 'fired');
setting('get_credentials' => 'echo community=%HOST%');
my @got = get_external_credentials(
Test::FakeDevice->new(ip => '192.0.2.10',
dns => "x.example.com'; touch '$marker"), 'read');
ok !-e $marker, 'the injected command did not run';
is $got[0]->{community}, '192.0.2.10', 'the address was used in place of the name';
};
subtest 'getExternalCredentials__a_name_with_a_metacharacter__falls_back_to_the_address' => sub {
setting('get_credentials' => 'echo community=%HOST%');
foreach my $dns ('a b', 'a;b', 'a`b`', 'a$(b)', 'a|b', '-leading', 'trailing-') {
my @got = get_external_credentials(
Test::FakeDevice->new(ip => '192.0.2.10', dns => $dns), 'read');
is $got[0]->{community}, '192.0.2.10', "[$dns] fell back to the address";
}
};
done_testing;
xt/97-api-auth-failure-status.t view on Meta::CPAN
# nothing. Two separate clients hit this against the queue endpoints before
# anyone noticed the token was simply not an admin's.
#
# Only the DB edges are faked: the token lookup, the user row, and the roles
# the provider would read from the user_role view. The before hook, the
# require_role wrapper and the permission_denied hook are the real ones.
#
# The browser case is asserted too, because the hook sits in front of the
# redirect for every request and must only answer for the API.
#
# The same file covers a request that sends no credentials at all. The before
# hook sends those to the handler for '/', which recognises the API by the
# Accept header alone, so a client without one got the login page as HTML with
# a 200 and a JSON Content-Type, which it took for success.
use Test::More 0.88;
use App::Netdisco;
use App::Netdisco::Web;
# loaded now so that its subs exist to be replaced; the auth plugin would
# otherwise require it on the first request and put the real ones back
use App::Netdisco::Web::Auth::Provider::DBIC;
xt/97-api-auth-failure-status.t view on Meta::CPAN
like $api->content, qr{"error"\s*:\s*"insufficient role},
'with a JSON body that says why';
# a bad token is a different fault and keeps its own answer
my $bad = fresh_request(GET => '/api/v1/queue/status', { headers => [
'Authorization' => 'badtoken', 'Accept' => 'application/json' ] });
is $bad->status, 401, 'an invalid token is still unauthorized, not forbidden';
like $bad->content, qr/"error"\s*:\s*"invalid or expired token"/,
'and says why, where it used to send an empty body';
# no credentials at all, with and without an Accept header
foreach my $accept ([], ['Accept' => 'application/json']) {
my $what = (@$accept ? 'with' : 'without') . ' an Accept header';
my $none = fresh_request(GET => '/api/v1/queue/status', { headers => $accept });
is $none->status, 401, "no credentials $what is unauthorized";
my $body = $none->content; $body = do { local $/; <$body> } if ref $body;
like $body, qr/\A\s*\{.*"error"\s*:\s*"not authorized"/s, "and says so in JSON $what";
}
# a browser that lacks the role still goes to the denied page as before
# (an existing admin-only route; one defined here would lose to Web.pm's
# catch-all, which is registered first)
set trust_x_remote_user => 1;
my $web = fresh_request(GET => '/ajax/content/admin/snapshot_get', { headers => [
xt/js/request-helper.test.js view on Meta::CPAN
}
test('post__sending_a_body__carries_the_XMLHttpRequest_header', async () => {
let seen = null;
const ndRequest = load((url, init) => { seen = { url, init }; return Promise.resolve({ ok: true, status: 200 }); });
await ndRequest.post('/ajax/portcontrol', new URLSearchParams({ device: '192.0.2.1' }));
assert.equal(seen.url, '/ajax/portcontrol');
assert.equal(seen.init.method, 'POST');
assert.equal(seen.init.headers['X-Requested-With'], 'XMLHttpRequest');
assert.match(seen.init.headers['Content-Type'], /application\/x-www-form-urlencoded/);
assert.equal(seen.init.credentials, 'same-origin');
assert.equal(String(seen.init.body), 'device=192.0.2.1');
});
test('post__a_server_error__resolves_so_the_caller_can_read_the_status', async () => {
const ndRequest = load(() => Promise.resolve({ ok: false, status: 500 }));
const res = await ndRequest.post('/ajax/portcontrol', new URLSearchParams());
assert.equal(res.status, 500, 'fetch does not reject on 5xx and the helper must not hide that');
});
test('fields__over_a_row_of_inputs__skips_an_unchecked_box_and_a_nameless_field', () => {
( run in 1.389 second using v1.01-cache-2.11-cpan-3f962061493 )