What is ivulncheck?¶
Ivulncheck is a security tool to centralize the search of vulnerabilities and fixes concerning packages on a cluster.
A procedure is needed to initialize the API (restore an initial state of the MongoDB database, download ubuntu-cve-tracker tool, deploy CVE-Search modified API)
First, an agent script must be deployed to detect installed packages on the cluster (only Ubuntu and Debian are supported). This script generated a json formatted file and send it to ther API. Then, several commands must be executed to search vulnerabilities and determine available security updates (called here fixes). Four principals sources are used :
- debsecan for Debian hosts,
- ubuntu-cve-tracker for Ubuntu hosts,
- CVE-Search for custom installed application (manually configuration needed),
- madison-lite (to complete missing fixes due to possibly outdated sources, missing sources or unreferenced vulnerabilities)
Eventually, you can get informations (installed packages, monitored hosts, known vulnerabilities an available fixes) directly from the API Or a Web Interface provide a graphical rendering of these informations.
Installation¶
You have to set a compilation environment for debian unstable with sbuild, fakeroot
- Cloning from salsa git clone https://salsa.debian.org/gegger-guest/ivulncheck.git
- Compilation
- Commit last changes
- git commit -a
- Tag the new version to be compile
- git tag <version>
- Get in the debian branch
- git checkout debian/unstable
- Merge last modifications from tagged branch
- git merge -X theirs <version>
- Edit debian/changelog file with the new version and commit your changes in debian/unstable branch
- git commit -a -m “Changing version to <version>”
- Generate original archive
- ./debian/rules gen-orig-xz
- Compile the package
- gbp buildpackage
The .deb package goes to ../build-area/ivulncheck-xxx_<version>_all.deb
- Installation of the package dpkg -i ivulncheck-api_x.x.x-1_all.deb dpkg -i ivulncheck-web_x.x.x-1_all.deb apt-get -f update
- You have to start mongodb server to be able to use ivulncheck
- service mongodb start
- Or
- systemctl start mongodb
If you do not need mongodb authentication, you can delete username and password lines from /etc/ivulncheck/ivulncheck-api.ini
- If you want it, you can configure mongodb authentication and change the default password in /etc/ivulncheck/ivulncheck-api.ini
- mongo > use admin > db.createUser({ user: “admin”, pwd: “<password>”, roles: [{ role: “userAdminAnyDatabase”, db: “admin”}]}) > db.createUser({user: “ivulncheck”, pwd: “ivulncheck”, roles: [ { role: “readWrite”, db: “hostdb” }, { role: “readWrite”, db: “vulndb” }, { role: “readWrite”, db: “fixdb” } ] }) > exit
- You have to enable auth in /etc/mongodb.conf
- sudo sed -i ‘s/^#auth = true/auth = true/’ /etc/mongodb.conf
- You need to restart mongodb
- service mongodb restart
- Or
- systemctl restart mongodb
To log back as admin you can do $ mongo $ > use admin $ > db.auth(‘admin’, “<password>”)
- You can change ivulncheck password with
- > db.changeUserPassword(“ivulncheck”, “<new_password>”)
API needs apache2 to work and ivulncheck-api conf is added to an existing virtualhost and an alias is created. So you need apache server to be installed and running
service apache2 start
- MongoDB
You need to restore an initial database state before being able to use ivulncheck You can launch ivulncheck_mongodb script in /usr/share/ivulncheck-api directory (edit the password if needed) Or you can restore database manually
mongorestore -u ivulncheck -p <password> –authenticationDatabase=admin –gzip /usr/share/ivulncheck-api/emptydb
Be careful, some old mongodb version does not support –gzip option so you may have to deflate all files in emptydb directory.
- Apache2
- You need to create users for connectin to API and Web interface.
- sudo htpasswd /etc/ivulncheck/htpasswd_agent ivulncheck_agent sudo htpasswd /etc/ivulncheck/htpasswd_api ivulncheck_api sudo htpasswd /etc/ivulncheck/htpasswd_web ivulncheck_web
And do not forget to update configuration files in /etc/ivulncheck with new credentials.
- SSL
You can add SSL certificate to enable HTTPS Certificate’s files path are
/etc/apache2/ssl/server.pem /etc/apache2/ssl/server.key /etc/apache2/ssl/ca.pem
- Ubuntu CVE Tracker
Only if you want to manage Ubuntu hosts
ubuntu-cve-tracker tool need to be initialized.
(Not yet fully functionnal) You can launch ivulncheck_download script in /usr/share/ivulncheck-api directory
- Or you can manually set it
- bzr branch lp:~ubuntu-security/ubuntu-cve-tracker/master /var/lib/ivulncheck-api/ubuntu-cve-tracker
- Edit ubuntu-cve-tracker/scripts/packages-mirror and changed those 3 paths
- outPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/ubuntu-archive-metadata ... debianPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/debian-testing-archive-metadata ... partnerPath=/var/lib/ivulncheck-api/ubuntu-cve-tracker/partner-archive-metadata
- And launch those 2 commands
- cd /var/lib/ivulncheck-api/ubuntu-cve-tracker ./scripts/packages-mirror ./scripts/packages-mirror -p
- And do not forget to restore the right rights...
- chown -R ivulncheck:ivulncheck /var/lib/ivulncheck-api/ubuntu-cve-tracker/
- Madison-lite
To execute madison-lite tool offline (a lot faster than rmadison) you need to download part of Debian and Ubuntu repositories information An API function exists to do this
Launch this URL http://<ivulncheck-api-host>/ivulncheck_api/cmd/fake_repos
- Or this command with CLI interface
- ivulndownload -fakerepos
This will download only information about installed hosts suite and release
- CVE Search
CVE Search tool can be installed to make CVE link offline and faster.
This tool is also useful for vulnerability checks based on CPE (Common Product Enumeration) if you need to monitor products which are not Debian or Ubuntu package.
This functionality required a patched version of CVE-Search which I made and I have not yet published it or fork or made a feature request (because it was the first approach to check vuln but it is not convenient. There are a lot of vulnerabilities missing or false positives)
I will add this feature in a next release
Utilization¶
An agent is deployed on all monitored servers and works every day. It detects the OS version , lists installed packages and sends those information in JSON format to the server. (delayed on 1 hour to not overload the server)
The API full a database with those packages and checks vulnerabilities and available fixes on them. * DebSeCan approach :
- debsecan_vulns: checks vulnerabilities on Debian packages and tags fixed packages from debsecan tools
- debsecan_fixes: determines fixes from tags and on comparing installed version with fixed version
- Ubuntu approach (same as DebSecAn but with Ubuntu-CVE-Tracket tool) :
- ubuntu_vulns: checks vulnerabilities on Ubuntu packages and tags fixed packages from ubuntu-cve-tracker tools
- ubuntu_fixes: determines fixes from tags and on comparing installed version with fixed version
- *Madison Lite approach :
- madison_fixes: checks residual fixes on querying official security repositories from Debian and Ubuntu (downloaded locally for faster search)
residual fixes that are not detected by neither DebSecAn nor Ubuntu-CVE-Tracker can be minor fixes or fixes that are not concerned by a CVE
The MongoDB structure is:
1 Database hostdb with collections hosts and packages where are the list of installed packages from hosts
1 Database vulndb with collections pkg2cpe (an index of correspondance between CPE and package) and vulns where are all vulnerabilities with tags. The key index are <cve_id> and <package_name>
1 Database fixdb with collection fixes where are the fixes found from vulnerabilities
- Page d’accueil
This page shows global stats on vulnerabilities and fixes.
- Page hosts
This page shows hosts in database and links to the corresponding host page.
- Page Vulns
Cette page présente toutes les vulnérabilités avec des statistiques et des filtres de recherches. * Page Fixes
Cette page présente tous les patchs avec des statistiques et des filtres de recherches. * Page Host
Cette page présente les statistiques d’un hôte et les liens vers ces vulnérabilités et patchs.
Commands send request to the API and each command has his usage
ivulnlist: lists MongoDB info with only skip and limit options
ivulninsert: inserts manually an host with a JSON formated package list
ivulncheck: launches checks independently (3 approaches)
ivulndownload: download repositories from Debian and Ubuntu websites (only for detected installed versions)
ivulnpurge: purge MongoDB collections (but keep indexes)
API functions return JSON responses
- Search functions
http://<api_server>/ivulncheck_api/hosts
Méthodes supportées GET et POST
http://<api_server>/ivulncheck_api/pkgs
Méthodes supportées GET et POST
http://<api_server>/ivulncheck_api/vulns
Méthodes supportées GET et POST
http://<api_server>/ivulncheck_api/fixes
Méthodes supportées GET et POST
http://<api_server>/ivulncheck_api/host/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/host/pkgs/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/vulns/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/host/fixes/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/host/stats/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/pkg/<pkgvers>
Méthodes supportées GET
http://<api_server>/ivulncheck_api/vuln/<vuln_id>
Supported method GET
http://<api_server>/ivulncheck_api/os_versions
Supported method GET
http://<api_server>/ivulncheck_api/hosts/stats
Supported method GET
http://<api_server>/ivulncheck_api/vulns/stats
Supported method GET
http://<api_server>/ivulncheck_api/fixes/stats
Supported method GET
http://<api_server>/ivulncheck_api/vulns/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/host/fixes/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/host/stats/<hostname>
Supported method GET
http://<api_server>/ivulncheck_api/pkg/<pkgvers>
Méthodes supportées GET
http://<api_server>/ivulncheck_api/vuln/<vuln_id>
Supported method GET
http://<api_server>/ivulncheck_api/os_versions
Supported method GET
http://<api_server>/ivulncheck_api/hosts/stats
Supported method GET
http://<api_server>/ivulncheck_api/vulns/stats
Supported method GET
http://<api_server>/ivulncheck_api/fixes/stats
Supported method GET
- Command functions
http://<api_server>/ivulncheck_api/cmd/check/<collection>
Supported method GET
http://<api_server>/ivulncheck_api/cmd/purge/<action>
Supported method GET
Insert function
http://<api_server>/ivulncheck_api/insert/host
Supported method PUT