1 collectd - System information collection daemon
2 =================================================
8 collectd is a small daemon which collects system information periodically
9 and provides mechanisms to store and monitor the values in a variety of
16 * collectd is able to collect the following data:
19 Apache server utilization: Number of bytes transfered, number of
20 requests handled and detailed scoreboard statistics
23 APC UPS Daemon: UPS charge, load, input/output/battery voltage, etc.
26 Sensors in Macs running Mac OS X / Darwin: Temperature, fanspeed and
30 Various sensors in the Aquaero 5 watercooling board made by Aquacomputer.
33 Statistics about Ascent, a free server for the game `World of Warcraft'.
36 Reads absolute barometric pressure, air pressure reduced to sea level and temperature.
37 Supported sensors are MPL115A2 and MPL3115 from Freescale and BMP085 from Bosch.
40 Batterycharge, -current and voltage of ACPI and PMU based laptop
44 Name server and resolver statistics from the `statistics-channel'
45 interface of BIND 9.5, 9,6 and later.
48 CPU accounting information for process groups under Linux.
51 Number of nf_conntrack entries.
54 Number of context switches done by the operating system.
57 CPU utilization: Time spent in the system, user, nice, idle, and related
61 CPU frequency (For laptops with speed step or a similar technology)
64 Parse statistics from websites using regular expressions.
67 Retrieves JSON data via cURL and parses it according to user
71 Retrieves XML data via cURL and parses it according to user
75 Executes SQL statements on various databases and interprets the returned
79 Mountpoint usage (Basically the values `df(1)' delivers)
82 Disk utilization: Sectors read/written, number of read/write actions,
83 average time an IO-operation took to complete.
86 DNS traffic: Query types, response codes, opcodes and traffic/octets
90 Email statistics: Count, traffic, spam scores and checks.
91 See collectd-email(5).
94 Amount of entropy available to the system.
97 Network interface card statistics.
100 Values gathered by a custom program or script.
101 See collectd-exec(5).
104 Count the number of files in directories.
107 Linux file-system based caching framework statistics.
110 Receive multicast traffic from Ganglia instances.
113 Harddisk temperatures using hddtempd.
116 Interface traffic: Number of octets, packets and errors for each
120 Iptables' counters: Number of bytes that were matched by a certain
124 IPMI (Intelligent Platform Management Interface) sensors information.
127 IPVS connection statistics (number of connections, octets and packets
128 for each service and destination).
129 See http://www.linuxvirtualserver.org/software/index.html.
132 IRQ counters: Frequency in which certain interrupts occur.
135 Integrates a `Java Virtual Machine' (JVM) to execute plugins in Java
136 bytecode. See “Configuring with libjvm” below.
139 System load average over the last 1, 5 and 15 minutes.
142 Detailed CPU statistics of the “Logical Partitions” virtualization
143 technique built into IBM's POWER processors.
146 CPU, memory, disk and network I/O statistics from virtual machines.
149 Size of “Logical Volumes” (LV) and “Volume Groups” (VG) of Linux'
150 “Logical Volume Manager” (LVM).
153 Queries very detailed usage statistics from wireless LAN adapters and
154 interfaces that use the Atheros chipset and the MadWifi driver.
157 Motherboard sensors: temperature, fanspeed and voltage information,
161 Linux software-RAID device information (number of active, failed, spare
165 Query and parse data from a memcache daemon (memcached).
168 Statistics of the memcached distributed caching system.
169 <http://www.danga.com/memcached/>
172 Memory utilization: Memory occupied by running processes, page cache,
173 buffer cache and free.
176 Collects CPU usage, memory usage, temperatures and power consumption from
177 Intel Many Integrated Core (MIC) CPUs.
180 Reads values from Modbus/TCP enabled devices. Supports reading values
181 from multiple "slaves" so gateway devices can be used.
184 Information provided by serial multimeters, such as the `Metex
188 MySQL server statistics: Commands issued, handlers triggered, thread
189 usage, query cache utilization and traffic/octets sent and received.
192 Plugin to query performance values from a NetApp storage system using the
193 “Manage ONTAP” SDK provided by NetApp.
196 Very detailed Linux network interface and routing statistics. You can get
197 (detailed) information on interfaces, qdiscs, classes, and, if you can
198 make use of it, filters.
201 Receive values that were collected by other hosts. Large setups will
202 want to collect the data on one dedicated machine, and this is the
203 plugin of choice for that.
206 NFS Procedures: Which NFS command were called how often. Only NFSv2 and
210 Collects statistics from `nginx' (speak: engine X), a HTTP and mail
214 NTP daemon statistics: Local clock drift, offset to peers, etc.
217 Network UPS tools: UPS current, voltage, power, charge, utilisation,
218 temperature, etc. See upsd(8).
221 Information about Non-Uniform Memory Access (NUMA).
224 Queries routing information from the “Optimized Link State Routing”
227 - onewire (EXPERIMENTAL!)
228 Read onewire sensors using the owcapu library of the owfs project.
229 Please read in collectd.conf(5) why this plugin is experimental.
232 RX and TX of each client in openvpn-status.log (status-version 2).
233 <http://openvpn.net/index.php/documentation/howto.html>
236 Query data from an Oracle database.
239 The perl plugin implements a Perl-interpreter into collectd. You can
240 write your own plugins in Perl and return arbitrary values using this
241 API. See collectd-perl(5).
244 Query statistics from BSD's packet filter "pf".
247 Receive and dispatch timing values from Pinba, a profiling extension for
251 Network latency: Time to reach the default gateway or another given
255 PostgreSQL database statistics: active server connections, transaction
256 numbers, block IO, table row manipulations.
259 PowerDNS name server statistics.
262 Process counts: Number of running, sleeping, zombie, ... processes.
265 Counts various aspects of network protocols such as IP, TCP, UDP, etc.
268 The python plugin implements a Python interpreter into collectd. This
269 makes it possible to write plugins in Python which are executed by
270 collectd without the need to start a heavy interpreter every interval.
271 See collectd-python(5) for details.
274 The redis plugin gathers information from a redis server, including:
275 uptime, used memory, total connections etc.
278 Query interface and wireless registration statistics from RouterOS.
281 RRDtool caching daemon (RRDcacheD) statistics.
284 System sensors, accessed using lm_sensors: Voltages, temperatures and
288 RX and TX of serial interfaces. Linux only; needs root privileges.
291 Uses libsigrok as a backend, allowing any sigrok-supported device
292 to have its measurements fed to collectd. This includes multimeters,
293 sound level meters, thermometers, and much more.
296 Read values from SNMP (Simple Network Management Protocol) enabled
297 network devices such as switches, routers, thermometers, rack monitoring
298 servers, etc. See collectd-snmp(5).
301 Acts as a StatsD server, reading values sent over the network from StatsD
302 clients and calculating rates and other aggregates out of these values.
305 Pages swapped out onto harddisk or whatever is called `swap' by the OS..
308 Parse table-like structured files.
311 Follows (tails) logfiles, parses them by lines and submits matched
315 Follows (tails) files in CSV format, parses each line and submits
319 Bytes and operations read and written on tape devices. Solaris only.
322 Number of TCP connections to specific local and remote ports.
325 TeamSpeak2 server statistics.
328 Plugin to read values from `The Energy Detective' (TED).
331 Linux ACPI thermal zone information.
334 Reads the number of records and file size from a running Tokyo Tyrant
338 System uptime statistics.
341 Users currently logged in.
344 Various statistics from Varnish, an HTTP accelerator.
347 Virtual memory statistics, e. g. the number of page-ins/-outs or the
348 number of pagefaults.
351 System resources used by Linux VServers.
352 See <http://linux-vserver.org/>.
355 Link quality of wireless cards. Linux only.
358 Bitrate and frequency of music played with XMMS.
361 Statistics for ZFS' “Adaptive Replacement Cache” (ARC).
363 * Output can be written or sent to various destinations by the following
367 Sends JSON-encoded data to an Advanced Message Queuing Protocol (AMQP)
368 server, such as RabbitMQ.
371 Write to comma separated values (CSV) files. This needs lots of
372 diskspace but is extremely portable and can be analysed with almost
373 every program that can analyse anything. Even Microsoft's Excel..
376 Send the data to a remote host to save the data somehow. This is useful
377 for large setups where the data should be saved by a dedicated machine.
380 Of course the values are propagated to plugins written in Perl, too, so
381 you can easily do weird stuff with the plugins we didn't dare think of
382 ;) See collectd-perl(5).
385 It's possible to implement write plugins in Python using the python
386 plugin. See collectd-python(5) for details.
389 Output to round-robin-database (RRD) files using the RRDtool caching
390 daemon (RRDcacheD) - see rrdcached(1). That daemon provides a general
391 implementation of the caching done by the `rrdtool' plugin.
394 Output to round-robin-database (RRD) files using librrd. See rrdtool(1).
395 This is likely the most popular destination for such values. Since
396 updates to RRD-files are somewhat expensive this plugin can cache
397 updates to the files and write a bunch of updates at once, which lessens
401 One can query the values from the unixsock plugin whenever they're
402 needed. Please read collectd-unixsock(5) for a description on how that's
406 Sends data to Carbon, the storage layer of Graphite using TCP or UDP. It
407 can be configured to avoid logging send errors (especially useful when
411 Sends the values collected by collectd to a web-server using HTTP POST
412 requests. The transmitted data is either in a form understood by the
413 Exec plugin or formatted in JSON.
416 Sends data to Apache Kafka, a distributed queue.
419 Sends data to MongoDB, a NoSQL database.
422 Sends the values to a Redis key-value database server.
425 Sends data to Riemann, a stream processing and monitoring system.
427 * Logging is, as everything in collectd, provided by plugins. The following
428 plugins keep up informed about what's going on:
431 Writes log messages to a file or STDOUT/STDERR.
434 Log messages are propagated to plugins written in Perl as well.
435 See collectd-perl(5).
438 It's possible to implement log plugins in Python using the python plugin.
439 See collectd-python(5) for details.
442 Logs to the standard UNIX logging mechanism, syslog.
445 Writes log messages formatted as logstash JSON events.
447 * Notifications can be handled by the following plugins:
450 Send a desktop notification to a notification daemon, as defined in
451 the Desktop Notification Specification. To actually display the
452 notifications, notification-daemon is required.
453 See http://www.galago-project.org/specs/notification/.
456 Send an E-mail with the notification message to the configured
460 Execute a program or script to handle the notification.
461 See collectd-exec(5).
464 Writes the notification message to a file or STDOUT/STDERR.
467 Send the notification to a remote host to handle it somehow.
470 Notifications are propagated to plugins written in Perl as well.
471 See collectd-perl(5).
474 It's possible to implement notification plugins in Python using the
475 python plugin. See collectd-python(5) for details.
477 * Value processing can be controlled using the "filter chain" infrastructure
478 and "matches" and "targets". The following plugins are available:
480 - match_empty_counter
481 Match counter values which are currently zero.
484 Match values using a hash function of the hostname.
487 Match values by their identifier based on regular expressions.
490 Match values with an invalid timestamp.
493 Select values by their data sources' values.
495 - target_notification
496 Create and dispatch a notification.
499 Replace parts of an identifier using regular expressions.
502 Scale (multiply) values by an arbitrary value.
505 Set (overwrite) entire parts of an identifier.
507 * Miscellaneous plugins:
510 Selects multiple value lists based on patterns or regular expressions
511 and creates new aggregated values lists from those.
514 Checks values against configured thresholds and creates notifications if
515 values are out of bounds. See collectd-threshold(5) for details.
518 Sets the hostname to an unique identifier. This is meant for setups
519 where each client may migrate to another physical host, possibly going
520 through one or more name changes in the process.
522 * Performance: Since collectd is running as a daemon it doesn't spend much
523 time starting up again and again. With the exception of the exec plugin no
524 processes are forked. Caching in output plugins, such as the rrdtool and
525 network plugins, makes sure your resources are used efficiently. Also,
526 since collectd is programmed multithreaded it benefits from hyperthreading
527 and multicore processors and makes sure that the daemon isn't idle if only
528 one plugin waits for an IO-operation to complete.
530 * Once set up, hardly any maintenance is necessary. Setup is kept as easy
531 as possible and the default values should be okay for most users.
537 * collectd's configuration file can be found at `sysconfdir'/collectd.conf.
538 Run `collectd -h' for a list of builtin defaults. See `collectd.conf(5)'
539 for a list of options and a syntax description.
541 * When the `csv' or `rrdtool' plugins are loaded they'll write the values to
542 files. The usual place for these files is beneath `/var/lib/collectd'.
544 * When using some of the plugins, collectd needs to run as user root, since
545 only root can do certain things, such as craft ICMP packages needed to ping
546 other hosts. collectd should NOT be installed setuid root since it can be
547 used to overwrite valuable files!
549 * Sample scripts to generate graphs reside in `contrib/' in the source
550 package or somewhere near `/usr/share/doc/collectd' in most distributions.
551 Please be aware that those script are meant as a starting point for your
552 own experiments.. Some of them require the `RRDs' Perl module.
553 (`librrds-perl' on Debian) If you have written a more sophisticated
554 solution please share it with us.
556 * The RRAs of the automatically created RRD files depend on the `step'
557 and `heartbeat' settings given. If change these settings you may need to
558 re-create the files, losing all data. Please be aware of that when changing
559 the values and read the rrdtool(1) manpage thoroughly.
562 collectd and chkrootkit
563 -----------------------
565 If you are using the `dns' plugin chkrootkit(1) will report collectd as a
566 packet sniffer ("<iface>: PACKET SNIFFER(/usr/sbin/collectd[<pid>])"). The
567 plugin captures all UDP packets on port 53 to analyze the DNS traffic. In
568 this case, collectd is a legitimate sniffer and the report should be
569 considered to be a false positive. However, you might want to check that
570 this really is collectd and not some other, illegitimate sniffer.
576 To compile collectd from source you will need:
578 * Usual suspects: C compiler, linker, preprocessor, make, ...
580 * A POSIX-threads (pthread) implementation.
581 Since gathering some statistics is slow (network connections, slow devices,
582 etc) collectd is parallelized. The POSIX threads interface is being
583 used and should be found in various implementations for hopefully all
586 * aerotools-ng (optional)
587 Used by the `aquaero' plugin. Currently, the `libaquaero5' library, which
588 is used by the `aerotools-ng' toolkit, is not compiled as a shared object
589 nor does it feature an installation routine. Therefore, you need to point
590 collectd's configure script at the source directory of the `aerotools-ng'
592 <https://github.com/lynix/aerotools-ng>
594 * CoreFoundation.framework and IOKit.framework (optional)
595 For compiling on Darwin in general and the `apple_sensors' plugin in
597 <http://developer.apple.com/corefoundation/>
599 * libclntsh (optional)
600 Used by the `oracle' plugin.
602 * libcredis (optional)
603 Used by the `redis' plugin. Please note that you require a 0.2.2 version
604 or higher. <http://code.google.com/p/credis/>
607 If you want to use the `apache', `ascent', `bind', `curl', `curl_json',
608 `curl_xml', `nginx', or `write_http' plugin.
609 <http://curl.haxx.se/>
612 Used by the `dbi' plugin to connect to various databases.
613 <http://libdbi.sourceforge.net/>
615 * libesmtp (optional)
616 For the `notify_email' plugin.
617 <http://www.stafford.uklinux.net/libesmtp/>
619 * libganglia (optional)
620 Used by the `gmond' plugin to process data received from Ganglia.
621 <http://ganglia.info/>
623 * libgcrypt (optional)
624 Used by the `network' plugin for encryption and authentication.
625 <http://www.gnupg.org/>
628 If present, the `uuid' plugin will check for UUID from HAL.
629 <http://hal.freedesktop.org/>
631 * libi2c-dev (optional)
632 Used for the plugin `barometer', provides just the i2c-dev.h header file
633 for user space i2c development.
636 For querying iptables counters.
637 <http://netfilter.org/>
640 Library that encapsulates the `Java Virtual Machine' (JVM). This library is
641 used by the `java' plugin to execute Java bytecode. See “Configuring with
643 <http://openjdk.java.net/> (and others)
646 Used by the `lvm' plugin.
647 <ftp://sources.redhat.com/pub/lvm2/>
649 * libmemcached (optional)
650 Used by the `memcachec' plugin to connect to a memcache daemon.
651 <http://tangent.org/552/libmemcached.html>
654 Used by the `netlink' plugin.
655 <http://www.netfilter.org/projects/libmnl/>
657 * libmodbus (optional)
658 Used by the `modbus' plugin to communicate with Modbus/TCP devices. The
659 `modbus' plugin works with version 2.0.3 of the library – due to frequent
660 API changes other versions may or may not compile cleanly.
661 <http://www.libmodbus.org/>
663 * libmysqlclient (optional)
664 Unsurprisingly used by the `mysql' plugin.
665 <http://dev.mysql.com/>
667 * libnetapp (optional)
668 Required for the `netapp' plugin.
669 This library is part of the “Manage ONTAP SDK” published by NetApp.
671 * libnetsnmp (optional)
672 For the `snmp' plugin.
673 <http://www.net-snmp.org/>
675 * libnotify (optional)
676 For the `notify_desktop' plugin.
677 <http://www.galago-project.org/>
679 * libopenipmi (optional)
680 Used by the `ipmi' plugin to prove IPMI devices.
681 <http://openipmi.sourceforge.net/>
683 * liboping (optional)
684 Used by the `ping' plugin to send and receive ICMP packets.
685 <http://octo.it/liboping/>
687 * libowcapi (optional)
688 Used by the `onewire' plugin to read values from onewire sensors (or the
690 <http://www.owfs.org/>
693 Used to capture packets by the `dns' plugin.
694 <http://www.tcpdump.org/>
696 * libperfstat (optional)
697 Used by various plugins to gather statistics under AIX.
700 Obviously used by the `perl' plugin. The library has to be compiled with
701 ithread support (introduced in Perl 5.6.0).
702 <http://www.perl.org/>
705 The PostgreSQL C client library used by the `postgresql' plugin.
706 <http://www.postgresql.org/>
708 * libprotobuf-c, protoc-c (optional)
709 Used by the `pinba' plugin to generate a parser for the network packets
710 sent by the Pinba PHP extension, and by the `write_riemann' plugin to
711 generate events to be sent to a Riemann server.
712 <http://code.google.com/p/protobuf-c/>
714 * libpython (optional)
715 Used by the `python' plugin. Currently, Python 2.3 and later and Python 3
717 <http://www.python.org/>
719 * librabbitmq (optional; also called “rabbitmq-c”)
720 Used by the `amqp' plugin for AMQP connections, for example to RabbitMQ.
721 <http://hg.rabbitmq.com/rabbitmq-c/>
723 * librdkafka (optional; also called “rdkafka”)
724 Used by the `write_kafka' plugin for producing messages and sending them
726 <https://github.com/edenhill/librdkafka>
728 * librouteros (optional)
729 Used by the `routeros' plugin to connect to a device running `RouterOS'.
730 <http://octo.it/librouteros/>
733 Used by the `rrdtool' and `rrdcached' plugins. The latter requires RRDtool
734 client support which was added after version 1.3 of RRDtool. Versions 1.0,
735 1.2 and 1.3 are known to work with the `rrdtool' plugin.
736 <http://oss.oetiker.ch/rrdtool/>
738 * librt, libsocket, libkstat, libdevinfo (optional)
739 Various standard Solaris libraries which provide system functions.
740 <http://developers.sun.com/solaris/>
742 * libsensors (optional)
743 To read from `lm_sensors', see the `sensors' plugin.
744 <http://www.lm-sensors.org/>
746 * libsigrok (optional)
747 Used by the `sigrok' plugin. In addition, libsigrok depends on glib,
748 libzip, and optionally (depending on which drivers are enabled) on
749 libusb, libftdi and libudev.
751 * libstatgrab (optional)
752 Used by various plugins to collect statistics on systems other than Linux
754 <http://www.i-scream.org/libstatgrab/>
756 * libtokyotyrant (optional)
757 Used by the `tokyotyrant' plugin.
758 <http://1978th.net/tokyotyrant/>
760 * libupsclient/nut (optional)
761 For the `nut' plugin which queries nut's `upsd'.
762 <http://networkupstools.org/>
765 Collect statistics from virtual machines.
766 <http://libvirt.org/>
769 Parse XML data. This is needed for the `ascent', `bind', `curl_xml' and
771 <http://xmlsoft.org/>
774 <http://www.xmms.org/>
777 Parse JSON data. This is needed for the `curl_json' and `log_logstash'
779 <http://github.com/lloyd/yajl>
781 * libvarnish (optional)
782 Fetches statistics from a Varnish instance. This is needed for the
784 <http://varnish-cache.org>
786 Configuring / Compiling / Installing
787 ------------------------------------
789 To configure, build and install collectd with the default settings, run
790 `./configure && make && make install'. For detailed, generic instructions
791 see INSTALL. For a complete list of configure options and their description,
792 run `./configure --help'.
794 By default, the configure script will check for all build dependencies and
795 disable all plugins whose requirements cannot be fulfilled (any other plugin
796 will be enabled). To enable a plugin, install missing dependencies (see
797 section `Prerequisites' above) and rerun `configure'. If you specify the
798 `--enable-<plugin>' configure option, the script will fail if the depen-
799 dencies for the specified plugin are not met. In that case you can force the
800 plugin to be built using the `--enable-<plugin>=force' configure option.
801 This will most likely fail though unless you're working in a very unusual
802 setup and you really know what you're doing. If you specify the
803 `--disable-<plugin>' configure option, the plugin will not be built. If you
804 specify the `--enable-all-plugins' or `--disable-all-plugins' configure
805 options, all plugins will be enabled or disabled respectively by default.
806 Explicitly enabling or disabling a plugin overwrites the default for the
807 specified plugin. These options are meant for package maintainers and should
808 not be used in everyday situations.
810 By default, collectd will be installed into `/opt/collectd'. You can adjust
811 this setting by specifying the `--prefix' configure option - see INSTALL for
812 details. If you pass DESTDIR=<path> to `make install', <path> will be
813 prefixed to all installation directories. This might be useful when creating
814 packages for collectd.
816 Configuring with libjvm
817 -----------------------
819 To determine the location of the required files of a Java installation is not
820 an easy task, because the locations vary with your kernel (Linux, SunOS, …)
821 and with your architecture (x86, SPARC, …) and there is no ‘java-config’
822 script we could use. Configuration of the JVM library is therefore a bit
825 The easiest way to use the `--with-java=$JAVA_HOME' option, where
826 `$JAVA_HOME' is usually something like:
827 /usr/lib/jvm/java-1.5.0-sun-1.5.0.14
829 The configure script will then use find(1) to look for the following files:
835 If found, appropriate CPP-flags and LD-flags are set and the following
836 library checks succeed.
838 If this doesn't work for you, you have the possibility to specify CPP-flags,
839 C-flags and LD-flags for the ‘Java’ plugin by hand, using the following three
840 (environment) variables:
846 For example (shortened for demonstration purposes):
848 ./configure JAVA_CPPFLAGS="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux"
850 Adding "-ljvm" to the JAVA_LDFLAGS is done automatically, you don't have to
856 To compile correctly collectd needs to be able to initialize static
857 variables to NAN (Not A Number). Some C libraries, especially the GNU
858 libc, have a problem with that.
860 Luckily, with GCC it's possible to work around that problem: One can define
861 NAN as being (0.0 / 0.0) and `isnan' as `f != f'. However, to test this
862 ``implementation'' the configure script needs to compile and run a short
863 test program. Obviously running a test program when doing a cross-
864 compilation is, well, challenging.
866 If you run into this problem, you can use the `--with-nan-emulation'
867 configure option to force the use of this implementation. We can't promise
868 that the compiled binary actually behaves as it should, but since NANs
869 are likely never passed to the libm you have a good chance to be lucky.
871 Likewise, collectd needs to know the layout of doubles in memory, in order
872 to craft uniform network packets over different architectures. For this, it
873 needs to know how to convert doubles into the memory layout used by x86. The
874 configure script tries to figure this out by compiling and running a few
875 small test programs. This is of course not possible when cross-compiling.
876 You can use the `--with-fp-layout' option to tell the configure script which
877 conversion method to assume. Valid arguments are:
879 * `nothing' (12345678 -> 12345678)
880 * `endianflip' (12345678 -> 87654321)
881 * `intswap' (12345678 -> 56781234)
887 For questions, bug reports, development information and basically all other
888 concerns please send an email to collectd's mailing list at
889 <list at collectd.org>.
891 For live discussion and more personal contact visit us in IRC, we're in
892 channel #collectd on freenode.
898 Florian octo Forster <octo at collectd.org>,
899 Sebastian tokkee Harl <sh at tokkee.org>,
900 and many contributors (see `AUTHORS').
902 Please send bug reports and patches to the mailing list, see `Contact'