3 collectd-python - Documentation of collectd's C<python plugin>
12 ModulePath "/path/to/your/python/modules"
18 spam "wonderful" "lovely"
24 The C<python plugin> embeds a Python-interpreter into collectd and provides an
25 interface to collectd's plugin system. This makes it possible to write plugins
26 for collectd in Python. This is a lot more efficient than executing a
27 Python-script every time you want to read a value with the C<exec plugin> (see
28 L<collectd-exec(5)>) and provides a lot more functionality, too.
30 At least python I<version 2.3> is required.
36 =item B<LoadPlugin> I<Plugin>
38 Loads the Python plugin I<Plugin>. Unlike most other LoadPlugin lines, this one
39 should be a block containing the line "Globals true". This will cause collectd
40 to export the name of all objects in the python interpreter for all plugins to
41 see. If you don't do this or your platform does not support it, the embeded
42 interpreter will start anywa but you won't be able to load certain python
45 =item B<Encoding> I<Name>
47 The default encoding for Unicode objects you pass to collectd. If you omit this
48 option it will default to B<ascii> on I<Python 2> and B<utf-8> on I<Python 3>.
49 This is hardcoded in Python and will ignore everything else, including your
52 =item B<ModulePath> I<Name>
54 Appends I<Name> to B<sys.path>. You won't be able to import any scripts you
55 wrote unless they are located in one of the directories in this list. Please
56 note that it only has effect on plugins loaded after this option. You can
57 use multiple B<ModulePath> lines to add more than one directory.
59 =item B<LogTraces> I<bool>
61 If a python script throws an exception it will be logged by collectd with the
62 name of the exception and the message. If you set this option to true it will
63 also log the full stacktrace just like the default output of an interactive
64 python interpreter. This should probably be set to false most of the time but
65 is very useful for development and debugging of new modules.
67 =item B<Interactive> I<bool>
69 This option will cause the module to launch an interactive python interpreter
70 that reads from and writes to the terminal. Note that collectd will terminate
71 right after starting up if you try to run it as a daemon while this option is
72 enabled to make sure to start collectd with the B<-f> option.
74 The B<collectd> module is I<not> imported into the interpreter's globals. You
75 have to do it manually. Be sure to read the help text of the module, it can be
76 used as a reference guide during coding.
78 This interactive session will behave slightly differently from a daemonized
79 collectd script as well as from a normal python interpreter:
85 B<1.> collectd will try to import the B<readline> module to give you a decent
86 way of entering your commands. The daemonized collectd won't do that.
90 B<2.> collectd will block I<SIGINT>. Pressing I<Ctrl+C> will usually cause
91 collectd to shut down. This would be problematic in an interactive session,
92 therefore this signal will be blocked. You can still use it to interrupt
93 syscalls like sleep and pause but it won't generate a I<KeyboardInterrupt>
96 To quit collectd send I<EOF> (press I<Ctrl+D> at the beginning of a new line).
100 =item E<lt>B<Module> I<Name>E<gt> block
102 This block may be used to pass on configuration settings to a Python module.
103 The configuration is converted into an instance of the B<Config> class which is
104 passed to the registered configuration callback. See below for details about
105 the B<Config> class and how to register callbacks.
107 The I<name> identifies the callback.
113 There are a lot of places where strings are send from collectd to python and
114 from python to collectd. How exactly this works depends on wheather byte or
115 unicode strings or python2 or python3 are used.
117 Python2 has I<str>, which is just bytes, and I<unicode>. Python3 has I<str>,
118 which is a unicode object, and I<bytes>.
120 When passing strings from python to collectd all of these object are supported
121 in all places, however I<str> should be used if possible. These strings must
122 not contain a NUL byte. Ignoring this will result in a I<TypeError> exception.
123 If a byte string was used it will be used as is by collectd. If a unicode
124 object was used it will be encoded using the default encoding (see above). If
125 this is not possible python will raise a I<UnicodeEncodeError> exception.
127 Wenn passing strings from collectd to python the behavior depends on the
128 python version used. Python2 will always receive a I<str> object. Python3 will
129 usually receive a I<str> object as well, however the original string will be
130 decoded to unicode using the default encoding. If this fails because the
131 string is not a valid sequence for this encoding a I<bytes> object will be
134 =head1 WRITING YOUR OWN PLUGINS
136 Writing your own plugins is quite simple. collectd manages plugins by means of
137 B<dispatch functions> which call the appropriate B<callback functions>
138 registered by the plugins. Any plugin basically consists of the implementation
139 of these callback functions and initializing code which registers the
140 functions with collectd. See the section "EXAMPLES" below for a really basic
141 example. The following types of B<callback functions> are known to collectd
142 (all of them are optional):
146 =item configuration functions
148 This type of functions is called during configuration if an appropriate
149 B<Module> block has been encountered. It is called once for each B<Module>
150 block which matches the name of the callback as provided with the
151 B<register_config> method - see below.
153 Python thread support has not been initialized at this point so do not use any
154 threading functions here!
158 This type of functions is called once after loading the module and before any
159 calls to the read and write functions. It should be used to initialize the
160 internal state of the plugin (e.E<nbsp>g. open sockets, ...). This is the
161 earliest point where you may use threads.
165 This type of function is used to collect the actual data. It is called once
166 per interval (see the B<Interval> configuration option of collectd). Usually
167 it will call B<plugin_dispatch_values> to dispatch the values to collectd
168 which will pass them on to all registered B<write functions>. If this function
169 throws any kind of exception the plugin will be skipped for an increasing
170 amount of time until it returns normally again.
172 =item write functions
174 This type of function is used to write the dispatched values. It is called
175 once for every value that was dispatched by any plugin.
177 =item flush functions
179 This type of function is used to flush internal caches of plugins. It is
180 usually triggered by the user only. Any plugin which caches data before
181 writing it to disk should provide this kind of callback function.
185 This type of function is used to pass messages of plugins or the daemon itself
188 =item notification function
190 This type of function is used to act upon notifications. In general, a
191 notification is a status message that may be associated with a data instance.
192 Usually, a notification is generated by the daemon if a configured threshold
193 has been exceeded (see the section "THRESHOLD CONFIGURATION" in
194 L<collectd.conf(5)> for more details), but any plugin may dispatch
195 notifications as well.
197 =item shutdown functions
199 This type of function is called once before the daemon shuts down. It should
200 be used to clean up the plugin (e.g. close sockets, ...).
204 Any function (except log functions) may set throw an exception in case of any
205 errors. The exception will be passed on to the user using collectd's logging
206 mechanism. If a log callback throws an exception it will be printed to standard
209 See the documentation of the various B<register_> methods in the section
210 "FUNCTIONS" below for the number and types of arguments passed to each
211 B<callback function>. This section also explains how to register B<callback
212 functions> with collectd.
214 To enable a module, copy it to a place where Python can find it (i.E<nbsp>e. a
215 directory listed in B<sys.path>) just as any other Python plugin and add
216 an appropriate B<Import> option to the configuration file. After restarting
217 collectd you're done.
221 The following complex types are used to pass values between the Python plugin
226 The Config class is an object which keeps the informations provided in the
227 configuration file. The sequence of children keeps one entry for each
228 configuration option. Each such entry is another Config instance, which
229 may nest further if nested blocks are used.
233 This represents a piece of collectd's config file. It is passed to scripts with
234 config callbacks (see B<register_config>) and is of little use if created
237 It has no methods beyond the bare minimum and only exists for its data members.
239 Data descriptors defined here:
245 This represents the parent of this node. On the root node
246 of the config tree it will be None.
250 This is the keyword of this item, i.e. the first word of any given line in the
251 config file. It will always be a string.
255 This is a tuple (which might be empty) of all value, i.e. words following the
256 keyword in any given line in the config file.
258 Every item in this tuple will be either a string or a float or a boolean,
259 depending on the contents of the configuration file.
263 This is a tuple of child nodes. For most nodes this will be empty. If this node
264 represents a block instead of a single line of the config file it will contain
265 all nodes in this block.
271 This should not be used directly but it is the base class for both Values and
272 Notification. It is used to identify the source of a value or notification.
274 class PluginData(object)
276 This is an internal class that is the base for Values and Notification. It is
277 pretty useless by itself and was therefore not exported to the collectd module.
279 Data descriptors defined here:
285 The hostname of the host this value was read from. For dispatching this can be
286 set to an empty string which means the local hostname as defined in
291 The name of the plugin that read the data. Setting this member to an empty
292 string will insert "python" upon dispatching.
294 =item plugin_instance
296 Plugin instance string. May be empty.
300 This is the Unix timestamp of the time this value was read. For dispatching
301 values this can be set to zero which means "now". This means the time the value
302 is actually dispatched, not the time it was set to 0.
306 The type of this value. This type has to be defined in your I<types.db>.
307 Attempting to set it to any other value will raise a I<TypeError> exception.
308 Assigning a type is mandatory, calling dispatch without doing so will raise a
309 I<RuntimeError> exception.
313 Type instance string. May be empty.
319 A Value is an object which features a sequence of values. It is based on then
320 I<PluginData> type and uses its members to identify the values.
322 class Values(PluginData)
324 A Values object used for dispatching values to collectd and receiving values
325 from write callbacks.
327 Method resolution order:
339 Methods defined here:
343 =item B<dispatch>([type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.
345 Dispatch this instance to the collectd process. The object has members for each
346 of the possible arguments for this method. For a detailed explanation of these
347 parameters see the member of the same same.
349 If you do not submit a parameter the value saved in its member will be
350 submitted. If you do provide a parameter it will be used instead, without
353 =item B<write>([destination][, type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.
355 Write this instance to a single plugin or all plugins if "destination" is
356 omitted. This will bypass the main collectd process and all filtering and
357 caching. Other than that it works similar to "dispatch". In most cases
358 "dispatch" should be used instead of "write".
362 Data descriptors defined here:
368 The interval is the timespan in seconds between two submits for the same data
369 source. This value has to be a positive integer, so you can't submit more than
370 one value per second. If this member is set to a non-positive value, the
371 default value as specified in the config file will be used (default: 10).
373 If you submit values more often than the specified interval, the average will
374 be used. If you submit less values, your graphs will have gaps.
378 These are the actual values that get dispatched to collectd. It has to be a
379 sequence (a tuple or list) of numbers. The size of the sequence and the type of
380 its content depend on the type member your I<types.db> file. For more
381 information on this read the L<types.db(5)> manual page.
383 If the sequence does not have the correct size upon dispatch a I<RuntimeError>
384 exception will be raised. If the content of the sequence is not a number, a
385 I<TypeError> exception will be raised.
391 A notification is an object defining the severity and message of the status
392 message as well as an identification of a data instance by means of the members
393 of I<PluginData> on which it is based.
395 class Notification(PluginData)
396 The Notification class is a wrapper around the collectd notification.
397 It can be used to notify other plugins about bad stuff happening. It works
398 similar to Values but has a severity and a message instead of interval
400 Notifications can be dispatched at any time and can be received with
401 register_notification.
403 Method resolution order:
415 Methods defined here:
419 =item B<dispatch>([type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None. Dispatch a value list.
421 Dispatch this instance to the collectd process. The object has members for each
422 of the possible arguments for this method. For a detailed explanation of these
423 parameters see the member of the same same.
425 If you do not submit a parameter the value saved in its member will be
426 submitted. If you do provide a parameter it will be used instead, without
431 Data descriptors defined here:
437 Some kind of description what's going on and why this Notification was
442 The severity of this notification. Assign or compare to I<NOTIF_FAILURE>,
443 I<NOTIF_WARNING> or I<NOTIF_OKAY>.
449 The following functions provide the C-interface to Python-modules.
453 =item B<register_*>(I<callback>[, I<data>][, I<name>]) -> identifier
455 There are eight different register functions to get callback for eight
456 different events. With one exception all of them are called as shown above.
462 I<callback> is a callable object that will be called every time the event is
467 I<data> is an optional object that will be passed back to the callback function
468 every time it is called. If you omit this parameter no object is passed back to
469 your callback, not even None.
473 I<name> is an optional identifier for this callback. The default name is
474 B<python>.I<module>. I<module> is taken from the B<__module__> attribute of
475 your callback function. Every callback needs a unique identifier, so if you
476 want to register the same callback multiple time in the same module you need to
477 specify a name here. Otherwise it's save to ignore this parameter I<identifier>
478 is the full identifier assigned to this callback.
482 These functions are called in the various stages of the daemon (see the section
483 L<"WRITING YOUR OWN PLUGINS"> above) and are passed the following arguments:
487 =item register_config
489 The only argument passed is a I<Config> object. See above for the layout of this
491 Note that you can not receive the whole config files this way, only B<Module>
492 blocks inside the Python configuration block. Additionally you will only
493 receive blocks where your callback identifier matches B<python.>I<blockname>.
497 The callback will be called without arguments.
499 =item register_read(callback[, interval][, data][, name]) -> identifier
501 This function takes an additional parameter: I<interval>. It specifies the
502 time between calls to the callback function.
504 The callback will be called without arguments.
506 =item register_shutdown
508 The callback will be called without arguments.
512 The callback function will be called with one arguments passed, which will be a
513 I<Values> object. For the layout of I<Values> see above.
514 If this callback function throws an exception the next call will be delayed by
515 an increasing interval.
519 Like B<register_config> is important for this callback because it determines
520 what flush requests the plugin will receive.
522 The arguments passed are I<timeout> and I<identifier>. I<timeout> indicates
523 that only data older than I<timeout> seconds is to be flushed. I<identifier>
524 specifies which values are to be flushed.
528 The arguments are I<severity> and I<message>. The severity is an integer and
529 small for important messages and high for less important messages. The least
530 important level is B<LOG_DEBUG>, the most important level is B<LOG_ERR>. In
531 between there are (from least to most important): B<LOG_INFO>, B<LOG_NOTICE>,
532 and B<LOG_WARNING>. I<message> is simply a string B<without> a newline at the
535 If this callback throws an exception it will B<not> be logged. It will just be
536 printed to B<sys.stderr> which usually means silently ignored.
538 =item register_notification
540 The only argument passed is a I<Notification> object. See above for the layout of this
545 =item B<unregister_*>(I<identifier>) -> None
547 Removes a callback or data-set from collectd's internal list of callback
548 functions. Every I<register_*> function has an I<unregister_*> function.
549 I<identifier> is either the string that was returned by the register function
550 or a callback function. The identifier will be constructed in the same way as
551 for the register functions.
553 =item B<flush>(I<plugin[, I<timeout>][, I<identifier>]) -> None
555 Flush one or all plugins. I<timeout> and the specified I<identifiers> are
556 passed on to the registered flush-callbacks. If omitted, the timeout defaults
557 to C<-1>. The identifier defaults to None. If the B<plugin> argument has been
558 specified, only named plugin will be flushed.
560 =item B<error>, B<warning>, B<notice>, B<info>, B<debug>(I<message>)
562 Log a message with the specified severity.
568 Any Python module will start similar to:
572 A very simple read function might look like:
575 vl = collectd.Values(type='gauge')
576 vl.plugin='python.spam'
577 vl.dispatch(values=[random.random() * 100])
579 A very simple write function might look like:
581 def write(vl, data=None):
583 print "%s (%s): %f" % (vl.plugin, vl.type, i)
585 To register those functions with collectd:
587 collectd.register_read(read);
588 collectd.register_write(write);
590 See the section L<"CLASSES"> above for a complete documentation of the data
591 types used by the read, write and match functions.
599 Please feel free to send in new plugins to collectd's mailinglist at
600 E<lt>collectdE<nbsp>atE<nbsp>verplant.orgE<gt> for review and, possibly,
601 inclusion in the main distribution. In the latter case, we will take care of
602 keeping the plugin up to date and adapting it to new versions of collectd.
604 Before submitting your plugin, please take a look at
605 L<http://collectd.org/dev-info.shtml>.
615 collectd is heavily multi-threaded. Each collectd thread accessing the python
616 plugin will be mapped to a Python interpreter thread. Any such thread will be
617 created and destroyed transparently and on-the-fly.
619 Hence, any plugin has to be thread-safe if it provides several entry points
620 from collectd (i.E<nbsp>e. if it registers more than one callback or if a
621 registered callback may be called more than once in parallel).
625 The Python thread module is initialized just before calling the init callbacks.
626 This means you must not use Python's threading module prior to this point. This
627 includes all config and possibly other callback as well.
631 The python plugin exports the internal API of collectd which is considered
632 unstable and subject to change at any time. We try hard to not break backwards
633 compatibility in the Python API during the life cycle of one major release.
634 However, this cannot be guaranteed at all times. Watch out for warnings
635 dispatched by the python plugin after upgrades.
645 This plugin is not compatible with python3. Trying to compile it with python3
646 will fail because of the ways string, unicode and bytearray bahavior was
651 Not all aspects of the collectd API are accessible from python. This includes
652 but is not limited to meta-data, filters and data sets.
667 The C<python plugin> has been written by
668 Sven Trenkel E<lt>collectdE<nbsp>atE<nbsp>semidefinite.deE<gt>.
670 This manpage has been written by Sven Trenkel
671 E<lt>collectdE<nbsp>atE<nbsp>semidefinite.deE<gt>.
672 It is based on the L<collectd-perl(5)> manual page by
673 Florian Forster E<lt>octoE<nbsp>atE<nbsp>verplant.orgE<gt> and
674 Sebastian Harl E<lt>shE<nbsp>atE<nbsp>tokkee.orgE<gt>.