1 # Permission is hereby granted, free of charge, to any person obtaining a
2 # copy of this software and associated documentation files (the "Software"),
3 # to deal in the Software without restriction, including without limitation
4 # the rights to use, copy, modify, merge, publish, distribute, sublicense,
5 # and/or sell copies of the Software, and to permit persons to whom the
6 # Software is furnished to do so, subject to the following conditions:
8 # The above copyright notice and this permission notice shall be included in
9 # all copies or substantial portions of the Software.
15 collectd-python - Documentation of collectd's C<python plugin>
22 ModulePath "/path/to/your/python/modules"
28 spam "wonderful" "lovely"
34 The C<python plugin> embeds a Python-interpreter into collectd and provides an
35 interface to collectd's plugin system. This makes it possible to write plugins
36 for collectd in Python. This is a lot more efficient than executing a
37 Python-script every time you want to read a value with the C<exec plugin> (see
38 L<collectd-exec(5)>) and provides a lot more functionality, too.
40 The minimum required Python version is I<2.6>.
46 =item B<LoadPlugin> I<Plugin>
48 Loads the Python plugin I<Plugin>.
50 =item B<Encoding> I<Name>
52 The default encoding for Unicode objects you pass to collectd. If you omit this
53 option it will default to B<ascii> on I<Python 2>. On I<Python 3> it will
54 always be B<utf-8>, as this function was removed, so this will be silently
56 These defaults are hardcoded in Python and will ignore everything else,
57 including your locale.
59 =item B<ModulePath> I<Name>
61 Prepends I<Name> to B<sys.path>. You won't be able to import any scripts you
62 wrote unless they are located in one of the directories in this list. Please
63 note that it only has effect on plugins loaded after this option. You can
64 use multiple B<ModulePath> lines to add more than one directory.
66 =item B<LogTraces> I<bool>
68 If a Python script throws an exception it will be logged by collectd with the
69 name of the exception and the message. If you set this option to true it will
70 also log the full stacktrace just like the default output of an interactive
71 Python interpreter. This does not apply to the CollectError exception, which
72 will never log a stacktrace.
73 This should probably be set to false most of the time but is very useful for
74 development and debugging of new modules.
76 =item B<Interactive> I<bool>
78 This option will cause the module to launch an interactive Python interpreter
79 that reads from and writes to the terminal. Note that collectd will terminate
80 right after starting up if you try to run it as a daemon while this option is
81 enabled so make sure to start collectd with the B<-f> option.
83 The B<collectd> module is I<not> imported into the interpreter's globals. You
84 have to do it manually. Be sure to read the help text of the module, it can be
85 used as a reference guide during coding.
87 This interactive session will behave slightly differently from a daemonized
88 collectd script as well as from a normal Python interpreter:
94 B<1.> collectd will try to import the B<readline> module to give you a decent
95 way of entering your commands. The daemonized collectd won't do that.
99 B<2.> Python will be handling I<SIGINT>. Pressing I<Ctrl+C> will usually cause
100 collectd to shut down. This would be problematic in an interactive session,
101 therefore Python will be handling it in interactive sessions. This allows you
102 to use I<Ctrl+C> to interrupt Python code without killing collectd. This also
103 means you can catch I<KeyboardInterrupt> exceptions which does not work during
106 To quit collectd send I<EOF> (press I<Ctrl+D> at the beginning of a new line).
110 B<3.> collectd handles I<SIGCHLD>. This means that Python won't be able to
111 determine the return code of spawned processes with system(), popen() and
112 subprocess. This will result in Python not using external programs like less
113 to display help texts. You can override this behavior with the B<PAGER>
114 environment variable, e.g. I<export PAGER=less> before starting collectd.
115 Depending on your version of Python this might or might not result in an
116 B<OSError> exception which can be ignored.
118 If you really need to spawn new processes from Python you can register an init
119 callback and reset the action for SIGCHLD to the default behavior. Please note
120 that this I<will> break the exec plugin. Do not even load the exec plugin if
121 you intend to do this!
123 There is an example script located in B<contrib/python/getsigchld.py> to do
124 this. If you import this from I<collectd.conf> SIGCHLD will be handled
125 normally and spawning processes from Python will work as intended.
129 =item B<Import> I<Name>
131 Imports the python script I<Name> and loads it into the collectd
132 python process. If your python script is not found, be sure its
133 directory exists in python's B<sys.path>. You can prepend to the
134 B<sys.path> using the B<ModulePath> configuration option.
136 =item E<lt>B<Module> I<Name>E<gt> block
138 This block may be used to pass on configuration settings to a Python module.
139 The configuration is converted into an instance of the B<Config> class which is
140 passed to the registered configuration callback. See below for details about
141 the B<Config> class and how to register callbacks.
143 The I<name> identifies the callback.
149 There are a lot of places where strings are sent from collectd to Python and
150 from Python to collectd. How exactly this works depends on whether byte or
151 unicode strings or Python2 or Python3 are used.
153 Python2 has I<str>, which is just bytes, and I<unicode>. Python3 has I<str>,
154 which is a unicode object, and I<bytes>.
156 When passing strings from Python to collectd all of these object are supported
157 in all places, however I<str> should be used if possible. These strings must
158 not contain a NUL byte. Ignoring this will result in a I<TypeError> exception.
159 If a byte string was used it will be used as is by collectd. If a unicode
160 object was used it will be encoded using the default encoding (see above). If
161 this is not possible Python will raise a I<UnicodeEncodeError> exception.
163 When passing strings from collectd to Python the behavior depends on the
164 Python version used. Python2 will always receive a I<str> object. Python3 will
165 usually receive a I<str> object as well, however the original string will be
166 decoded to unicode using the default encoding. If this fails because the
167 string is not a valid sequence for this encoding a I<bytes> object will be
170 =head1 WRITING YOUR OWN PLUGINS
172 Writing your own plugins is quite simple. collectd manages plugins by means of
173 B<dispatch functions> which call the appropriate B<callback functions>
174 registered by the plugins. Any plugin basically consists of the implementation
175 of these callback functions and initializing code which registers the
176 functions with collectd. See the section "EXAMPLES" below for a really basic
177 example. The following types of B<callback functions> are known to collectd
178 (all of them are optional):
182 =item configuration functions
184 These are called during configuration if an appropriate
185 B<Module> block has been encountered. It is called once for each B<Module>
186 block which matches the name of the callback as provided with the
187 B<register_config> method - see below.
189 Python thread support has not been initialized at this point so do not use any
190 threading functions here!
194 These are called once after loading the module and before any
195 calls to the read and write functions. It should be used to initialize the
196 internal state of the plugin (e.E<nbsp>g. open sockets, ...). This is the
197 earliest point where you may use threads.
201 These are used to collect the actual data. It is called once
202 per interval (see the B<Interval> configuration option of collectd). Usually
203 it will call B<plugin_dispatch_values> to dispatch the values to collectd
204 which will pass them on to all registered B<write functions>. If this function
205 throws any kind of exception the plugin will be skipped for an increasing
206 amount of time until it returns normally again.
208 =item write functions
210 These are used to write the dispatched values. It is called
211 once for every value that was dispatched by any plugin.
213 =item flush functions
215 These are used to flush internal caches of plugins. It is
216 usually triggered by the user only. Any plugin which caches data before
217 writing it to disk should provide this kind of callback function.
221 These are used to pass messages of plugins or the daemon itself
224 =item notification function
226 These are used to act upon notifications. In general, a
227 notification is a status message that may be associated with a data instance.
228 Usually, a notification is generated by the daemon if a configured threshold
229 has been exceeded (see the section "THRESHOLD CONFIGURATION" in
230 L<collectd.conf(5)> for more details), but any plugin may dispatch
231 notifications as well.
233 =item shutdown functions
235 These are called once before the daemon shuts down. It should
236 be used to clean up the plugin (e.g. close sockets, ...).
240 Any function (except log functions) may throw an exception in case of
241 errors. The exception will be passed on to the user using collectd's logging
242 mechanism. If a log callback throws an exception it will be printed to standard
245 See the documentation of the various B<register_> methods in the section
246 "FUNCTIONS" below for the number and types of arguments passed to each
247 B<callback function>. This section also explains how to register B<callback
248 functions> with collectd.
250 To enable a module, copy it to a place where Python can find it (i.E<nbsp>e. a
251 directory listed in B<sys.path>) just as any other Python plugin and add
252 an appropriate B<Import> option to the configuration file. After restarting
253 collectd you're done.
257 The following complex types are used to pass values between the Python plugin
262 This is an exception. If any Python script raises this exception it will
263 still be treated like an error by collectd but it will be logged as a
264 warning instead of an error and it will never generate a stacktrace.
266 class CollectdError(Exception)
268 Basic exception for collectd Python scripts.
269 Throwing this exception will not cause a stacktrace to be logged, even if
270 LogTraces is enabled in the config.
274 The Signed class is just a long. It has all its methods and behaves exactly
275 like any other long object. It is used to indicate if an integer was or should
276 be stored as a signed or unsigned integer object.
280 This is a long by another name. Use it in meta data dicts
281 to choose the way it is stored in the meta data.
285 The Unsigned class is just a long. It has all its methods and behaves exactly
286 like any other long object. It is used to indicate if an integer was or should
287 be stored as a signed or unsigned integer object.
291 This is a long by another name. Use it in meta data dicts
292 to choose the way it is stored in the meta data.
296 The Config class is an object which keeps the information provided in the
297 configuration file. The sequence of children keeps one entry for each
298 configuration option. Each such entry is another Config instance, which
299 may nest further if nested blocks are used.
303 This represents a piece of collectd's config file. It is passed to scripts with
304 config callbacks (see B<register_config>) and is of little use if created
307 It has no methods beyond the bare minimum and only exists for its data members.
309 Data descriptors defined here:
315 This represents the parent of this node. On the root node
316 of the config tree it will be None.
320 This is the keyword of this item, i.e. the first word of any given line in the
321 config file. It will always be a string.
325 This is a tuple (which might be empty) of all value, i.e. words following the
326 keyword in any given line in the config file.
328 Every item in this tuple will be either a string, a float or a boolean,
329 depending on the contents of the configuration file.
333 This is a tuple of child nodes. For most nodes this will be empty. If this node
334 represents a block instead of a single line of the config file it will contain
335 all nodes in this block.
341 This should not be used directly but it is the base class for both Values and
342 Notification. It is used to identify the source of a value or notification.
344 class PluginData(object)
346 This is an internal class that is the base for Values and Notification. It is
347 pretty useless by itself and was therefore not exported to the collectd module.
349 Data descriptors defined here:
355 The hostname of the host this value was read from. For dispatching this can be
356 set to an empty string which means the local hostname as defined in
361 The name of the plugin that read the data. Setting this member to an empty
362 string will insert "python" upon dispatching.
364 =item plugin_instance
366 Plugin instance string. May be empty.
370 This is the Unix timestamp of the time this value was read. For dispatching
371 values this can be set to zero which means "now". This means the time the value
372 is actually dispatched, not the time it was set to 0.
376 The type of this value. This type has to be defined in your I<types.db>.
377 Attempting to set it to any other value will raise a I<TypeError> exception.
378 Assigning a type is mandatory, calling dispatch without doing so will raise a
379 I<RuntimeError> exception.
383 Type instance string. May be empty.
389 A Value is an object which features a sequence of values. It is based on the
390 I<PluginData> type and uses its members to identify the values.
392 class Values(PluginData)
394 A Values object used for dispatching values to collectd and receiving values
395 from write callbacks.
397 Method resolution order:
409 Methods defined here:
413 =item B<dispatch>([type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.
415 Dispatch this instance to the collectd process. The object has members for each
416 of the possible arguments for this method. For a detailed explanation of these
417 parameters see the member of the same same.
419 If you do not submit a parameter the value saved in its member will be
420 submitted. If you do provide a parameter it will be used instead, without
423 =item B<write>([destination][, type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.
425 Write this instance to a single plugin or all plugins if "destination" is
426 omitted. This will bypass the main collectd process and all filtering and
427 caching. Other than that it works similar to "dispatch". In most cases
428 "dispatch" should be used instead of "write".
432 Data descriptors defined here:
438 The interval is the timespan in seconds between two submits for the same data
439 source. This value has to be a positive integer, so you can't submit more than
440 one value per second. If this member is set to a non-positive value, the
441 default value as specified in the config file will be used (default: 10).
443 If you submit values more often than the specified interval, the average will
444 be used. If you submit less values, your graphs will have gaps.
448 These are the actual values that get dispatched to collectd. It has to be a
449 sequence (a tuple or list) of numbers. The size of the sequence and the type of
450 its content depend on the type member your I<types.db> file. For more
451 information on this read the L<types.db(5)> manual page.
453 If the sequence does not have the correct size upon dispatch a I<RuntimeError>
454 exception will be raised. If the content of the sequence is not a number, a
455 I<TypeError> exception will be raised.
459 These are the meta data for this Value object.
460 It has to be a dictionary of numbers, strings or bools. All keys must be
461 strings. I<int> and <long> objects will be dispatched as signed integers unless
462 they are between 2**63 and 2**64-1, which will result in a unsigned integer.
463 You can force one of these storage classes by using the classes
464 B<collectd.Signed> and B<collectd.Unsigned>. A meta object received by a write
465 callback will always contain B<Signed> or B<Unsigned> objects.
471 A notification is an object defining the severity and message of the status
472 message as well as an identification of a data instance by means of the members
473 of I<PluginData> on which it is based.
475 class Notification(PluginData)
476 The Notification class is a wrapper around the collectd notification.
477 It can be used to notify other plugins about bad stuff happening. It works
478 similar to Values but has a severity and a message instead of interval
480 Notifications can be dispatched at any time and can be received with
481 register_notification.
483 Method resolution order:
495 Methods defined here:
499 =item B<dispatch>([type][, message][, plugin_instance][, type_instance][, plugin][, host][, time][, severity][, meta]) -> None. Dispatch a notification.
501 Dispatch this instance to the collectd process. The object has members for each
502 of the possible arguments for this method. For a detailed explanation of these
503 parameters see the member of the same same.
505 If you do not submit a parameter the value saved in its member will be
506 submitted. If you do provide a parameter it will be used instead, without
511 Data descriptors defined here:
517 Some kind of description of what's going on and why this Notification was
522 The severity of this notification. Assign or compare to I<NOTIF_FAILURE>,
523 I<NOTIF_WARNING> or I<NOTIF_OKAY>.
527 These are the meta data for the Notification object.
528 It has to be a dictionary of numbers, strings or bools. All keys must be
529 strings. I<int> and I<long> objects will be dispatched as signed integers unless
530 they are between 2**63 and 2**64-1, which will result in a unsigned integer.
531 One of these storage classes can be forced by using the classes
532 B<collectd.Signed> and B<collectd.Unsigned>. A meta object received by a
533 notification callback will always contain B<Signed> or B<Unsigned> objects.
539 The following functions provide the C-interface to Python-modules.
543 =item B<register_*>(I<callback>[, I<data>][, I<name>]) -> identifier
545 There are eight different register functions to get callback for eight
546 different events. With one exception all of them are called as shown above.
552 I<callback> is a callable object that will be called every time the event is
557 I<data> is an optional object that will be passed back to the callback function
558 every time it is called. If you omit this parameter no object is passed back to
559 your callback, not even None.
563 I<name> is an optional identifier for this callback. The default name is
564 B<python>.I<module>. I<module> is taken from the B<__module__> attribute of
565 your callback function. Every callback needs a unique identifier, so if you
566 want to register the same callback multiple times in the same module you need to
567 specify a name here. Otherwise it's safe to ignore this parameter.
571 I<identifier> is the full identifier assigned to this callback.
575 These functions are called in the various stages of the daemon (see the section
576 L<"WRITING YOUR OWN PLUGINS"> above) and are passed the following arguments:
580 =item register_config
582 The only argument passed is a I<Config> object. See above for the layout of this
584 Note that you cannot receive the whole config files this way, only B<Module>
585 blocks inside the Python configuration block. Additionally you will only
586 receive blocks where your callback identifier matches B<python.>I<blockname>.
590 The callback will be called without arguments.
592 =item register_read(callback[, interval][, data][, name]) -> I<identifier>
594 This function takes an additional parameter: I<interval>. It specifies the
595 time between calls to the callback function.
597 The callback will be called without arguments.
599 =item register_shutdown
601 The callback will be called without arguments.
605 The callback function will be called with one argument passed, which will be a
606 I<Values> object. For the layout of I<Values> see above.
607 If this callback function throws an exception the next call will be delayed by
608 an increasing interval.
612 Like B<register_config> is important for this callback because it determines
613 what flush requests the plugin will receive.
615 The arguments passed are I<timeout> and I<identifier>. I<timeout> indicates
616 that only data older than I<timeout> seconds is to be flushed. I<identifier>
617 specifies which values are to be flushed.
621 The arguments are I<severity> and I<message>. The severity is an integer and
622 small for important messages and high for less important messages. The least
623 important level is B<LOG_DEBUG>, the most important level is B<LOG_ERR>. In
624 between there are (from least to most important): B<LOG_INFO>, B<LOG_NOTICE>,
625 and B<LOG_WARNING>. I<message> is simply a string B<without> a newline at the
628 If this callback throws an exception it will B<not> be logged. It will just be
629 printed to B<sys.stderr> which usually means silently ignored.
631 =item register_notification
633 The only argument passed is a I<Notification> object. See above for the layout of this
638 =item B<unregister_*>(I<identifier>) -> None
640 Removes a callback or data-set from collectd's internal list of callback
641 functions. Every I<register_*> function has an I<unregister_*> function.
642 I<identifier> is either the string that was returned by the register function
643 or a callback function. The identifier will be constructed in the same way as
644 for the register functions.
646 =item B<get_dataset>(I<name>) -> I<definition>
648 Returns the definition of a dataset specified by I<name>. I<definition> is a list
649 of tuples, each representing one data source. Each tuple has 4 values:
655 A string, the name of the data source.
659 A string that is equal to either of the variables B<DS_TYPE_COUNTER>,
660 B<DS_TYPE_GAUGE>, B<DS_TYPE_DERIVE> or B<DS_TYPE_ABSOLUTE>.
664 A float or None, the minimum value.
668 A float or None, the maximum value.
672 =item B<flush>(I<plugin[, timeout][, identifier]) -> None
674 Flush one or all plugins. I<timeout> and the specified I<identifiers> are
675 passed on to the registered flush-callbacks. If omitted, the timeout defaults
676 to C<-1>. The identifier defaults to None. If the B<plugin> argument has been
677 specified, only named plugin will be flushed.
679 =item B<error>, B<warning>, B<notice>, B<info>, B<debug>(I<message>)
681 Log a message with the specified severity.
687 Any Python module will start similar to:
691 A very simple read function might look like:
696 vl = collectd.Values(type='gauge')
697 vl.plugin='python.spam'
698 vl.dispatch(values=[random.random() * 100])
700 A very simple write function might look like:
702 def write(vl, data=None):
704 print "%s (%s): %f" % (vl.plugin, vl.type, i)
706 To register those functions with collectd:
708 collectd.register_read(read)
709 collectd.register_write(write)
711 See the section L<"CLASSES"> above for a complete documentation of the data
712 types used by the read, write and match functions.
720 collectd is heavily multi-threaded. Each collectd thread accessing the Python
721 plugin will be mapped to a Python interpreter thread. Any such thread will be
722 created and destroyed transparently and on-the-fly.
724 Hence, any plugin has to be thread-safe if it provides several entry points
725 from collectd (i.E<nbsp>e. if it registers more than one callback or if a
726 registered callback may be called more than once in parallel).
730 The Python thread module is initialized just before calling the init callbacks.
731 This means you must not use Python's threading module prior to this point. This
732 includes all config and possibly other callback as well.
736 The python plugin exports the internal API of collectd which is considered
737 unstable and subject to change at any time. We try hard to not break backwards
738 compatibility in the Python API during the life cycle of one major release.
739 However, this cannot be guaranteed at all times. Watch out for warnings
740 dispatched by the python plugin after upgrades.
750 Not all aspects of the collectd API are accessible from Python. This includes
751 but is not limited to filters.
766 The C<python plugin> has been written by
767 Sven Trenkel E<lt>collectdE<nbsp>atE<nbsp>semidefinite.deE<gt>.
769 This manpage has been written by Sven Trenkel
770 E<lt>collectdE<nbsp>atE<nbsp>semidefinite.deE<gt>.
771 It is based on the L<collectd-perl(5)> manual page by
772 Florian Forster E<lt>octoE<nbsp>atE<nbsp>collectd.orgE<gt> and
773 Sebastian Harl E<lt>shE<nbsp>atE<nbsp>tokkee.orgE<gt>.