Panoptes Docs

Welcome to the Panoptes project!

Panoptes is a personal distributed reactive automation framework written in modern Perl 5.

This project contains a working implementation but is still under development and is considered alpha. Stay tuned.

This project is hosted on github:

The documentation is now online here:

The panoptes project aims to deliver the following components:

Panoptes::Monitor

The Panoptes::Monitor component contains the monitoring plugins that observe events and then generate a messages for each event. The message may be a simple hash of key-value pairs, or may contain complex data structures.

The monitors are built on POE, a powerful cooperative multitasking framework. The monitors share a DBM::Deep database for storing state information between checks and agent restarts.

The idea is that no significant processing or analysis of the data is done in this layer--that takes place in the back-end rules processing (see Panoptes::Rule).

The existing monitoring plugins include:

To see how a monitor is configured, check out the 'getting started' tutorial:

If you're interested in developing your own monitor(s), see:

Panoptes::Messaging

This component connects the monitoring front-end to the reactive back-end.

Panoptes::Messaging serializes the data structure as YAML and stores it on disk in a queue to be picked up by Panoptes::Rule. This queue may be picked up on the same machine or on a remote machine if the machine is an agent.

The messaging is asynchronous (a mailbox) to ensure that no messages will be missed if the back-end is down, and also that the front-end will never get stuck waiting for the more cpu and/or disk intensive back-end.

Currently a local queue is implemented as a mailbox (maildir). There is an IPC::DirQueue planned. Panoptes::Messaging is a factory, so the most appropriate messaging implementation can be chosen at runtime.

The distributed client/server messaging component is incomplete, but will be a PoCo::Server::TCP speaking YAML with queuing. I have been doing a lot of research into other CPAN modules that could be used here but have not yet found one that fits the current list of requirements. I do plan to eventually provide a mature messaging implementation option here, although it might not be implemented in perl.

Panoptes::Rules

This component parses and evaluates the rules for each message in the event stream. Each rule is composed of two parts: condition(s) and action(s).

Right now there are a number of simple rule conditions, e.g., 'matches', which checks that a message contains a field that matches a regexp. I intend to develop some much more interesting rule conditions that will be useful for CEP. For example, conditions that can read shared state/context information generated by other plugins. This would allow rule conditions to specify contexts including the user's current location, network connectivity, or idle status. Another example is floating time windows which would allow conditions matching a certain number of events within a specific period. Yet another example would be the lack of occurrance of some event withing a specific window of time.

A few basic actions have been developed so far. A couple of notifications plugins have been built, one that uses Growl and another that displays colorized messages to the console. Other plugins include a simple add_field plugin that allows rules to add additional context to an event. Right now this is mostly useful to affect the notifications (e.g. setting flags to colorize messages based on content). Actions have also been created to write message data (or portions thereof) to SQLite and RRD databases.

A Mojolicious::Lite web interface is also under development to view this data.

Currently the 'reactive engine' is the simplest thing that could possibly work, i.e. one big list of rules. Every event is run through the condition(s) of every rule. This is obviously a hack and is not efficient for large sets of rules. I'm working on building a much more efficient implementation, and I plan to allow for and provide more than one engine implementation so that the most appropriate one can be selected at runtime. Since the reactive engine is loosely coupled to the monitoring engine, there is also the possibility of using other more powerful and mature tools here, e.g. Esper.

To see an example of rules in action, see the 'getting started' tutorial:

GRID::Launcher (formerly Panoptes::GRID)

This component is responsible for distributing panoptes agents to remote machines. This is still under development and is not yet fully functional.

Panoptes can deploy it's own agents using GRID::Machine to any machine that run ssh and perl. The only requirement of the client is that you can log in over ssh and execute perl 5. Additionally, if you want all agent messaging forwarded over SSH, the ssh server must allow port forwarding (to a port on localhost).

Once the agent is running on the remote machine, it can quickly and efficiently monitor local events (e.g. running the equivalent of 'tail -f' on your log files, checking resources and services at regular intervals, etc.). Any interesting events that are seen can be processed by local rules, sent back to the remote spawning process (the matriarch), sent to peer agents, or even processed through intermediate agents.

The agents may continue to operate independently of the matriarch if the connection is lost. All events will be queued and delivered in order when the connection is re-established.

Panoptes::Config

The configuration of a distributed monitoring system can get hairy. The Panoptes::Config module tries to encapsulate and simplify the configuration of multiple plugins on multiple hosts and collections of hosts.

The current implementation relies heavily on Config::Merge.

This is still a work in progress, but might hopefully someday evolve into GRID::Config.

There is now a sample-conf directory in the root of the panoptes directory that has some sample configuration files. Please be aware that the config schema will be in flux for a while as the project continues to develop.

See Also