Panoptes Docs

Why

In the process of interacting with panoptes, you may occasionally stop and ask: why?

This document is designed to help answer some of those questions.

Design

A few years ago I discovered growl for OS X. It pops up pretty little transparent notifications when some interesting event occurs. Lots of applications use this tool to display notifications for, e.g. new email, new rss feed content, irc messages, and more.

It even came with a perl library that let me display pop up notifications for all my perl scripts. It was so great that I got used to it. I depended on that little notification to help remind me to return to one task while I multitasking on others. I relied on them to let me know when when I had important new email in my mail client or web mail. I would also see if I missed any interesting or important info coming in from my various rss feeds. I even used it to notify me when there was a new problem on one of the many computers on my network. And then I started to run into some issues. Not being a computer myself, every now and then I step away from my monitor for a couple of minutes. In that period of time I would sometimes miss notifications. Fortunately there was a simple fix for that--just set every reasonably important notifications to be 'sticky'. Then the notification remains on the screen until you click it.

The problem is that I get a lot of notifications. I spend nearly all my waking life looking into a computer screen, but every now and then I need some sleep. When I wake up in the morning, my screen would often be covered in sticky notifications. Occasionally there would be screens of backed up notifications waiting to be displayed. Sometimes I needed to do something immediately on my computer, so I would click through all the notifications without reading them. But what did I miss? Did I miss something important? An email or IM from a co-worker? A new bug report? A new software release? A system or service that went down?

I got increasingly frustrated with these issues. I described my problem to my wife, who is also a programmer, and her response was that I was trying to use the tool to do something it wasn't designed to do. And then I realized she was right!

So I went back to the drawing board.

At the heart of the problem was that I was tired of feeling like a monkey, constantly polling all the various feeder bars for new information. I was tired of constantly polling my various email boxes, my multiple IRC and other IM clients, my RSS feeds, twitter, my system logs and utilization levels, nagios, my top web sites, and every other external service/tool/window to see what was happening. Can't a computer do this for me? I wanted the information brought to me, and I wanted to do things like prioritized based on a few (mostly) simple rules. I wanted to know what's going on right now! And then quick--without lifting a finger--how about now?

I started working on a new unified tool that polled all my information sources for me. The result is panoptes. It isn't my email client, but it knows if I have new email. How could I replace mutt or Mail.app? It isn't my IM client, but it knows if someone is trying to get in touch with me. I love irssi, I wouldn't want to use another interface for IRC. Many of these tools have options to configure notifications, but the rules tend to be very limited and/or simplistic, and the options vary greatly from program to program. Somehow I could never configure the notifications in any of these programs to match the configuration I wanted.

Panoptes can send a notifications over growl, or over email, or IM, even send a text message to your phone. Or it could do all of those things. It can know your location, your idle time, and your sleep schedule, and it can take those things into account when trying to notify you. It's your very own expert system that's tapped into your electronic life. And (hopefully) eventually it won't be much more complicated than using email filters.

It might be worth mentioning that I've written 6 fully functional prototypes for panoptes. The previous versions were designed similar to many other distributed monitoring and notification tools. Most of the monitoring and notification logic was tightly coupled in the system, i.e. the logic of whether or not a fault had occurred existed or was handled within the monitoring plugins. For example, the load plugin would examine whether or not the load had exceeded some configured thresholds. This turned out to be a mistake. For one, the range of what is considered a problem varies greatly. This led to some complex code for dealing with all the possible edge cases or error conditions. This was all really similar but slightly different from plugin to plugin. Also, all the error condition checking had to be done in code, so when someone defined a new error condition, I had no choice but to add some new config params and write some more code to do the error checking. I was continually searching for ways to separate all the error condition checking logic from the monitoring logic without making the monitoring plugins too complex.

I found the pattern I was looking for in the AtomsMasher project.

While AtomsMasher only processed RSS feeds, they separated the monitoring logic from the rules logic. The rules in AtomsMasher were similar to the sort of rules that might be used in an email client. So no code need be written to check for new error conditions. All the error checking could be done in a very generic and reusable way and defined in configuration rather than in code. It was infinitely more flexible. It was a Eureka moment for this project.

So, I threw away all my old code and started writing the current version of panoptes.

I've been slowly porting over plugins from my previous prototypes and working through a few issues in the new design. I had some pretty useful stuff hard-coded into the old monitors, and it's going to take me a while to develop the rules engine enough to be able to replicate those features.

Another piece of my long-term vision for this project is to become increasingly PIM aware. I have already started writing test cases for a BBDB inspired rule to track info about users across events. This would allow you to bring up a specific user's record and see all your interactions with that person across all monitors (email, im, twitter, rss feeds, etc). I am very GTD oriented and plan to incorporate some features inspired by this approach.

I am also very influenced by wearable computing and the idea of machine augmented intelligence. I originally started researching this project for use with a wearable computer.

I have been experimenting with an interface to view events using exhibit:

My apologies for the ugliness of the current website. It will get a makeover in the upcoming months.