Tuesday 18 September 2007

Storing monitoring data in a database? no problem!

Ganglia is a monitoring system that uses RRDTool for its storage and graphs. This provides an excellent solution for monitoring, but suffers from data becoming less detailed ("averaged out") when you look further back in time. This is deliberate, but does make later analysis of the data difficult.

If you wanted to keep detailed records of monitoring data with MonAMI that don't degrade over time, now you can, I've committed changes to the mysql plugin in CVS. In addition to monitoring a MySQL database, the plugin can now store information. You tell it which table and how to map the information into that table and it does the rest, it'll even create the table if it doesn't exist.

2 comments:

YHT said...

Hi Paul,
I want to collect new metric for ganglia, fortunately i found your project.
You said that MonAMI can stored data in a database right? so if it store monitor data in database, how can ganglia access it? I mean what i should change in ganglia, can you help me the way?
Another thing is if i use MonAMI to monitor all of processes, will it increase network load? how can i deal with real-time monitoring problem?
Best regards,
--Thanh
thanh.phamhong[at]niithoasen.com

Paul Millar said...

Hi Thanh,

First the database question. It's perhaps easiest to think of what MonAMI does as being in two parts: first getting the data, then storing it somewhere. MonAMI can store the data it gathers in multiple places; for example, to store the data in a MySQL table and to send it to Ganglia. So, MonAMI doesn't act as a bridge between MySQL and Ganglia, the two data stores are independent.

Network load: yes it is inevitable that sending monitoring information to Ganglia will increase the network load. If you're storing data in a MySQL server running on a remote machine then that, too, will increase the network load. However, these effects should be small; on modern networks (100baseT or faster) you shouldn't notice it.

(I guess if you monitored a large enough number of metrics you might notice an effect, but I've not noticed any problems when monitoring many hundred metrics)

BTW, the new tutorial has a section on storing data in MySQL. It's available from:
http://monami.sourceforge.net/about.html

HTH,

Paul.