Last week, we introduced VividCortex support for Redis. One of the reasons we expanded our product capabilities is because of our own use of Redis. According to Baron, “We rely on Redis to help analyze the massive amounts of time-series data we receive from agents running on customer systems.” Let’s go a little deeper…
VividCortex keeps tables with all the known metrics and queries for many hosts measured at 1-second granularity. We use Redis to avoid repeating the “insert on duplicate update” into the metric/query tables. We are able to do this by keeping hourly buckets of seen queries and metrics. When a batch of metrics/queries arrive, before insert/updating them in the mysql tables, we check that we haven’t already seen them during the current and previous hours.
Essentially, the data we put into Redis is a compact boolean 1 hour resolution timeseries of existing queries and metrics. It is trivial to get the union of all buckets within a time range and use that to quickly tell which metrics are active within that timespan. We use that for example, to tell if a query had warnings/errors/etc within a timespan.
Redis has helped us conserve resources, and now that we are using VividCortex to monitor our Redis instances, they are running even more efficiently.