Blog

Published by Alex Slotnick on May 4, 2017 10:16:57 AM

Percona Live 2017 Replay: Baron Schwartz's Keynote Address

Last week, Percona Live 2017 gave us the chance to meet with our friends and peers at the best open-source database conference in the world. Hundreds of thought leaders and experts came together to discuss cutting edge ideas in our industry, demonstrating how rapidly monitoring solutions and data technologies continue to evolve.

Read More
Published by Alex Slotnick on Nov 20, 2015 5:01:57 PM

The Value of Log Structured Merge Trees

Indexes are usually built by way of a data structure; typically, that structure takes the form of a “tree.” Most commonly, the structure of choice is a B-Tree, which is a hierarchical organization defined by the arrangement and interactions of its nodes.

Read More
Published by Alex Slotnick on Sep 9, 2015 11:01:48 AM

VividCortex on Tech.Co: Can Your Business Benefit From a Cloud Database?

Should you move your database to the cloud? Over on Tech.Co, VividCortex's CEO and founder Baron Schwartz has published an article that covers some of the major benefits of such a shift. Cloud computing has changed tremendously over the past decade, and the potential for databases in the cloud is only growing. Baron's article explains what some of those changes might mean for you and your data.

Read More
Published by Baron Schwartz on May 26, 2015 6:31:00 AM

How We Ensure VividCortex Never Loses Data

Adrian Cockcroft really nailed it when he said that a monitoring system has to be more reliable than what it’s monitoring. I don’t mind admitting that in our first year or so, we had some troubles with losing telemetry data. Customers were never sure whether their systems were offline, the agents were down, or we were not collecting the data. Even a few seconds of missing data is glaringly obvious when you have 1-second resolution data. There’s nowhere to hide.

Read More
Published by Baron Schwartz on Feb 24, 2015 3:16:00 AM

Schemaless Databases Don't Exist

There’s no such thing as a schemaless database. I know, lots of people want a schemaless database, and lots of companies are promoting their products as schemaless DBMSs. And schemaless DBMSs exist. But schemaless databases are mythical beasts because there is always a schema somewhere. Usually in multiple places, which I will later claim is what causes grief.

Read More
Published by Baron Schwartz on Jan 27, 2015 6:18:00 AM

How VividCortex's Agents Manage Logs

It can be scary to run agents on your critical servers. Misbehaving agent software can cause harm, including pegging your CPUs, filling up your disks, or eating all your memory and making your server swap. Fortunately, at VividCortex we have many years of experience with these problems and we designed our agents to avoid them from day one.

Our agents are self-limiting in every aspect of resource consumption, including log files. Unlike some software you may have experience with, VividCortex’s agents won’t fill your disks with logs or temp files, nor will they cause a lot of I/O. We ensure this through a variety of measures, but there are three basic techniques I’d like to mention here.

  1. The agents don’t generate or keep spools of data. We don’t spool metrics to disk, for example, nor do we create temporary files or other caches. Our metrics aggregator agent will retry failed transmissions up to 5 times by default, but that represents a very minimal amount of memory and doesn’t impact disk at all.
  2. We only log minimal information. It’s much better to avoid logging than to deal with the consequences of verbose logging. In normal operation, VividCortex’s agents will only log a periodic indication that they’re successfully communicating with our APIs. It’s basically a liveness heartbeat.
  3. The log messages that the agents do write won’t grow indefinitely. They essentially have a built-in logrotate mechanism. They will truncate and expire their own logs. This protects you against filling up your disks.

This last bit may sound silly – why don’t we just use standard logrotate, instead? – but it’s actually very important. My experience as a consultant taught me that relying on system-provided functionality is a path to madness, because facilities like logrotate have bugs and undesired behaviors in a variety of circumstances. Furthermore, many of these might be triggered by nonstandard system configurations out of our ability to anticipate, detect, and control. For these reasons we ship a known good, clean implementation of everything we need (except for certain core system libraries), baked right into our agents.

Our agents are fully configurable, so all of the above are subject to tweaking and tuning if customers want. However, out of the box, our agents are configured to “just work” and be very respectful of system resources. One of the fastest ways to kill a system is to fill up its disk or cause a lot of I/O. High-performance servers, in particular, often run very close to a performance cliff, a delicate state where just a little bad behavior can cause the whole thing to tip over and die or freeze.

And how do we know that our agents don’t cause these problems? Well, because we capture per-process performance data in 1-second resolution – including CPU, memory, network, disk and more – if there’s a problem, we can see it. And because our Adaptive Fault Detection technology detects micro-stalls as short as 1 second long, we also know when a badly-behaved program impacts others. And we do see this, all the time – with other software, and sometimes even with other monitoring software. But not with our agents.

Sign up for your free trial and experience the benefits of VividCortex for yourself!

Read More

Recent Posts

Posts by Topic

see all