Tag

logging Archives - AWS Managed Services by Anchor

Slick log aggregation with Kibana

By | Technical | No Comments

Log aggregation is awesome-cool, you already know this. There’s no shortage of solutions for doing this, and you’ve probably heard of a few like Splunk, Graylog, Logstash, Scribe. There’s plenty more that integrate and gather data in various ways, and you can always can roll your own. Having all your logs in one place makes it much easier to spot patterns and perform analysis. This is particularly useful when you have a cluster of machines doing roughly the same thing, as it makes anomalies more obvious and less prone to noise from false positives. One of our devs was itching for a frontend tie into the service-oriented architecture stuff that we’re building at the moment, particularly one that could be easily hacked up and modified, so he went looking and…

Read More

Interesting failure modes, episode 2501

By | Technical | No Comments

I got woken up by a SMS for low diskspace the other night on one of our customer’s servers. Okay, so that’s a lie, I never sleep, but the SMS is real. Oh great, they’re making whoopie on their mailing lists again and making some stupidly huge logfile. Little did I know just how huge that file was. How about 735gb huge, in the space of 12hrs? This customer is already a bit of an oddball, what with 1.4TiB of usable space in their server. “Oh that’s nothing”, you say. Sure, I’ve got a few TiB of kitten pictures on my machine at home, just like you, but to put things in perspective: 300GiB of space would be “big” for most Anchor customers. SCSI disks cost about $1.70/Gb, compared to…

Read More