Tag

kyoto cabinet Archives - AWS Managed Services by Anchor

Second strike with Lightning!

By | Technical | 2 Comments

We put Kyoto Cabinet under the gun recently as a means to improve Redis. The Anchor Propulsion/Internet Laboratory validated Kyoto Cabinet as “fresh”, but extended live testing has revealed sub-optimal behaviour in some situations. To recap, we used Kyoto Cabinet to give Redis near-realtime disk persistence with a greatly reduced memory footprint; we called this “NDS” and published the code. Dirty keys are flushed from memory periodically into Kyoto Cabinet’s backing store. This works fantastically most of the time, but we’ve discovered that some operations cause a massive blowout in the on-disk files. Kyoto Cabinet is a key-value store. When you update a value in Kyoto Cabinet it can be rewritten in-place, unless the new value is longer than the old one. In this case, Kyoto Cabinet makes a new…

Read More

Redis Rethought: Exciting Extremes with Larger-Than-Memory Datasets

By | Technical | 2 Comments

In a recent post we talked about Redeye, a way to manage a cluster of Redis shards. Sharding is important for pure storage capacity, but also to manage availability and performance, seeing as Redis can be CPU bound in a suffuciently large environment. This time we’re tackling the other side of the equation, simply being able to hold all your data without dropping it. You’ll recall that Redis is purely an in-memory datastore. This makes it amazingly fast, but volatile. To get around this you can have persistence across restarts using RDB and AOF files. The catch is that you’re ultimately limited by the amount of RAM you have available – if you have 500gb of data to store then you need 500gb of RAM, either in a single server…

Read More