Stash: A Go Package for Disk-based Blob Cache

Web services like Amazon S3 or Google Cloud Storage make file storage so trivial that you can’t help but fall in love with them.

Although, they make your life easier when it comes to storage, for most real applications you still need to transfer files back and forth between the storage service and your application.

When designing Toph’s architecture, we took a very no-excuse approach to saving files. We save everything file-like to Amazon S3.

This means, whenever you try to take a look at something (like one of your recent submissions), we need to fetch the relevant files back from Amazon S3. And, that, not only adds to the response time, it also makes unnecessary API calls to Amazon Web Services.

And, that is why we have made Stash!

Using Stash

Stash is a Go package that sits between Toph and the external storage service and caches the most recently used files.

You can download Stash using go get:

go get gopkg.in/stash.v1

Using Stash is as simple as it gets:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

package main

import "gopkg.in/stash.v1"

func main() {
    // Create a cache with a storage of 2MB and 10 files.
    c, _ := stash.New("/tmp/stash", 204800, 10)

    c.Put("hello", []byte("Hello, world!\n"))

    rc, _ := c.Get("hello")

    io.ReadAll(rc) // => []byte("Hello, world!\n")

    rc.Close()
}

Stash ensures that it will store at most a predefined amount of bytes and number of files. If necessary, it will evict the least recently used files from the storage.

Future Plan

We plan to improve Stash over time and add some features that we think are essential and will make the package more useful. We do want to make sure that the package is as simple, and doesn’t try to do a lot.

Some of the things that we intend to do soon are:

• Make eviction policy configurable
• Allow cache instances to persist across sessions (issue)
• Add more tests
• Improve the documentation

Contributions

You can send a pull request addressing one of the open issues or make an issue yourself to discuss new features–contributions are always welcome.

For our other open source contributions, you can check out Furqan Software on GitHub.