readme

Signed-off-by: Jason Hall <imjasonh@gmail.com>

README.md

@@ -0,0 +1,38 @@
+# `gcsbuf`
+
+This Go library gives you an `io.Writer` to streamily write an object to [Google Cloud Storage](https://cloud.google.com/storage), and at the same time, streamily read that same object using an `io.Reader`.
+
+### Writing
+
+Writing works by buffering the input data and periodically flushing updates to GCS. Object updates are implemented using [composite objects](https://docs.cloud.google.com/storage/docs/composing-objects). When appending a new chunk of data, that data is first uploaded to a new temp object, then composed with the existing object contents over itself.
+
+When the object reaches the max number of compositions, the count is reset by copying and then renaming the object back to its original location -- this is purely a metadata operation and does not require the client to read or write all the data again.
+
+When the writer calls `Close()`, the object is finalized by updating object metadata, to tell consumers that the object is done being written.
+
+Each compose/append and copy/reset operation is guarded with a precondition, so that the operations fail if the object changed due to outside operations, preventing silent corruption.
+
+You can configure the flush timeout (default: 5s) and buffer size (default: 4MB), and the writer will flush when either of those conditions are met.
+
+```go
+w := gcsbuf.NewWriter(...)
+fmt.Fprintln(w, "hello")
+_ = w.Flush() // immediately flush to GCS
+fmt.Fprintln(w, "world")
+_ = w.Close() // flush and finalize
+````
+
+### Reading
+
+Reading works by iteratively making [HTTP Range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests) to the object and polling until either newly appended data is found, or until the object is marked as finalized.
+
+```go
+r := gcsbuf.NewReader(...)
+_, _ = io.Copy(os.Stdout, r) // reads until the object is finalized
+```
+
+In the example above, if the writer is still writing data, `io.Copy` will block until the writer completes, and receive new content each time it flushes.
+
+### Background
+
+I originally used a ~hack~ scheme like this when creating [Google Cloud Build](https://cloud.google.com/build) to efficiently write logs to durable storage, and to allow clients like `gcloud` to simultaneously stream the logs objects to users until build completion. Logs were separately written to Google Cloud Logging, which didn't support a fast logs tailing API at the time.