Skip to main content

What is Bigtable?

· 2 min read
Sanjeev Sarda
Sanjeev Sarda
High Performance Developer

What is Bigtable?

Bigtable

What is Bigtable?​

It's a storage system for structured data that runs in a distributed fashion. It was developed at Google pre 2006, the original paper can be found here.

A single master distributed data store for structured data.

Data Model​

Bigtable supports sparse rows, is persistent and sorted. Rows are encoded as strings, with the table being ordered by the key.

public class Row<T> {
    Map<String, StringEncodable<T>> rowElements;
}

Row level writes and reads are atomic, Bigtable only supports row level transactions. Each row can be made up of multiple columns, with a logical grouping of columns (of the same type) being referred to as a column family.

public class ColumnFamily<T> {
    Set<Row<T>> group; 
}

The table is horizontally sharded with the unit of sharding being referred to as a tablet in the original 2006 paper.

Each item can be accessed using a key:

public class Key {
    String row;
    String column;
    long time;
}

The timestamp field allows us to have multiple values over time - the system maintains timestamped versions for rollback. The values can be periodically garbage collected, or a maximum number of versions can be configured.

What Other Systems is Bigtable Dependent Upon?​

GFS​

Under the hood, Bigtable uses GFS (Google File System) for storing log and data files. Data is stored in SSTable format which is basically an immutable map. SSTable format is block oriented with a standard 64kb block size and can be indexed in memory.

Chubby​

Bigtable also uses a distributed lock system (to make sure there is only one leader for example) called Chubby which uses the Paxos algorithm for concensus.

When client sessions drop, the file backed lock is released (all locks are file backed in Chubby providing a natural namespace for locks). Chubby also provides callbacks on lock state changes or session expiry.