Notes from the 2021 paper from Meta/Facebook Research on log structured protocol usage in their Delos system (a distributed database used as part of Facebook's control plane layer). Also covers replicated state machines.

Notes from the classic 1987 paper from Birman and Joseph on Virtual Synchrony, a computation model for distributed systems.

Notes and ideas on annotations and LLMs. Using annotations in conjunction with LLM dev tooling as well as generating annotation processors with LLMs

I never write about caches and caching, so I thought I'd cover some basics on LLM caching. Covers inference and prompt caching.

Notes on the TensorZero LLM gateway. Covers templates, schemas, feedback, retries, evals, DICL, MIPRO, model-prompt-inference optimization.

Some basics on Ollama. Includes some details on quantization, vector DBs, model storage, model format and modelfiles.

Comparisons of the OpenAI service offering with that of Anthropic. Includes context window, rate limits and model optimization.

My notes on the design of Anthropic's APIs and some general design considerations for provider based APIs and SDKs. Covers rate limiting, service tiers, SSE flow and some of the REST API endpoints.

Notes on the evolution of Java Unsafe and off heap memory access - touches on Unsafe, FFM API and the Agrona DirectBuffer.

Some notes on false sharing and cache line padding.