notifier

A Go server that fans a realtime event stream out to many WebSocket clients, with a gRPC ingest side and Prometheus instrumentation throughout.

This is the Go fan-out half. The Java pipeline that feeds it (Twitter → Kafka → gRPC) lives in twitter-kafka-streaming.

producer ──gRPC (Receive)──►  notifier gRPC :6565
                                   │  fan-out
                                   ▼
                              subscribers
                                   │
                      grpc-gateway + wsproxy :8081
                                   ▼
                            WebSocket clients

Design

• gRPC server (cmd/) — registers TweetService with two streams: Receive (ingest) and Stream (subscribe). Prometheus stream/unary interceptors and a /metrics endpoint on :9100.
• Fan-out core (internal/notify) — Connection keeps a slice of subscriber channels behind a mutex. In() drops each packet on a buffered data channel; a pool of ~20 notify goroutines drains it and delivers a copy to every subscriber channel, so one slow client can't block ingest.
• Gateway (gateway/) — grpc-gateway translates the gRPC stream to HTTP, wrapped in a WebSocket proxy so browsers can subscribe over ws://. Handlers are wrapped in a Prometheus instrumentation chain (in-flight, count, duration, response size).
• The protobuf contract is shared with twitter-kafka-streaming.

Run

# 1. protobuf stubs (contract shared with twitter-kafka-streaming)
protoc -I proto/ proto/notifier/**/*.proto --go_out=plugins=grpc:grpc

# 2. gRPC server on :6565 (+ metrics on :9100)
go run ./cmd

# 3. gateway on :8081 (HTTP/WebSocket)
go run ./gateway/cmd

Stack

Go 1.13 · gRPC 1.27 · grpc-gateway · gorilla/websocket · Prometheus client

---

Built 2020 to learn gRPC streaming, grpc-gateway, and channel-based fan-out in Go. Not maintained.