ScaleDB (Key-Value Database)

A Redis like in Scala

ScaleDB is a persistent, in‑memory key‑value store inspired by Redis and developed in Scala. It implements a textual protocol close to RESP (Redis Serialization Protocol), support for time‑to‑live (TTL) and an extensible type system. The project includes a network server with connection handling, concurrent command execution and automatic snapshot persistence.

Features

• Core data types: IntD, DoubleD, StringD, ListD (heterogeneous but restricted to a single element type per list)
• Extensible type system via DataType trait and DataTypeObject companion
• In‑memory database with TTL expiration
• Textual command protocol (RESP‑like)
• Commands: SET, GET, DEL, EXISTS, EXPIRE, TTL, INCR
• Database loader that replays a command log to restore state
• Network server accepting TCP connections and dispatching commands
• Concurrency control with read/write locks on the database
• Automatic snapshot saving (periodic and on graceful shutdown)

Architecture

classDiagram
    namespace core {
        class Database {
            <<case class>>
            -data: Map[String, DbEntry]
            +get(key: String): Option[DbEntry]
            +saveSnapshot(filePath: String): Unit
        }
        class DbEntry {
            <<case class>>
            +data: DataType
            +ttl: Option[DateTime]
            +toBytes(): Array[Byte]
        }
        class DatabaseLoader {
            <<case class>>
            +filePath: String
            +getDb(): Result[Database, DbErr]
        }
    }

    namespace types {
        class DataType {
            <<trait>>
            +toBytes(): Array[Byte]
            +increment(other: DataType): Result[DataType, DbErr]
        }
        class DataTypeObject {
            <<trait>>
            +Repr: Byte
            +fromBytes(bytes: Array[Byte]): Result[DataType, DbErr]
        }
    }

    namespace commands {
        class Command {
            <<trait>>
            +execute(db: Database): Array[Byte]
        }
        class CommandDef {
            <<trait>>
            +decode(bytes: Array[Byte]): Result[Command, DbErr]
        }
    }

    namespace parser {
        class Parser {
            <<trait>>
            +parse(input: Iterator[String]): Result[Database, DbErr]
        }
        class RESP {
            <<object>>
            +parse(input: Iterator[String]): Result[Database, DbErr]
        }
    }

    namespace errors {
        class Result {
            <<enumeration>>
            +Ok(A)
            +Err(E)
        }
        class DbErr {
            <<enumeration>>
            +FileParser(String)
            +KeywordParser(String)
            +ValueParser(String)
        }
    }

    DatabaseLoader --> Parser : Depends from abstraction
    RESP ..|> Parser : Implement the abstraction
    Database *-- DbEntry : Compose
    DbEntry *-- DataType : Compose
    Command ..> Database : Execute effects
    CommandDef ..> Result : Use
    Parser ..> DataTypeObject : Use for deserialization
    DataTypeObject ..> DataType : Create

Loading

Data Types

All stored values implement the DataType trait which provides:

• toString : human‑readable representation
• toBytes : binary serialisation (starts with a type marker)
• increment : generic addition used by INCR (polymorphic) Each type has a companion DataTypeObject that carries the byte marker ('i', 'd', 's', 'l') and deserialisation logic. Available types: IntD (32‑bit integer), DoubleD (IEEE 754 double), StringD (UTF‑8 string), ListD (list of homogeneous type).

Database

The database is a mutable Map[String, DbEntry] where each entry holds a DataType value and an optional expiration timestamp. On every access the TTL is checked; expired entries are automatically removed. A DatabaseLoader can populate the database from a file containing a sequence of command lines (the same textual protocol).

Commands

Every command is a singleton object with:

• isMatching(bytes) - detects whether a received byte array corresponds to this command
• decode(bytes) - parses the message into a command instance
• execute(db) - applies the command to the database and returns the response as a byte array

Server Overview

The server layer is organised under server/ and consists of:

• Server.scala - main entry point, binds to a TCP port and accepts connections.
• ConnectionHandler.scala - manages a single client session (read/write loop).
• RequestReader.scala - accumulates bytes and splits messages on \r\n.
• CommandDispatcher.scala / CommandRegistry.scala - detect and instantiate the correct command object.
• DatabaseLock.scala / LockMode.scala - provide fine‑grained read/write locking.
• Protocol.scala - constants and helpers for the textual protocol.
• SnapshotScheduler.scala - triggers periodic and shutdown snapshot creation.

Concurrency is handled through a read‑write lock that protects the underlying mutable database; read operations acquire a shared lock, write operations an exclusive lock.

Supported Commands

The communication protocol is line‑oriented, with \r\n line separators. The first line is the command name in uppercase, followed by arguments.

Command	Format	Example	Success response
SET	SET\r\n<key>\r\n<type>\r\n<value>\r\n	SET\r\nmyKey\r\ni\r\n-90\r\n	"OK"
GET	GET\r\n<key>\r\n	GET\r\nmyKey\r\n	binary encoded value
DEL	DEL\r\n<key>\r\n	DEL\r\nmyKey\r\n	binary encoded deleted value
EXISTS	EXISTS\r\n<key1>\r\n<key2>...\r\n	EXISTS\r\nk1\r\nk2\r\n	"true;false;"
EXPIRE	EXPIRE\r\n<key>\r\n<seconds>\r\n	EXPIRE\r\nmyKey\r\n15\r\n	"OK"
TTL	TTL\r\n<key>\r\n	TTL\r\nmyKey\r\n	"0 days, 0 hours, … 30 seconds"
INCR	INCR\r\n<key>\r\n[<type>\r\n<value>]	INCR\r\nmyKey\r\ni\r\n-90\r\n	"OK"

INCR is polymorphic:

• For IntD/DoubleD it performs arithmetic addition.
• For StringD it concatenates the string representation of the increment.
• For ListD it appends the element (type must match list element type). If the key does not exist, it is created using the provided value (or with IntD(1) if no value is given).

Type Serialisation (binary)

Each type stores its byte marker followed by the data:

• IntD : 'i' + 4 bytes big‑endian integer
• DoubleD : 'd' + 8 bytes IEEE 754 double
• StringD : 's' + 4 bytes length + UTF‑8 bytes
• ListD : 'l' + element type marker + "\r\n", then each element serialised with "\r\n" separators, and finally "\rl\n\r\n"

Snapshot Persistence

The database can be saved to a snapshot file that is a log of SET (and EXPIRE) commands, re‑playable on startup.

• Snapshots are written atomically (temp file + rename).
• Triggered by a configurable timer (e.g., every 5 minutes).
• Also performed on graceful shutdown (SIGTERM) via a shutdown hook.
• Loading at startup is done by the existing DatabaseLoader.

Building & Running

Requires Scala (2.13+ or 3.x) and sbt.

# compile
sbt compile

# run the server (default port 6379)
sbt run

# run tests
sbt test

ScaleDB is a work‑in‑progress (personal) educational/experimental project inspired by Redis.