Add KVIN ingestion benchmarks for CSV and JSON processing

[wmehling]

Changes

• Implement KvinIngestionCsvDiagnosticBenchmark for performance testing of CSV ingestion.
• Create KvinIngestionJsonDiagnosticBenchmark for benchmarking JSON ingestion.
• Introduce KvinIngestionWorkload to generate test data for benchmarks.
• Add unit tests for KvinIngestionCsvDiagnosticBenchmark and KvinIngestionWorkload.
• Remove obsolete KvinServiceBenchmark Scala file.
• Document benchmark setup and execution in markdown format

How to review

Look at doc kvin-ingestion.md and run benchmarks and see results.

Results from my tests (max values)

Overall

• putBatch: ~925 k tuples/s
• putCsvDirect: ~480 k tuples/s
• postCsv: ~415 k tuples/s
• postCsvSequentialFiles: ~371 k tuples/s
• postJson: ~127 k tuples/s

CSV (as known) significantly faster with some headroom to faster parse so we can achieve closer to raw speed of DB around 900 k tuples/s

CSV diagnostics

Steps building on each other with more and more of the actual postCSV pipeline:

• consumePrebuilt: ~69689 / ~59060 k tuples/s (consumes already-built tuples through a Blackhole → Iteration fast, no bottleneck)
• decodeCsvAndConsumeFields: ~1404 / ~1725 k tuples/s (adds raw CSV decoding with OpenCSV reader, tokenization, field string creation. Still very fast)
• parseCsvAndConsumeTuples: ~675 / ~621 k tuples/s (adds header mapping, trimming, type interpretation, and actual KvinTuple construction (still no LevelDB) -> Here quite some performance lost
• postCsvParseOnly: ~512 / ~602 k tuples/s (overhead for request / routing: adds real in-process Lift route and routing on top, with a sink that discards tuples (no persistence)

So would try to make the parsing and consuming faster in a seperate issue building on that branch.

[kenwenzel]

I propose to remove the use of reflection and to think about reworking the generation of tuples with more diversity in item, property and time.

[kenwenzel]

Benchmark                                       Mode  Cnt   Score    Error  Units
KvinIngestionBenchmark.postCsv                 thrpt    3  25.755 ± 21.888  ops/s
KvinIngestionBenchmark.postCsvSequentialFiles  thrpt    3  24.823 ± 16.835  ops/s
KvinIngestionBenchmark.postJson                thrpt    3   8.209 ±  2.571  ops/s
KvinIngestionBenchmark.putBatch                thrpt    3  57.127 ± 52.010  ops/s
KvinIngestionBenchmark.putCsvDirect            thrpt    3  34.088 ± 17.124  ops/s

[kenwenzel]

The JSON data was not sorted by item and property. Therefore, the items and property names were constantly repeated.

Benchmark                                       Mode  Cnt   Score    Error  Units
KvinIngestionBenchmark.postCsv                 thrpt    3  25.042 ±  4.808  ops/s
KvinIngestionBenchmark.postCsvSequentialFiles  thrpt    3  26.544 ±  4.818  ops/s
KvinIngestionBenchmark.postJson                thrpt    3  17.587 ±  9.703  ops/s
KvinIngestionBenchmark.putBatch                thrpt    3  52.496 ± 70.869  ops/s
KvinIngestionBenchmark.putCsvDirect            thrpt    3  29.174 ± 62.365  ops/s