Moogle!

A text search engine over a folder of .txt documents, built in C# (.NET 6) with a Blazor Server frontend. Given a query, it ranks documents by relevance using a TF-IDF vector space model, with support for query operators, proximity scoring, and fuzzy/similar-word matching.

What it does

• Indexes every .txt file under Content/ into a TF-IDF vector space (VectorSpace, Vector, WordInfo, Dictionary), splitting each document into overlapping pieces (MaxPieceSize = 100 words, half-overlap) so it can also return the most relevant snippet within a matched document.
• Parses the user's query (Moogle.Query) and expands it into a set of similar queries using near-matches from the indexed vocabulary (SelectBestQuery), so typos or close variants of a word still surface relevant results.
• Ranks documents with a combined score: cosine similarity in the vector space, multiplied by a per-snippet score, weighted by how similar the matched words are to the original query terms (via Math.Pow(1000, similarity) * IDF).
• Returns up to the top 10 results (TotalAnswerLimit), each with the source document name, the best-matching snippet, and a score, plus a suggested rewrite of the query (FixQueryWithNewWords) built from the best-matching indexed words.

Query operators

Implemented in Parsing.cs:

• !word, the word must not appear in the returned document.
• ^word, the word must appear in the returned document.
• word1 ~ word2, boosts the score the closer these words appear together in the document.
• *word, boosts the word's importance in the score; stacking multiple * increases the effect further.

Project structure

• MoogleEngine/, class library with the search logic:
  • Moogle.cs, entry point (Init() builds the index, Query() runs a search).
  • Libraries/, VectorSpace.cs, Vector.cs, Dictionary.cs, WordInfo.cs (TF-IDF model), Parsing.cs (query operators), SelectBestQuery.cs (similar-query expansion), TextPreprocessing.cs (tokenizing, file reading, snippet extraction), Request.cs (optional synonym lookup, currently commented out in Moogle.cs).
  • SearchItem.cs / SearchResult.cs, result data types.
• MoogleServer/, Blazor Server app that renders the search UI and calls into MoogleEngine.
• Content/, the folder of .txt documents to index and search over.
• files-script.py, a small utility that strips non-printable characters from every .txt file in Content/.

Tech stack

• C# targeting .NET 6.0
• Blazor Server (MoogleServer) for the web UI
• TF-IDF vector space model implemented from scratch (no external search/ML libraries)

Setup and usage

Requires the .NET 6 SDK.

From the repository root, on Linux (or WSL on Windows):

make dev

This runs dotnet watch run --project MoogleServer, which is also the direct command if you don't have make available:

dotnet watch run --project MoogleServer

Place the .txt documents you want to search over in Content/ before running. If any of them contain non-printable characters, files-script.py can be run beforehand to sanitize them.