IDMS — Intelligent Document Management System
Production-grade async backend with vector search.
The Problem
Document management systems typically offer basic CRUD and file storage. They don't understand document content — searching means keyword matching, not semantic understanding. For a university capstone, I wanted to build a system that demonstrates production-grade engineering practices while adding vector search capabilities.
The Approach
I designed IDMS as an async Python backend with clean service boundaries and a vector search layer, built with the engineering practices I'd want in a production system.
Key architectural decisions
Async from the ground up — FastAPI with asyncpg for non-blocking database operations. The entire I/O path is async.
Qdrant for vector search instead of pgvector — deliberately chose a dedicated vector database to demonstrate a different approach from the Document Q&A SaaS. Qdrant gives dedicated indexing, filtering, and scaling for vector operations.
5-service Docker Compose stack — PostgreSQL, Redis, Qdrant, FastAPI, React. Each service has a clear responsibility and clean interfaces.
80% test coverage enforced via CI — not aspirational, enforced. GitHub Actions pipeline rejects PRs below the threshold.
Technical Deep-Dive
Authentication & security
- ▸JWT auth with bcrypt password hashing
- ▸Timing-safe login (constant-time comparison prevents timing attacks on password verification)
- ▸RBAC with role-based endpoint protection
- ▸PDF upload with magic-byte validation (checks actual file content, not just extension)
Data integrity
- ▸Cascading deletion across three systems: when a document is deleted, its PostgreSQL record, FileStorage file, AND Qdrant vectors are all removed in a single transaction-like operation
- ▸Repository pattern for data access — controllers never touch the database directly
Testing
- ▸20 test files covering auth flows, document CRUD, repository layers, and edge cases
- ▸1,499 lines of test code
- ▸Both unit tests (isolated repository methods) and integration tests (full API request → database → response)
- ▸80% coverage enforced via GitHub Actions — every push triggers the full test suite
Key Metrics
Challenges & Solutions
Challenge 1:Cascading deletion across systems
Deleting a document requires removing data from PostgreSQL, FileStorage, and Qdrant. If any step fails, you get orphaned data. Fix: Implemented ordered deletion with error handling — PostgreSQL first (source of truth), then FileStorage, then Qdrant. If a downstream deletion fails, it's logged for manual cleanup rather than rolling back the entire operation.
Challenge 2:Magic-byte validation
Users could upload malicious files with .pdf extensions. Checking the extension isn't enough. Fix: Read the first bytes of the uploaded file and verify the PDF magic bytes (%PDF-) before accepting the upload.
Challenge 3:Test coverage as a gate
Getting to 80% required testing auth flows, error paths, and edge cases that are easy to skip. Fix: Wrote a coverage configuration that measures branch coverage, not just line coverage. The CI pipeline fails if coverage drops below 80%.
Lessons Learned
Enforced coverage is different from aspirational coverage. When CI rejects your PR for dropping below 80%, you actually write the tests. When it's a goal in a README, you don't.
Async Python is worth it for I/O-heavy backends. The performance difference on concurrent requests is significant compared to sync Django/Flask.
Repository pattern pays off at scale. When you need to swap Qdrant for Milvus, you change one file instead of every controller.