Siddhartha Bhat

Experience

1. Avaamo (opens in a new tab)

   Jan 2024 — Present

   Machine Learning Engineer · Bengaluru, Karnataka

   Search and retrieval

   • –Built a document extraction pipeline across more than 20 PDF and HTML parsers, with section hierarchy kept through token-budget chunking. Prototyped LLM-based structure correction for malformed documents.
   • –Improved retrieval recall by 20% on identifier- and keyword-heavy queries with hybrid Milvus dense retrieval and Elasticsearch BM25, using weighted reciprocal rank fusion and denormalized attribute filtering.
   • –Added LLM-based query correction, NER-driven spell correction, and alternate-query generation, with alternate fan-out capped by retriever score before reranking.
   • –Built a text-to-SQL tool for PostgreSQL, MySQL, and CSV/Excel, with LLM-guided table filtering, foreign-key graphs, and generated table and column descriptions.

   Model serving and scaling

   • –Built a self-hosted vLLM service on Kubernetes for an FP8-quantized 27B model, exposed as an OpenAI-compatible streaming API. It handles 100K+ requests a day. Moving that traffic off a hosted API reduced LLM spend from about $7,000 to $1,000 a month, and batch-size and KV-cache tuning increased throughput per GPU by 67%.
   • –Moved reranker and embedding serving from ONNX Runtime to NVIDIA Triton, with gRPC connection pooling, dynamic batching, and separate priority for query and ingestion. Query latency fell by up to 40×.
   • –Added exact and semantic caching of generated answers, and a citations-only generation mode that cut output tokens by 60%. Concurrent identical requests are de-duplicated with Redis stream replay.
   • –Added KEDA autoscaling for Celery workers across four Kubernetes clusters, scaling from Redis queue depth.

   Evaluation

   • –Set up LLM-as-a-judge regression tests across customer deployments, checking correctness and semantic faithfulness. This cut manual QA effort by 70%.

   Contributions

   • –Streaming ASR over WebSocket using faster-whisper and Qwen3-ASR, with a wav2vec2 check to drop hallucinated transcripts during silence.
   • –PII detection covering 40+ entity types, moving from GLiNER to LLM-based extraction and adding Redis caching.

2. Samsung R&D Institute India

   Sep 2023 — Dec 2023

   PRISM Intern · Remote

   • –Fine-tuned CodeT5 for text-to-code generation on CVE security patches. CodeBLEU improved by 2% over the baseline. Built and cleaned a multi-task training dataset.
   • –Received a Certificate of Excellence, ranked among the top five projects in the internship.

3. IoTracX Pvt. Ltd.

   Jun 2023 — Aug 2023

   AI Intern · Manipal, Karnataka

   • –Built a YOLOv8 model for locating people in forest-fire zones, at 93.4% test accuracy on drone imagery. Deployed it as TensorFlow Lite on an NVIDIA Torizon device via Docker.

Education

B.Tech, Computer Science and Engineering · CGPA 8.81 / 10 · Manipal, Karnataka

Skills

Languages

PythonSQLC/C++

ML / AI

PyTorchHugging FacevLLMNVIDIA TritonONNX RuntimeAzure OpenAIVector searchReranking

Frameworks and data

FastAPIDjangoCeleryMilvusElasticsearchPostgreSQLMySQLRedisLangChain

Infrastructure

KubernetesDockerKEDAAWS (EC2, S3, SQS)PrometheusGrafanaCI/CDgRPC