Work & Research

Engineering work across production AI agents, applied research, and intelligent systems.

A curated view of company systems, open-source products, research programs, publications, and patent-backed engineering.

Open-source productAI Interview
Research venuesETRA / ICPR
Applied IP15 patents
FocusAgents + ML
Product Engineering

Products and systems built end to end.

AI systems spanning real-time interaction, agent orchestration, multimodal understanding, and practical user workflows.

Private projectAI educationLearning tools

ByteMentor

An AI education platform focused on personalized technical learning support, structured practice, and mentor-style guidance for programming concepts.

Company workFull stackAgent infrastructureMultimodal AI

AI Agent Platform for Creative Workflows at Kyoso ↗

At Kyoso, built and advanced a full-stack production platform for autonomous creative work. Worked across the agent runtime, backend services, tool execution, state and context, brand data, canvas behavior, and product-facing multimodal workflows, with a focus on orchestration, subagents, custom models, memory, reliability, latency, personalization, and cost.

CollaborationPublic repoPresentations

Slideo AI

A collaborative AI presentation-generation project that turns documents into slide decks with document parsing, AI content planning, real-time collaboration, and PowerPoint export workflows.

Agent Systems

Full-stack agent systems and applied AI infrastructure.

Selected company and research work covering production orchestration, tool use, long-context memory, research, and workflow automation.

PrivateLLM agentsPatent-linked

System-Level LLM Agents

Led LLM agent development for system applications such as intelligent image editing and color recommendation tools, combining open models, tool use, and user-context understanding.

Company workAgent memoryEvaluations

Hierarchical Conversation Memory

Designed a tiered summary tree for long-running agents. Summaries act as an index over older turns, allowing the agent to navigate multiple levels and recover evidence from original conversations without loading the full history. The architecture supports conversations spanning thousands of turns while testing rollup correctness, summary faithfulness, and end-to-end retrieval separately.

Agent infrastructureVirtual filesystemSandboxing

File-Backed Agent Workspaces

Designed virtual filesystems as an external workspace for agents. Large tool outputs are stored in files to keep prompts compact; agents use file references and targeted search or reads to recover details only when needed. The workspace also persists plans and progress, carries artifacts and findings between subagents and the main agent, and stores evolving user notes or brand knowledge. Explicit read, write, list, and search tools operate inside sandboxed permission boundaries that control accessible paths, allowed operations, and approval requirements.

Distributed systemsAgent runtimeReliability

Stateless and Resumable Agent Execution

Designed horizontally scalable agent services around stateless workers and externally persisted execution state. Tasks checkpoint conversation state, progress, and tool results at meaningful transitions; atomic queue claims, leases, coordination locks, and idempotent steps prevent duplicate execution. A fast in-memory store carries locks, heartbeats, and interruption signals, while persistent cloud storage provides the durable source of truth needed to resume after failure, move work between workers, or apply user updates to an active task.

Agent infrastructureEvaluationReliability

Agent Evaluation Framework

Designed an agent evaluation framework centered on calibrated LLM/VLM judges. Approved real scenarios seed reviewed synthetic reference packages; blinded relative grading compares candidate outputs, tool use, traces, and context against grounded expectations before the results become regression signals.

Company workRuntime migrationDeep Agents

Claude Agent SDK to LangChain Deep Agents

Planned a behavior-preserving migration around ten explicit contracts spanning turn execution, history, usage, production limits, multimodal interaction, documents, user context, subagents, evaluation, and cutover. Reused the transport-neutral tool pipeline, removed SDK-specific assumptions, and sequenced the work around an observable integration gate.

Agent infrastructureCLI toolsOptimization

CLI-First Agent Tooling

Evaluated MCP-style integrations against CLI-based execution for equivalent tasks. Where command-line interfaces fit the workflow, smaller schemas and outputs reduced context use and cost to approximately one-quarter, with a similar improvement in latency. Multi-command composition also allowed complex work to run in one execution rather than across repeated reasoning and tool-call cycles.

PrivateDeep researchRAG

Deep Research Agent

Designed a RAG-based research agent for report creation with smart editing, multi-source merging, automated visualization, and model routing across reasoning and generation models.

PrivateBrowser automationWorkflow agents

Workflow Automation Agent

Built an LLM-based browser automation agent that interprets recorded workflows, reasons over HTML, identifies target elements, and executes actions across websites.

PrivateFunction callingPatent-linked

Gesture Recognition With LLM Function Calling

Integrated gesture recognition models with LLM function calling to interpret stylus gestures and map them to application actions in creative tablet workflows.

Healthcare ML And Devices

Applied machine learning for medical and wearable systems.

Healthcare work includes wearable biomarker modeling, medical imaging, firmware-adjacent systems, and patent-backed device pipelines.

PrivatePatent-linkedWearables

Cardioid Algorithm ML

Machine learning work for heart-risk and renal-risk monitoring using smartwatch biomarker data at General Prognostics.

PrivateSampling algorithmsHealthcare

Smart Sampling

Algorithm work for efficient sampling strategies in medical and wearable data pipelines.

PrivateFirmwareWearables

Watch Firmware

Firmware-adjacent work supporting smartwatch-based sensing and health-data collection workflows.

Public repoMedical imagingCNN

RetiSpec Project

CNN-based binary classification using RGB and near-infrared image inputs, with YACS configuration, TensorBoard logging, and evaluation artifacts.

Case studyPatent-linkedMedical devices

IoT-Enabled HbA1c Medical Device

Robotic and IoT-enabled medical device work for remote HbA1c testing from finger-prick blood samples, including embedded and mobile integration.

Public repoPaper-linkedRetinal imaging

Diabetic Retinopathy

Code related to exudate segmentation in retinal fundus images for diabetic retinopathy detection.

Computer Vision And HCI

Eye tracking, gaze estimation, and visual perception projects.

Computer vision work spanning VR eye tracking, pupil estimation, retinal imaging, scene perception, and human-computer interaction.

Publication-linkedPatent-linkedEye tracking

Advanced Computer Vision for Gaze Tracking

Research and deployment of gaze tracking methods for VR, desktop, and mobile settings, including RGB/IR camera pipelines and accuracy improvements.

Public repoPupil centerCNN

Pupil Center Estimation

CNN architecture for estimating pupil centers from infrared smartphone eye images, including image enhancement experiments and mean pixel error analysis.

Public repoLane detectionSegmentation

Lane Detection

Semantic-segmentation approach to lane detection using BDD100K images, generated labels, model outputs, and feature-map visualizations.

Public repoComputer visionClassification

Emotion Classification

Computer-vision project for emotion classification experiments and model evaluation.

Formal Evidence

Papers, thesis, and patents.

Selected publications and patent-backed systems connected to the project areas above.

Conference
CSA-CNN: A Contrastive Self-Attention Neural Network for Pupil Segmentation in Eye Gaze Tracking
Soumil Chugh, Juntao Ye, Yuqi Fu, Moshe Eizenman | Symposium on Eye Tracking Research and Applications
View paper
Conference
Detection and Correspondence Matching of Corneal Reflections for Eye Tracking Using Deep Learning
S. Chugh, B. Brousseau, J. Rose, M. Eizenman | ICPR 2021
View paper
Thesis
An Eye Tracking System for a Virtual Reality Headset
MASc thesis | University of Toronto | 2020
View thesis
Journal
Exudates Segmentation in Retinal Fundus Images for the Detection of Diabetic Retinopathy
Soumil Chugh, Jaskirat Kaur, Deepti Mittal | IJERT | 2014
View paper
Conference
Low Cost Calibration Free Pulse Oximeter
Soumil Chugh, Jaskirat Kaur | IEEE INDICON | 2015
View paper
Conference
Non-invasive Hemoglobin Monitoring Device
Soumil Chugh, Jaskirat Kaur | ICCC India | 2015
View paper
Conference
Effect of Different Signal Processing Techniques on a Calibration-Free Pulse Oximeter
S. Chugh, A. Akula | I2CT | 2018
View paper
Patent
Multi-modal interaction for selecting semantic regions in agent-based image editing
Yu Zhao, Abby Lu, Soumil Chugh, Che Yan, Yuan Deng | Submitted
Submitted
Patent
Methods for cross-media configuration on virtual keyboard theme
Yu Zhao, Yuan Deng, Soumil Chugh, Che Yan, Wei Li | Submitted
Submitted
Patent
A method for cursor positioning with gaze-assisted pen interaction
Juntao Ye, Shuyang Teng, Soumil Chugh, Andy Chow, Yu Zhao | Submitted
Submitted
Patent
Methods and systems for multi-camera corneal reflection gaze system
Soumil Chugh, Juntao Ye, Moshe Eizenman | Submitted
Submitted
Patent
Methods and Systems for Gaze Tracking using One Corneal Reflection
Soumil Chugh, Juntao Ye, Moshe Eizenman | WO2024059927A1
View patent
Patent
Methods and Systems for Gaze Tracking and Gaze Tracking Calibration
Soumil Chugh, Juntao Ye, Moshe Eizenman | WO2025113309A1
View patent
Patent
Predicting Blood Characteristics Using Digital Biomarkers
Michael Brown, Soumil Chugh, Javier J. Echenique, Shunsuke Matsuoka | WO2025106664A1
View patent
Patent
Methods and Devices for Gaze Estimation
Soumil Chugh, Juntao Ye | US12175014B2
View patent
Patent
Methods and Systems for Gaze-Assisted Interaction
Juntao Ye, Manpreet Singh Takkar, Soumil Chugh | US20240211034A1
View patent
Patent
Quality Control of User-Generated Biological Sample Cards
Michal Depa, Soumil Chugh, Sidhant Jena, Javier J. Echenique, Shunsuke Matsuoka, Theresa Tribble | WO2023091455A1
View patent