Gaurav Shinde
Ph.D. Student ยท Information Systems ยท UMBC
๐ค Multi-Robot Autonomy๐๏ธ Collaborative Perception๐ง Embodied AI
Hi, I'm Gaurav ๐ I'm a Ph.D. student in Information Systems at the University of Maryland, Baltimore County, advised by Prof. Nirmalya Roy. I solve challenges that robot teams face when operating autonomously in search-and-rescue and contested reconnaissance missions. My work spans collaborative perception, multimodal sensor fusion and VLM/LLM-based reasoning and planning for heterogeneous robot teams. I validate these systems in simulation environments and on real robots.
Before UMBC, I was a research assistant at the MiCoSys Lab at San Josรฉ State University with Prof. Saptarshi Sengupta, where I worked on time-series modeling using deep learning techniques to predict the remaining useful life of Li-ion batteries. I also worked as a software and ML engineer in industry. Beyond research, I serve in university leadership as GEARS Chair of the UMBC Graduate Student Association and as a council member of UMBC's Research and Creative Achievement Council.
๐ฌ Research threads
- ๐ค Multi-robot collaborative perception: visionโradio fusion for GPS-free ranging WoWMoM'26, task- and channel-aware image sharing PerCom-W'25, cooperative task execution DCOSS-IoT'25
- ๐๏ธโ๐จ๏ธ VLM reasoning for embodied agents: zero-shot multi-robot object-goal navigation with reflective reasoning ICRA'27 ยท under review, semanticโgeometric robot deployment HSCC+ICCPS'27 ยท under review, multi-agent VLM verification CVPR-W'26
- โก Efficient & edge AI: efficient VLMs WIREs DMKD'25, collaborative DNN inference across robots WoWMoM'26, class-aware federated learning PerCom'26
๐ฐ Recent News
| Nov 2026 | ๐ | Two papers submitted to HSCC+ICCPS 2027 (part of CPS-IoT Week 2027). |
|---|---|---|
| Sep 2026 | ๐ | Paper submitted to IEEE ICRA 2027. |
| Aug 2026 | ๐๏ธ | Joined UMBC's Research and Creative Achievement Council as a council member. |
| Aug 2026 | ๐จโ๐ซ | Teaching lab sections of IS 420: Database Application Development at UMBC (90 students). |
| Jul 2026 | ๐ค | Serving as GEARS Chair for the UMBC Graduate Student Association. |
| May 2026 | ๐ง | Reviewer for WIREs DMKD and IEEE TMC. |
| Apr 2026 | ๐ค | Organized the NSF TRACE Robotic Testbed Workshop at UMBC. |
| Apr 2026 | ๐ | VLS-RR accepted at IEEE/ACM CHASE 2026. |
| Mar 2026 | ๐ | CALIBRA accepted at the CVPR 2026 GRAIL-V Workshop. |
| Mar 2026 | ๐ | CAViAR and COHORT accepted at IEEE WoWMoM 2026. |
| Jan 2026 | ๐ | Inducted as a full member of the Sigma Xi Research Honor Society. |
| Dec 2025 | ๐ | Fed-CASQ accepted at IEEE PerCom 2026. |
| Sep 2025 | โ | Passed my Ph.D. comprehensive exam. |
| Sep 2025 | ๐ | RespFormer accepted at ICMLA. |
| Sep 2025 | ๐ง | Reviewer for Pattern Recognition Letters. |
| Aug 2025 | ๐งโโ๏ธ | TPC member for COMSNETS. |
| Aug 2025 | ๐ฐ๏ธ | Demonstrated remote multi-robot experimentation at Graces Quarters (DEVCOM ARL, ArtiAMAS). |
| Jun 2025 | ๐ | Efficient VLM Survey accepted at WIREs DMKD. |
| Apr 2025 | ๐ | E2RespUNet accepted at IEEE SmartComp and CoOpTex at IEEE DCOSS-IoT. |
| Mar 2025 | โ๏ธ | Awarded an NSF Student Travel Grant for IEEE PerCom 2025. |
| Jan 2025 | ๐ | TAVIC-DAS accepted at the IEEE PerCom 2025 Workshops. |
| Aug 2024 | ๐ | Joined UMBC as a Ph.D. student. |
๐ Publications
Google Scholar ยท 150+ citationsClick any figure to enlarge it.
-
IEEE WoWMoM 2026โ First author
CAViAR: Quality-Aware Vision-and-Radio Fusion for Relative Range Estimation among Collaborative Autonomous Agents
Fuses camera and radio range estimates with learned quality scores for GPS-free inter-robot ranging.
In mission-critical scenarios, autonomous agents often operate in GPS-denied or GPS-degraded environments, making it challenging to localize agents relative to themselves and objects of interest (e.g., potential cover) or adversarial robots. While vision- and radiobased modalities have individually been used to estimate relative ranges between agents and surrounding objects, each modality suffers from inherent limitations. Vision-based range estimation degrades significantly under low image overlap, partial visibility, or when an agent is too close to an object, resulting in effective zoom-in and loss of geometric context. Conversely, radio-based ranging is susceptible to multipath interference, signal fading, and environmental variability, which can substantially reduce range accuracy and reliability. To address these challenges, we introduce CAViAR, a quality-aware multimodal fusion framework for accurate relative range estimation among collaborative autonomous agents. CAViAR assigns modality-specific reliability scores and performs statistical fusion to adaptively weigh vision and radio inputs based on their estimated quality. The framework employs modality-specific quality estimators augmented with temporal features and integrates MBConv blocks to enable efficient feature processing on resource-constrained robotic platforms. We validate CAViAR on ROSbot 2 and ROSbot 2 Pro platforms using an in-house dataset collected across diverse indoor and outdoor environments. Experimental results demonstrate that our approach outperforms single-modality baselines by approximately 21% over vision-only and 36% over radio-only range estimates. Moreover, CAViAR adapts robustly to variations in scene structure, viewpoint overlap, and occlusions without requiring fine-tuning on new environments, highlighting its practicality for real-world deployment.
-
WIREs DMKD 2025โ First authorJournal ยท IF 11.7
A Survey on Efficient Vision-Language Models
Reviews the techniques that make VLMs practical on edge and resource-constrained devices.
Vision-language models (VLMs) integrate visual and textual information, enabling a wide range of applications such as image captioning and visual question answering, making them crucial for modern AI systems. However, their high computational demands pose challenges for real-time applications. This has led to a growing focus on developing efficient vision-language models. In this survey, we review key techniques for optimizing VLMs on edge and resource-constrained devices. We also explore compact VLM architectures, frameworks, and provide detailed insights into the performanceโmemory trade-offs of efficient VLMs. Furthermore, we establish a GitHub repository at MPSC-GitHub to compile all surveyed papers, which we will actively update. Our objective is to foster deeper research in this area.
-
IEEE PerCom Workshops 2025โ First author
TAVIC-DAS: Task and Channel-Aware Variable-Rate Image Compression for Distributed Autonomous System
Adapts image compression to the downstream task and the live radio channel so robot teams share images efficiently.
In network-constrained environments, distributed multi-agent systemsโsuch as UGVs and UAVsโmust communicate effectively to support computationally demanding scene perception tasks like semantic and instance segmentation. These tasks are challenging because they require high accuracy even when using low-quality images, and the network limitations restrict the amount of data that can be transmitted between agents. To overcome the above challenges, we propose TAVIC-DAS to perform a task and channel-aware variable-rate image compression to enable distributed task execution and minimize communication latency by transmitting compressed images. TAVIC-DAS proposes a novel image compression and decompression framework (distributed across agents) that integrates channel parameters such as RSSI and data rate into a task-specific "semantic segmentation" DNN to generate masks representing the object of interest in the scene (ROI maps) by determining a high pixel density needed to represent objects of interest and low density to represents surrounding pixels within an image. Additionally, to accommodate agents with limited computational resources, TAVIC-DAS incorporates resource-aware model quantization. We evaluated TAVIC-DAS on platforms such as ROSMaster X3 and Jetson Xavier, which communicated using a low-frequency proprietary Doodle radio operating at 915 MHz. The experimental results show that TAVIC-DAS achieves approximately 7.62% higher PSNR and is about 6.39% more resource efficient compared to state-of-the-art techniques.
-
IEEE BigData 2023โ First author
De-SaTE: Denoising Self-attention Transformer Encoders for Li-ion Battery Health Prognostics
Denoising self-attention encoders that predict Li-ion battery remaining useful life under heterogeneous noise.
The usage of Lithium-ion (Li-ion) batteries has gained widespread popularity across various industries, from powering portable electronic devices to propelling electric vehicles and supporting energy storage systems. A central challenge in Li-ion battery reliability lies in accurately predicting their Remaining Useful Life (RUL), which is a critical measure for proactive maintenance and predictive analytics. This study presents a novel approach that harnesses the power of multiple denoising modules, each trained to address specific types of noise commonly encountered in battery data. Specifically, a denoising auto-encoder and a wavelet denoiser are used to generate encoded/decomposed representations, which are subsequently processed through dedicated self-attention transformer encoders. After extensive experimentation on NASA and CALCE data, a broad spectrum of health indicator values are estimated under a set of diverse noise patterns. The reported error metrics on these data are on par with or better than the state-of-the-art reported in recent literature.
-
IEEE PerCom 2026
Fed-CASQ: Enhancing Class-Wise Accuracy in Pervasive Federated Learning with Class-Aware Scaling and Quantization
Couples class-aware filter scaling with resource-aware quantization so federated learning on edge devices keeps accuracy on hard-to-learn classes.
Federated Learning (FL) enables collaborative machine learning across decentralized devices and data sources, but resource constraints on pervasive devices necessitate efficient model compression. Existing approaches, such as quantization for on-device training, often degrade accuracy, especially for classes that are difficult to learn due to imbalance, poor-quality samples, or inherent complexity. This results in persistent accuracy gaps across classes. We propose Fed-CASQ's a novel framework that couples class-aware strategies into the quantization process to jointly improve efficiency and accuracy in pervasive FL. Unlike prior works that address quantization and imbalance separately, Fed-CASQ adaptively selects quantization levels based on device resources and leverages Layer-wise Relevance Propagation (LRP) to assess class-relevant convolutional neural network (CNN) filters on the client side. An adaptive weight scaling mechanism is then applied to amplify critical information for low-accuracy classes before aggregation. At the server, a complementary novel aggregation strategy mitigates global imbalance across clients, ensuring that underperforming classes receive proportional attention during model updates. We theoretically establish that Fed-CASQ achieves a convergence rate of O(ฮบโฯ^โฮด^Tโ) under non-convex settings. We empirically establish that quantization directly influences the performance of under sampled (minority) classes. Experimental results further show that Fed-CASQ substantially narrows the performance gap for low-accuracy classes, improving their accuracy by โ30%, while reducing training latency by over 56% on resource-constrained pervasive devices.
-
IEEE WoWMoM 2026
COHORT: Hybrid RL for Collaborative Large DNN Inference on Multi-Robot Systems Under Real-Time Constraints
Hybrid offlineโonline RL that schedules large DNN modules across a robot team to meet frame-rate and deadline constraints while saving battery..
Large deep neural networks (DNNs), especially transformer-based and multimodal architectures, are computationally demanding and challenging to deploy on resource-constrained edge platforms like field robots. These challenges intensify in mission-critical scenarios (e.g., disaster response), where robots must collaborate under tight constraints on bandwidth, latency, and battery life, often without infrastructure or server support. To address these limitations, we present COHORT, a collaborative DNN inference and task-execution framework for multi-robot systems built on the Robotic Operating System (ROS). COHORT employs a hybrid offline-online reinforcement learning (RL) strategy to dynamically schedule and distribute DNN module execution across robots. Our key contributions are threefold: (a) Offline RL policy learning combined with Advantage-Weighted Regression (AWR), trained on auction-based task allocation data from heterogeneous DNN workloads across distributed robots, (b) Online policy adaptation via Multi-Agent PPO (MAPPO), initialized from the offline policy and fine-tuned in real time, and (c) comprehensive evaluation of COHORT on vision-language model (VLM) inference tasks such as CLIP and SAM, analyzing scalability with increasing robot/workload and robustness under. We benchmark COHORT against genetic algorithms and multiple RL baselines. Experimental results demonstrate that COHORT reduces battery consumption by 15.4%, and increases GPU utilization by 51.67%, while satisfying frame-rate and deadline constraints 2.55ร of the time.
-
CVPR 2026 Workshops
CALIBRA: Calibration-Aware Multi-Agent Verification for Contactless Physiological Monitoring
Specialized VLM agents analyze evidence, a critic checks consistency, and an arbiter makes a calibrated respiratory-rate decision.
Contactless physiological monitoring offers a promising path toward low-burden health sensing, but reliable inference from short video segments remains difficult when visual evidence is weak, corrupted by motion, or spectrally ambiguous. We present CALIBRA, a calibration-aware multi-agent framework for contactless respiratory rate (RR) estimation that treats prediction as an evidence-grounded verification problem rather than a one-shot prompting task. The framework operates on a structured artifact bundle with lightweight signal-derived candidate rates, and performs routing, specialized vision-language evidence analysis, critic-based consistency checking, targeted repair, and calibrated arbitration with optional abstention. On our inhouse benchmark, it consistently outperforms promptingonly baselines across multiple backbones. With Gemini-2.5- Flash, it reduces mean absolute error (MAE) from 5.21 to 3.42, corresponding to a 34.4% reduction over zero-shot prompting and a 16.8% reduction over chain-of-thought prompting. With GPT-4o, it reduces MAE from 5.48 to 3.78, a 31.0% reduction over zero-shot prompting. It also improves grounding and consistency in judge-based evaluation, indicating better evidence alignment beyond numerical accuracy alone. Under resource constraints, a routed configuration maintains 93% coverage at 4.1 s average latency and 1.6 GB peak memory. These results show that robust contactless physiological inference depends not only on stronger multimodal backbones, but also on explicit verification of whether predictions are supported by the available evidence.
-
IEEE/ACM CHASE 2026
Decoupling Perception and Reasoning for Contactless Respiratory Rate with Vision Language and Small Language Models
A VLM turns video into textual evidence and compact small language models reason over it for on-device respiratory-rate estimation.
Contactless respiratory-rate (RR) monitoring from video is promising but remains sensitive to motion artifacts, viewpoint changes, and lighting variation. We propose VisualโLanguageโSLM Respiratory Rate (VLS-RR) auditor, a three-stage framework that decouples visual perception from downstream reasoning for robust and deployable RR estimation. VLS-RR first extracts fused respiratory motion signals and diagnostic visual summaries from video segments. A visionโlanguage model then generates textual evidence describing rhythm, breath count, and artifacts, while small language models perform structured chain-of-thought auditing over this evidence and signal-derived RR candidates to produce reliability-aware estimates. We evaluate VLS-RR on an in-house belt-synchronized video dataset comprising 50 videos (UMBC IRB #1863) with diverse breathing patterns, viewpoints, and lighting conditions. Compared with signal-only baselines, VLS-RR reduces segment-level MAE by โ 30% and improves R2 from 0.84 to 0.92, while achieving an additional โ 25% MAE reduction over numeric-only SLM baselines. On an embedded edge device, compact SLM auditors run with 0.82โ1.71 s latency per 6 s window, supporting accurate and resource-efficient RR estimation from video.
-
ICMLA 2025
RespFormer: A Motion-Guided Temporal-Frequency Multimodal Fusion Transformer for Contactless Respiratory Monitoring
An ensemble of time-series transformers with attention refinement module and meta-learner-based fusion for RGB, NIR and IR video motion.
Monitoring respiratory rate (RR) is essential for early identification of respiratory and metabolic abnormalities. However, the limitations of contact-based sensors and the lack of reliability in many contactless methods make continuous and accurate monitoring difficult in non-clinical settings. To address these challenges, we introduce RespFormer, an edge-optimized, motion-guided temporal-frequency multimodal fusion transformer framework for real-time, contactless RR estimation and breathing pattern classification. RespFormer integrates dense optical flow analysis with temporal, statistical, and frequency-domain features derived from video sequences and enhances them through a multi-stage signal processing pipeline. These features are modeled using an ensemble of three time series transformer architectures (ETSformer, Temporal Fusion Transformer, and Informer) to capture distinct aspects of temporal dynamics. A shared attention-based refinement module enhances the feature representations, and final predictions are fused using a stacking-based meta-learner. We validate RespFormer on a multimodal dataset comprising synchronized RGB, NIR, and IR video data, including a custom in-house dataset captured under various conditions. Experimental results demonstrate that RespFormer achieves a mean absolute error (MAE) โ 0.98 bpm, improving prediction accuracy by โ 11% and reducing memory usage by โ 26%, while maintaining real-time inference (โ 1.22 seconds) on resource-constrained devices. Furthermore, RespFormer accurately classifies breathing patterns (normal, bradypnea, tachypnea, and apnea) with 95% accuracy, underscoring itโs potential for practical application in telemedicine, clinical screening, and low-resource healthcare settings.
-
IEEE SmartComp 2025
E2RespUNet: End-to-End Respiratory Signal Reconstruction and Rate Prediction Using a Unified Attention-Enhanced U-Net
An end-to-end attention-enhanced U-Net that reconstructs the breathing signal from video and predicts respiratory rate.
Continuous, non-invasive respiratory rate (RR) monitoring is essential for the early diagnosis of many medi-cal problems. However, conventional contact-based sensors frequently under-perform in dynamic situations that can be un-comfortable and require human intervention. To overcome these limitations, we propose E2RespUNet, an end-to-end system that uses multimodal video data to estimate breathing rates and reconstruct respiratory signals using an attention-enhanced U-Net architecture. Our method combines optical flow analysis with preprocessing, detrending, and normalization to reliably extract chest motion features in a variety of settings. We validated our approach using a collection of RGB, near-infrared (NIR), and infrared (IR) videos from multiple sources, such as an in-house RGB dataset and a publicly available sleep dataset. According to our study in both the temporal and frequency domains, E2RespUNet surpasses existing baseline models by lowering the mean absolute error by up to 21 % in the sleep dataset and by up to 28% in the in-house dataset. Additionally, E2RespUNet addresses real-time deployment requirements on resource-constrained edge devices by processing each frame in less than 1 ms (0.99 ms in the Sleep dataset and 0.97 ms in the in-house dataset). These findings, together with a low KL divergence and a power spectral density (PSD) that corresponds to the ground truth, suggest that E2Resp UN et is accurate and fast enough for real-time use in emergency and clinical scenarios.
-
IEEE DCOSS-IoT 2025
CoOpTex: Multimodal Cooperative Perception and Task Execution in Time-Critical Distributed Autonomous System
Stitches multi-view RGB from several robots for 2D detection, fuses it with LiDAR 3D detections via a GNN, and offloads image stitching to meet time-critical deadlines.
Integrating multimodal data such as RGB and LiDAR from multiple views significantly increases computational and communication demands, which can be challenging for resource-constrained autonomous agents while meeting the time-critical deadlines required for various mission-critical applications. To address this challenge, we propose CoOpTex, a collaborative task execution framework designed for cooperative perception in distributed autonomous systems (DAS). CoOpTex contribution is twofold: (a) CoOpTex fuses multiview RGB images to create a panoramic camera view for 2D object detection and utilizes 360ยฐ LiDAR for 3D object detection, improving accuracy with a lightweight Graph Neural Network (GNN) that integrates object coordinates from both perspectives, (b) To optimize task execution and meet the deadline, CoOpTex dynamically offloads computationally intensive image stitching tasks to auxiliary devices when available and adjusts frame capture rates for RGB frames based on device mobility and processing capabilities. We implement CoOpTex in real-time on static and mobile heterogeneous autonomous agents, which helps to significantly reduce deadline violations by 100% while improving frame rates for 2D detection by 2.2ร in stationary and 2ร in mobile conditions, demonstrating its effectiveness in enabling real-time cooperative perception.
Research Experience
-
Graduate Research AssistantMobile, Pervasive & Sensor Computing Lab, UMBCAug 2024 โ PresentMultimodal LLMsAgentic AIFeed-Forward 3D ReconstructionVirtualโPhysical Co-Simulation
- Introduced CAViAR, quality-aware visionโradio fusion for GPS-free relative range estimation among collaborative autonomous agents (IEEE WoWMoM 2026).
- Proposed TAVIC-DAS, task- and channel-aware variable-rate image compression for bandwidth-efficient information sharing across robot teams (IEEE PerCom Workshops 2025).
- Developed COVER, a multi-robot framework that estimates object dimensions and their reliability from multi-view RGB images and assigns robots to cover objects using LLM-based compatibility scoring and constrained optimization (under review, HSCC+ICCPS 2027).
- Introduced SPERE-Nav, a zero-shot multi-robot object-goal navigation framework where a VLM assigns frontiers using retrieved visual evidence and object-level spatial relations, then verifies each assignment via reflective reasoning (under review, IEEE ICRA 2027).
- Led a survey on efficient vision-language models for resource-constrained devices (WIREs DMKD).
-
Graduate Research AssistantInterPARES Trust AIAug 2023 โ May 2024
- Collaborated with archivists to conduct a large-scale systematic review of AI techniques in archival science, highlighting practical applications and open research gaps (ASIS&T 2025).
-
Graduate Research AssistantMachine Intelligence & Complex Systems Lab, SJSUFeb 2023 โ May 2024Time-Series ModelingBattery Health Prognostics
- Developed De-SaTE, a denoising self-attention transformer architecture that accurately predicts the remaining useful life of Li-ion batteries under heterogeneous noise in battery operational data (IEEE BigData 2023).
Field Demos & Testbeds
Collaborative Remote Robotics Experiment using the Distributed Virtual Proving Ground
Demonstrated an agentic multi-robot system for visual-target navigation in contested environments.
NSF AI Ready TRACE Test Bed Workshop and Field Experiment Demonstration
Developed photorealistic digital twins of indoor environments in NVIDIA Isaac Sim using structure-from-motion and Gaussian-based neural reconstruction, and integrated collision geometry and physics to support virtualโphysical co-simulation with real robots.
๐ผ Industry
-
Machine Learning Engineer InternTech Mahindra ยท CaliforniaJun 2023 โ Aug 2023
- Detected and masked personally identifiable information (PII) for 60,000 records using Google Data Loss Prevention API, Presidio, and custom Transformer models.
- Spearheaded the development of a web application using Flask, generating custom alerts based on PII sensitivity.
-
Software EngineerInsta ICT Solutions Pvt Ltd ยท IndiaSep 2021 โ Jul 2022
- Implemented Redux architecture in a React application for efficient state management across components, achieving a 15% reduction in rendering time and improving the user experience.
- Developed a robust middleware logger, resulting in 20% decrease in issue resolution time.
- Achieved 30% reduction in unauthorized access attempts by deploying JWT-based token authentication.
-
Software Engineer InternData Glove Inc. ยท IndiaDec 2020 โ Mar 2021
- Integrated PySpark with Azure Synapse to orchestrate efficient data extraction, transformation, and visualization, resulting in a 15% reduction in data processing time.
- Built scalable data pipelines on Azure (Data Factory, Databricks, Data Lake Storage) to extract data from multiple sources for visualization in Power BI.
๐ค Service, Leadership & Teaching
๐ง Peer Review
- Journals: IEEE Transactions on Mobile Computing (2026), WIREs DMKD (2026), Pattern Recognition Letters (2025)
- Conferences: IEEE BigData (2026), COMSNETS (2025, 2026)
- Program Committee: TPC Member, COMSNETS
๐๏ธ Organizing
- Organizing Team, NSF TRACE Robotic Testbed Workshop, UMBC (2026)
- Local Organizer, IEEE PerCom (2025)
๐๏ธ University Leadership
- GEARS Chair, UMBC Graduate Student Association Jul 2026 โ PresentLead UMBC's campus-wide Research Symposium and Three Minute Thesis (3MT) competition that showcase student research from all three colleges.
- Council Member, Research & Creative Achievement Council Aug 2026 โ Present
๐จโ๐ซ Teaching & Mentoring
- IS 420: Database Application Development, UMBC Aug 2026 โ PresentLed weekly lab sections for 90 students covering SQL and PL/SQL, ER modeling, normalization, stored procedures, triggers, and transaction control; graded examinations and evaluated course projects.
- Mentored NSF REU undergraduate researchers (2025) and a UMBC Ph.D. student (2026 โ Present).
๐ Honors
- NSF Student Travel GrantIEEE PerCom 2025
- Sigma Xi Research Honor SocietyFull Member, 2026