Varun Srivastava

Adversarial Approximate Inference for Speech to Electroglottograph Conversion

Sept 2018 - Jan 2019 Prof. Prathosh AP, Dept of EE, IIT Delhi

A distribution transformation framework to map speech to the corresponding EGG signal, robust to noise, generalizing across recording conditions, speech pathologies and voice qualities.

• Optimized the Speech to Laryngograph encoder using adversarial training for the network.
• Created a cosine based loss function for enforcing amplitude invariance between ground truth and network output.
• Used a variational inference approach for learning optimal representations for the speech signal.
• Utilized continuous wavelet transforms using Ricker wavelets for robust peak picking.

Published in IEEE/ACM Transactions on Audio, Speech, and Language Processing. Manuscript available at arxiv link

Detection of Glottal Closure Instants using CNNs

Prof. Prathosh AP, Dept of EE, IIT Delhi

Glottal Closure Instants (GCIs) correspond to the temporal locations of significant excitation to the vocal tract occurring during the production of voiced speech. GCI detection is cast as a supervised multi-task learning problem solved using a deep convolutional neural network jointly optimizing a classification and regression cost.

• Devised state of the art deep dilated CNN to detect and locate GCIs from raw speech.
• Performed multi task learning (simultaneous regression and classification) on speech for temporal event detection.
• Created a robust Weighted Histogram based Clustering Algorithm for time series clustering.
• Beat State of the Art method with Lowest Variance in Localisation Error on noisy speech.

Published in INTERSPEECH 2019. Manuscript available at arxiv link

Mode matching in GANs through latent space learning and inversion

Prof. Prathosh AP, Dept of EE, IIT Delhi

NEMGAN addresses the problem of imposing desired modal properties on the generated space using a latent distribution, engineered in accordance with the modal properties of the true data distribution. This is achieved by training a latent space inversion network in tandem with the generative network using a divergence loss.

• Construction of a multi modal latent distribution by reparameterization of a mixture of continuous and discrete random variables employable in any GAN framework.
• Imposition of modal properties of the latent space the generated space using two-stage latent inversion.
• Learnt the mode priors of the latent distribution to follow true data distribution using sparse-supervision.

Manuscript available at arxiv link

Non Contact CardioPulmonary Assessment using Imaging Techniques

Prof. Prathosh AP, Dept of EE, IIT Delhi

This porject builds a single end to end model - for estimation of rPPG (and its derivatives) from videos, using a neural network for all modelling conditions (Lighting, Skin Complexion etc). It also explores efficacy of using Near Infrared spectrum (650 - 1350 nm), which is optimal for remote monitoring – as it has maximal penetration depth.

• Performing non contact estimation of human vitals (heart rate, oxygen saturation etc) using infrared cameras.
• Devising deep learning models to perform human skin segmentation from videos in near infrared spectrum.

Slides available at Google Slides

Awarded Qualcomm Research Innovation Fellowship with Rs 10,00,000 of funding for the project proposal

Analysis and Optimization of Text Generation Models

Prof. Jayadeva, Dept of EE, IIT Delhi

This project integrates contextual information from Brown Corpus and WordNet into a text generating model to produce grammatically correct, meaningful sentences in the long run.

• Designed a probabilistic discriminative model using Brown Corpus and WordNet to perform text prediction.
• Implemented a LSTM in Chainer (dynamic computational graph framework) for language modelling.
• Improved word perplexity by building a factored language model incorporating both words and parts of speech.

Slides available at Google Slides

Awarded Summer Undergraduate Research Award for successful completion

NAVI: Navigation Assistance for Visually Impaired

Prof. Balakrishnan, Dept of CS, IIT Delhi

NAVI (SmartCane 2.0) is a smart navigation device to help the visually impaired navigate using minimum external help. Equipped with several accessibility features, this device warns the user if they are heading wrong and gives tactile instructions for course correction. It also allows them to save new routes while navigating and add attributes like landmarks in real time.

• Designed a standalone device on Raspberry Pi (six person team) to assist visually impaired in navigation.
• Modeled navigation as a graph traversal in Python to determine best routes and perform course correction.
• Implemented a Kalman Filter to integrate data from a GPS and a 9 axis IMU for precise localization.

Blog post available at medium link