Yash Sharma

I am an AI researcher. I defended my PhD from the International Max Planck Research School for Intelligent Systems, after working in the lab of Matthias Bethge under the supervision of Wieland Brendel. I've completed my Bachelors and Masters from the Cooper Union, and have spent time at Entrepreneurs First, Flagship Pioneering, Google Brain, Meta AI, Amazon, Borealis AI and IBM Research.

My research has focused on adversarial robustness, representation learning, and compositional generalization.

Email  /  Twitter  /  CV  /  Google Scholar  /  Kaggle  /  Github

Compositional generalization of CLIP models on real-world retrieval tasks can consistently be predicted from pretraining frequencies.

Increasing training data frequency of concepts exponentially yields linear improvements in downstream "zero-shot" performance, thus constituting sample inefficient log-linear scaling.

Increasing training data diversity of attributes in unseen combinations reduces the systematicity gap in visual question answering.

Bounding the robustness of a predictor on downstream tasks by the robustness of the representation that underlies it.

Identifying ground-truth object representations, even in the presence of dependencies between objects, by learning an invertible and compositional inference model.

Uncovering causal relationships from observational data by relying on the Jacobian of the function inferring the underlying sources from the observed variables.

Match local features at different points in time by tracking points with optical flow.

Disentanglement, allowing for dependence between sequences of transitions, by identifying both the factors of variation and the sparse causal graphical model that relates them.

Enabled flexible compositions of discovered concepts, across modalities and datasets, by formulating sample generation as an optimization process on underlying energy functions.

We show that contrastive learning isolates the features data augmentations leave invariant, providing a proof of identifiability and empirical results on a novel visually complex benchmark with causal dependencies.

We show that contrastive learning can uncover the underlying factors of variation, with proofs and empirical success on a novel visually complex dataset.

Provide a video benchmark as well as an extension of a static method for unsupervised learning of object-centric representations for analysis.

We show that accounting for the temporally sparse nature of natural transitions leads to a proof of identifiability and reliable learning of disentangled representations on several established benchmark datasets, as well as contributed datasets with natural dynamics.

Contribute a model class that is well suited for modeling the dynamics of systems that only offer local views into their state, along with corresponding spatial locations of those views.

Directly maximizing input space margin removes the requirement of specifying a fixed distortion bound for improving adversarial robustness.

Defended ImageNet models based on adversarial training are roughly as vulnerable to low frequency perturbations as undefended models.

Across factorization structures, provide evidence that generative classifiers are more robust to adversarial attacks than discriminative classifiers.

Evolutionary strategies can be used to synthesize adversarial examples in the black-box setting with orders of magnitude fewer queries than previous approaches.

Placed 1st, 1st, and 3rd in the targeted attack, non-targeted attack, and defense competitions, respectively, winning the competition overall. Prize: $38,000.

Generate adversarial examples that fool well-trained sentiment analysis and textual entailment models in the black-box setting while preserving semantics and syntactics of the original.

Contributed to the CleverHans software library.

Thesis Advisor: Sam Keene

Can bypass the feature squeezing detection framework with adversarial examples of minimal visual distortion by simply evaluating with stronger attack configurations.

Models adversarially trained on the L_inf metric are vulnerable to L1-based adversarial examples of minimal visual distortion.

Encouraging sparsity in the perturbation with L1 minimization leads to improved attack transferability and complements adversarial training.

Directly estimate the gradients of the target model for generating adversarial examples in the black-box setting, sparing the need for training substitute models and avoiding the loss in attack transferability.

Built set of functions based on implementing solutions for the provided tasks, searched for the correct composition for a given task at runtime. Served in mainly an advisory role.

Teamed with UCNesl to finish with one gold and two silver medals in the competition track.

Built an autonomous vehicle which can navigate through maps consisting of various road topologies.

Stabilized the training of competitive agents under human-level action delay through adding recurrence to the DQN architecture.

Developed a client-server application which allows users to play the game of SET against each other over the network.

Implemented a multiple pairs trading strategy on major currency pairs and improved the APR over the evaluation period by factoring in forecasts of a series of pertinent macroeconomic variables by optimizing the weights of the trading signals.

Implemented the Split-Brain Autoencoder in TensorFlow and showed that the extracted features can help supervised learning when labeling is costly.

Built an android application which allows users to record notes about their meetings, write messages that will be displayed on a Twitter-like feed, and determine viability of initiatives regarding specific tickers through sentiment analysis run on written logs.

Built an 8-bit single-cycle processor with a 4-byte cache for data memory capable of executing nested procedures, leaf procedures, signed addition, loops, and recursion.

Traced the sources of pollution in the continental United States through compiling data from the EPA, NOAA, and Google Maps API, estimating the parameters of a gaussian dispersion model, and predicting pollutant concentration in the future with linear regression.

Template: here