Publications - Last Year

This paper introduces a novel and scalable framework for uncertainty
estimation and separation with applications in data driven modeling in science
and engineering tasks where reliable uncertainty quantification is critical.
Leveraging an ensemble of quantile regression (E-QR) models, our approach
enhances aleatoric uncertainty estimation while preserving the quality of
epistemic uncertainty, surpassing competing methods, such as Deep Ensembles
(DE) and Monte Carlo (MC) dropout. To address challenges in separating
uncertainty types, we propose an algorithm that iteratively improves separation
through progressive sampling in regions of high uncertainty. Our framework is
scalable to large datasets and demonstrates superior performance on synthetic
benchmarks, offering a robust tool for uncertainty quantification in
data-driven applications.

We demonstrate generating high-dynamic range (HDR) images using the concerted
action of multiple black-box, pre-trained low-dynamic range (LDR) image
diffusion models. Common diffusion models are not HDR as, first, there is no
sufficiently large HDR image dataset available to re-train them, and second,
even if it was, re-training such models is impossible for most compute budgets.
Instead, we seek inspiration from the HDR image capture literature that
traditionally fuses sets of LDR images, called "brackets", to produce a single
HDR image. We operate multiple denoising processes to generate multiple LDR
brackets that together form a valid HDR result. To this end, we introduce an
exposure consistency term into the diffusion process to couple the brackets
such that they agree across the exposure range they share. We demonstrate HDR
versions of state-of-the-art unconditional and conditional as well as
restoration-type (LDR2HDR) generative modeling.

Markov chain Monte Carlo (MCMC) algorithms come to rescue when sampling from
a complex, high-dimensional distribution by a conventional method is
intractable. Even though MCMC is a powerful tool, it is also hard to control
and tune in practice. Simultaneously achieving both local exploration of the
state space and global discovery of the target distribution is a challenging
task. In this work, we present a MCMC formulation that subsumes all existing
MCMC samplers employed in rendering. We then present a novel framework for
adjusting an arbitrary Markov chain, making it exhibit invariance with respect
to a specified target distribution. To showcase the potential of the proposed
framework, we focus on a first simple application in light transport
simulation. As a by-product, we introduce continuous-time MCMC sampling to the
computer graphics community. We show how any existing MCMC-based light
transport algorithm can be embedded into our framework. We empirically and
theoretically prove that this embedding is superior to running the standalone
algorithm. In fact, our approach will convert any existing algorithm into a
highly parallelizable variant with shorter running time, smaller error and less
variance.

We introduce a theoretical and practical framework for efficient importance
sampling of mini-batch samples for gradient estimation from single and multiple
probability distributions. To handle noisy gradients, our framework dynamically
evolves the importance distribution during training by utilizing a
self-adaptive metric. Our framework combines multiple, diverse sampling
distributions, each tailored to specific parameter gradients. This approach
facilitates the importance sampling of vector-valued gradient estimation.
Rather than naively combining multiple distributions, our framework involves
optimally weighting data contribution across multiple distributions. This
adapted combination of multiple importance yields superior gradient estimates,
leading to faster training convergence. We demonstrate the effectiveness of our
approach through empirical evaluations across a range of optimization tasks
like classification and regression on both image and point cloud datasets.

Classical generative diffusion models learn an isotropic Gaussian denoising
process, treating all spatial regions uniformly, thus neglecting potentially
valuable structural information in the data. Inspired by the long-established
work on anisotropic diffusion in image processing, we present a novel
edge-preserving diffusion model that is a generalization of denoising diffusion
probablistic models (DDPM). In particular, we introduce an edge-aware noise
scheduler that varies between edge-preserving and isotropic Gaussian noise. We
show that our model's generative process converges faster to results that more
closely match the target distribution. We demonstrate its capability to better
learn the low-to-mid frequencies within the dataset, which plays a crucial role
in representing shapes and structural information. Our edge-preserving
diffusion process consistently outperforms state-of-the-art baselines in
unconditional image generation. It is also more robust for generative tasks
guided by a shape-based prior, such as stroke-to-image generation. We present
qualitative and quantitative results showing consistent improvements (FID
score) of up to 30% for both tasks. We provide source code and supplementary
content via the public domain edge-preserving-diffusion.mpi-inf.mpg.de .

In this paper, we propose a novel encoder-decoder architecture, named SABER,
to learn the 6D pose of the object in the embedding space by learning shape
representation at a given pose. This model enables us to learn pose by
performing shape representation at a target pose from RGB image input. We
perform shape representation as an auxiliary task which helps us in learning
rotations space for an object based on 2D images. An image encoder predicts the
rotation in the embedding space and the DeepSDF based decoder learns to
represent the object's shape at the given pose. As our approach is shape based,
the pipeline is suitable for any type of object irrespective of the symmetry.
Moreover, we need only a CAD model of the objects to train SABER. Our pipeline
is synthetic data based and can also handle symmetric objects without symmetry
labels and, thus, no additional labeled training data is needed. The
experimental evaluation shows that our method achieves close to benchmark
results for both symmetric objects and asymmetric objects on Occlusion-LineMOD,
and T-LESS datasets.

While consumer displays increasingly support more than 10 stops of dynamic
range, most image assets such as internet photographs and generative AI content
remain limited to 8-bit low dynamic range (LDR), constraining their utility
across high dynamic range (HDR) applications. Currently, no generative model
can produce high-bit, high-dynamic range content in a generalizable way.
Existing LDR-to-HDR conversion methods often struggle to produce photorealistic
details and physically-plausible dynamic range in the clipped areas. We
introduce LEDiff, a method that enables a generative model with HDR content
generation through latent space fusion inspired by image-space exposure fusion
techniques. It also functions as an LDR-to-HDR converter, expanding the dynamic
range of existing low-dynamic range images. Our approach uses a small HDR
dataset to enable a pretrained diffusion model to recover detail and dynamic
range in clipped highlights and shadows. LEDiff brings HDR capabilities to
existing generative models and converts any LDR image to HDR, creating
photorealistic HDR outputs for image generation, image-based lighting (HDR
environment map generation), and photographic effects such as depth of field
simulation, where linear HDR data is essential for realistic quality.

Data-driven generation of molecules with desired properties, also known as
inverse molecular design (IMD), has attracted significant attention in recent
years. Despite the significant progress in the accuracy and diversity of
solutions, existing IMD methods lag behind in terms of trustworthiness. The
root issue is that the design process of these methods is increasingly more
implicit and indirect, and this process is also isolated from the native
forward process (NFP), the ground-truth function that models the molecular
dynamics. Following this insight, we propose TrustMol, an IMD method built to
be trustworthy. For this purpose, TrustMol relies on a set of technical
novelties including a new variational autoencoder network. Moreover, we propose
a latent-property pairs acquisition method to effectively navigate the
complexities of molecular latent optimization, a process that seems intuitive
yet challenging due to the high-frequency and discontinuous nature of molecule
space. TrustMol also integrates uncertainty-awareness into molecular latent
optimization. These lead to improvements in both explainability and reliability
of the IMD process. We validate the trustworthiness of TrustMol through a wide
range of experiments.

Accurate property prediction is crucial for accelerating the discovery of new
molecules. Although deep learning models have achieved remarkable success,
their performance often relies on large amounts of labeled data that are
expensive and time-consuming to obtain. Thus, there is a growing need for
models that can perform well with limited experimentally-validated data. In
this work, we introduce MoleVers, a versatile pretrained model designed for
various types of molecular property prediction in the wild, i.e., where
experimentally-validated molecular property labels are scarce. MoleVers adopts
a two-stage pretraining strategy. In the first stage, the model learns
molecular representations from large unlabeled datasets via masked atom
prediction and dynamic denoising, a novel task enabled by a new branching
encoder architecture. In the second stage, MoleVers is further pretrained using
auxiliary labels obtained with inexpensive computational methods, enabling
supervised learning without the need for costly experimental data. This
two-stage framework allows MoleVers to learn representations that generalize
effectively across various downstream datasets. We evaluate MoleVers on a new
benchmark comprising 22 molecular datasets with diverse types of properties,
the majority of which contain 50 or fewer training labels reflecting real-world
conditions. MoleVers achieves state-of-the-art results on 20 out of the 22
datasets, and ranks second among the remaining two, highlighting its ability to
bridge the gap between data-hungry models and real-world conditions where
practically-useful labels are scarce.