Publications - Last Year

Human performance capture is a highly important computer vision problem with
many applications in movie production and virtual/augmented reality. Many
previous performance capture approaches either required expensive multi-view
setups or did not recover dense space-time coherent geometry with
frame-to-frame correspondences. We propose a novel deep learning approach for
monocular dense human performance capture. Our method is trained in a weakly
supervised manner based on multi-view supervision completely removing the need
for training data with 3D ground truth annotations. The network architecture is
based on two separate networks that disentangle the task into a pose estimation
and a non-rigid surface deformation step. Extensive qualitative and
quantitative evaluations show that our approach outperforms the state of the
art in terms of quality and robustness. This work is an extended version of
DeepCap where we provide more detailed explanations, comparisons and results as
well as applications.

Bayesian optimization (BO) provides a powerful framework for optimizing
black-box, expensive-to-evaluate functions. It is therefore an attractive tool
for engineering design problems, typically involving multiple objectives.
Thanks to the rapid advances in fabrication and measurement methods as well as
parallel computing infrastructure, querying many design problems can be heavily
parallelized. This class of problems challenges BO with an unprecedented setup
where it has to deal with very large batches, shifting its focus from sample
efficiency to iteration efficiency. We present a novel Bayesian optimization
framework specifically tailored to address these limitations. Our key
contribution is a highly scalable, sample-based acquisition function that
performs a non-dominated sorting of not only the objectives but also their
associated uncertainty. We show that our acquisition function in combination
with different Bayesian neural network surrogates is effective in
data-intensive environments with a minimal number of iterations. We demonstrate
the superiority of our method by comparing it with state-of-the-art
multi-objective optimizations. We perform our evaluation on two real-world
problems -- airfoil design and 3D printing -- showcasing the applicability and
efficiency of our approach. Our code is available at:
github.com/an-on-ym-ous/lbn_mobo

Image deblurring aims to recover the latent sharp image from its blurry
counterpart and has a wide range of applications in computer vision. The
Convolution Neural Networks (CNNs) have performed well in this domain for many
years, and until recently an alternative network architecture, namely
Transformer, has demonstrated even stronger performance. One can attribute its
superiority to the multi-head self-attention (MHSA) mechanism, which offers a
larger receptive field and better input content adaptability than CNNs.
However, as MHSA demands high computational costs that grow quadratically with
respect to the input resolution, it becomes impractical for high-resolution
image deblurring tasks. In this work, we propose a unified lightweight CNN
network that features a large effective receptive field (ERF) and demonstrates
comparable or even better performance than Transformers while bearing less
computational costs. Our key design is an efficient CNN block dubbed LaKD,
equipped with a large kernel depth-wise convolution and spatial-channel mixing
structure, attaining comparable or larger ERF than Transformers but with a
smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over
the state-of-the-art Restormer on defocus / motion deblurring benchmark
datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments
demonstrate the superior performance of our network and the effectiveness of
each module. Furthermore, we propose a compact and intuitive ERFMeter metric
that quantitatively characterizes ERF, and shows a high correlation to the
network performance. We hope this work can inspire the research community to
further explore the pros and cons of CNN and Transformer architectures beyond
image deblurring tasks.