From 6bccf77d6e8b8df23fe942cbcf8d319c185083c0 Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 07:57:25 +0100 Subject: [PATCH 1/7] Fix PDF wrap hyphens in _posts abstracts. Repair letter- space artifacts (including cross-line YAML folds) so pmlint posts --check passes on published volume correction content. --- _posts/2018-08-06-adams18a.md | 8 ++++---- _posts/2018-08-06-aditya18a.md | 25 ++++++++++++------------- _posts/2018-08-06-agrawal18a.md | 12 ++++++------ _posts/2018-08-06-atzmon18a.md | 14 +++++++------- _posts/2018-08-06-babichev18a.md | 6 +++--- _posts/2018-08-06-berg18a.md | 8 ++++---- _posts/2018-08-06-bixler18a.md | 10 +++++----- _posts/2018-08-06-bloem-reddy18a.md | 12 ++++++------ _posts/2018-08-06-blom18a.md | 8 ++++---- _posts/2018-08-06-broka18a.md | 12 ++++++------ _posts/2018-08-06-callaway18a.md | 18 +++++++++--------- _posts/2018-08-06-chaudhuri18a.md | 8 ++++---- _posts/2018-08-06-chen18a.md | 12 ++++++------ _posts/2018-08-06-chen18b.md | 8 ++++---- _posts/2018-08-06-cheng18a.md | 16 ++++++++-------- _posts/2018-08-06-chou18a.md | 6 +++--- _posts/2018-08-06-cui18a.md | 17 ++++++++--------- _posts/2018-08-06-davidson18a.md | 14 +++++++------- _posts/2018-08-06-de-asis18a.md | 12 ++++++------ _posts/2018-08-06-degenne18a.md | 4 ++-- _posts/2018-08-06-denevi18a.md | 15 +++++++-------- _posts/2018-08-06-derman18a.md | 6 +++--- _posts/2018-08-06-ding18a.md | 12 ++++++------ _posts/2018-08-06-donner18a.md | 6 +++--- _posts/2018-08-06-dvijotham18a.md | 20 ++++++++++---------- _posts/2018-08-06-eismann18a.md | 15 +++++++-------- _posts/2018-08-06-fiez18a.md | 11 +++++------ _posts/2018-08-06-fitzsimons18a.md | 10 +++++----- _posts/2018-08-06-forre18a.md | 14 +++++++------- _posts/2018-08-06-garcia-duran18a.md | 12 ++++++------ _posts/2018-08-06-garg18a.md | 8 ++++---- _posts/2018-08-06-gotovos18a.md | 9 ++++----- _posts/2018-08-06-gregorova18a.md | 12 ++++++------ _posts/2018-08-06-guarnizo18a.md | 12 ++++++------ _posts/2018-08-06-hegde18a.md | 18 +++++++++--------- _posts/2018-08-06-hewitt18a.md | 10 +++++----- _posts/2018-08-06-huang18a.md | 14 +++++++------- _posts/2018-08-06-izmailov18a.md | 14 +++++++------- _posts/2018-08-06-jaber18a.md | 15 +++++++-------- _posts/2018-08-06-junges18a.md | 12 ++++++------ _posts/2018-08-06-kang18a.md | 28 +++++++++++++--------------- _posts/2018-08-06-karan18a.md | 9 ++++----- _posts/2018-08-06-kim18a.md | 17 ++++++++--------- _posts/2018-08-06-krishnan18a.md | 12 ++++++------ _posts/2018-08-06-kyrillidis18a.md | 6 +++--- _posts/2018-08-06-lacey18a.md | 16 +++++++--------- _posts/2018-08-06-lawton18a.md | 10 +++++----- _posts/2018-08-06-le-priol18a.md | 6 +++--- _posts/2018-08-06-lee18a.md | 10 +++++----- _posts/2018-08-06-liu18a.md | 8 ++++---- _posts/2018-08-06-lou18a.md | 10 +++++----- _posts/2018-08-06-lyu18a.md | 14 +++++++------- _posts/2018-08-06-ma18a.md | 8 ++++---- _posts/2018-08-06-madan18a.md | 16 ++++++++-------- _posts/2018-08-06-marchetti18a.md | 7 +++---- _posts/2018-08-06-meisami18a.md | 9 ++++----- _posts/2018-08-06-mogensen18a.md | 16 ++++++++-------- _posts/2018-08-06-monti18a.md | 14 +++++++------- _posts/2018-08-06-moore18a.md | 11 +++++------ _posts/2018-08-06-nabi18a.md | 18 +++++++++--------- _posts/2018-08-06-nair18a.md | 10 +++++----- _posts/2018-08-06-palmer18a.md | 11 +++++------ _posts/2018-08-06-pena18a.md | 6 +++--- _posts/2018-08-06-piatkowski18a.md | 16 ++++++++-------- _posts/2018-08-06-pinot18a.md | 12 ++++++------ _posts/2018-08-06-rainforth18a.md | 14 +++++++------- _posts/2018-08-06-ren18a.md | 18 +++++++++--------- _posts/2018-08-06-rubenstein18a.md | 4 ++-- _posts/2018-08-06-sabharwal18a.md | 19 +++++++++---------- _posts/2018-08-06-saemundsson18a.md | 13 ++++++------- _posts/2018-08-06-saha18a.md | 6 +++--- _posts/2018-08-06-sajed18a.md | 19 +++++++++---------- _posts/2018-08-06-salah18a.md | 18 +++++++++--------- _posts/2018-08-06-seong18a.md | 18 +++++++++--------- _posts/2018-08-06-sharma18a.md | 12 ++++++------ _posts/2018-08-06-sherstan18a.md | 14 +++++++------- _posts/2018-08-06-shilton18a.md | 8 ++++---- _posts/2018-08-06-shpakova18a.md | 9 ++++----- _posts/2018-08-06-shpitser18a.md | 18 +++++++++--------- _posts/2018-08-06-shpitser18b.md | 16 ++++++++-------- _posts/2018-08-06-smith18a.md | 14 +++++++------- _posts/2018-08-06-subbaswamy18a.md | 12 ++++++------ _posts/2018-08-06-tan18a.md | 15 +++++++-------- _posts/2018-08-06-tan18b.md | 16 ++++++++-------- _posts/2018-08-06-turner18a.md | 8 ++++---- _posts/2018-08-06-wang18a.md | 23 +++++++++++------------ _posts/2018-08-06-wang18b.md | 13 ++++++------- _posts/2018-08-06-wang18c.md | 12 ++++++------ _posts/2018-08-06-yu18a.md | 14 +++++++------- _posts/2018-08-06-yuan18a.md | 21 ++++++++++----------- _posts/2018-08-06-zhang18a.md | 10 +++++----- _posts/2018-08-06-zhang18b.md | 14 +++++++------- _posts/2018-08-06-zhang18c.md | 12 ++++++------ _posts/2018-08-06-zhao18a.md | 16 ++++++++-------- _posts/2018-08-06-zhao18b.md | 8 ++++---- _posts/2018-08-06-zhao18c.md | 17 ++++++++--------- _posts/2018-08-06-zheng18a.md | 16 ++++++++-------- _posts/2018-08-06-zheng18b.md | 18 +++++++++--------- _posts/2018-08-06-zhou18a.md | 8 ++++---- _posts/2018-08-06-zhu18a.md | 8 ++++---- _posts/2018-08-06-zou18a.md | 10 +++++----- 101 files changed, 620 insertions(+), 646 deletions(-) diff --git a/_posts/2018-08-06-adams18a.md b/_posts/2018-08-06-adams18a.md index cf5e929..96b6d5f 100644 --- a/_posts/2018-08-06-adams18a.md +++ b/_posts/2018-08-06-adams18a.md @@ -1,14 +1,14 @@ --- abstract: This paper considers the problem of learning time series segmentation models - when the la- beled data are subject to temporal uncertainty or noise. Our approach + when the labeled data are subject to temporal uncertainty or noise. Our approach augments the semi- Markov conditional random field (semi-CRF) model with a probabilistic model of the label observation process. This augmentation allows us to estimate the parameters of the semi-CRF from timestamps corresponding roughly to the occurrence of transitions between segments. We show how exact marginal inference can be performed - in the augmented model in polyno- mial time, enabling learning based on marginal + in the augmented model in polynomial time, enabling learning based on marginal likelihood maximization. Our experiments on two activity detection problems show - that the proposed approach can learn models from tem- porally imprecise labels, - and can successfully refine imprecise segmentations through poste- rior inference. + that the proposed approach can learn models from temporally imprecise labels, + and can successfully refine imprecise segmentations through posterior inference. Finally, we show how inference complexity can be reduced by a factor of 40 using static and model-based pruning of the inference dynamic program. title: Learning Time Series Segmentation Models from Temporally Imprecise Labels diff --git a/_posts/2018-08-06-aditya18a.md b/_posts/2018-08-06-aditya18a.md index 9918111..4b7c275 100644 --- a/_posts/2018-08-06-aditya18a.md +++ b/_posts/2018-08-06-aditya18a.md @@ -1,20 +1,19 @@ --- -abstract: The uncertainty associated with human per- ception is often reduced by one’s +abstract: The uncertainty associated with human perception is often reduced by one’s extensive prior experience and knowledge. Current datasets and systems do not emphasize - the ne- cessity and benefit of using such knowledge. In this work, we propose the + the necessity and benefit of using such knowledge. In this work, we propose the task of solving a genre of image-puzzles (“image riddles”) that require both capabilities - involving visual de- tection (including object, activity recognition) and, knowledge-based - or commonsense rea- soning. Each puzzle involves a set of images and the question + involving visual detection (including object, activity recognition) and, knowledge-based + or commonsense reasoning. Each puzzle involves a set of images and the question “what word connects these images?”. We compile a dataset of over 3k riddles where - each riddle consists of 4 im- ages and a groundtruth answer. The annota- tions are - validated using crowd-sourced eval- uation. We also define an automatic evalua- - tion metric to track future progress. Our task bears similarity with the commonly - known IQ tasks such as analogy solving, sequence fill- ing that are often used to - test intelligence. We develop a Probabilistic Reasoning-based ap- proach that utilizes - commonsense knowledge about words and phrases to answer these rid- dles with a reasonable - accuracy. Our approach achieves some promising results for these rid- dles and provides - a strong baseline for future attempts. We make the entire dataset and re- lated - materials publicly available to the com- munity (bit.ly/22f9Ala). + each riddle consists of 4 images and a groundtruth answer. The annotations are + validated using crowd-sourced evaluation. We also define an automatic evaluation metric to track future progress. Our task bears similarity with the commonly + known IQ tasks such as analogy solving, sequence filling that are often used to + test intelligence. We develop a Probabilistic Reasoning-based approach that utilizes + commonsense knowledge about words and phrases to answer these riddles with a reasonable + accuracy. Our approach achieves some promising results for these riddles and provides + a strong baseline for future attempts. We make the entire dataset and related + materials publicly available to the community (bit.ly/22f9Ala). title: Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving year: '2018' diff --git a/_posts/2018-08-06-agrawal18a.md b/_posts/2018-08-06-agrawal18a.md index 1f3375e..1545917 100644 --- a/_posts/2018-08-06-agrawal18a.md +++ b/_posts/2018-08-06-agrawal18a.md @@ -1,19 +1,19 @@ --- abstract: Decentralized planning under uncertainty for agent teams is a problem of - interest in many domains including (but not limited to) disas- ter rescue, sensor - networks and security pa- trolling. Decentralized MDPs, Dec-MDPs have traditionally - been used to represent such decen- tralized planning under uncertainty problems. + interest in many domains including (but not limited to) disaster rescue, sensor + networks and security patrolling. Decentralized MDPs, Dec-MDPs have traditionally + been used to represent such decentralized planning under uncertainty problems. However, in many domains, agents may not be dedicated to the team for the entire - time horizon. For instance, due to limited availabil- ity of resources, it is quite + time horizon. For instance, due to limited availability of resources, it is quite common for police personnel leaving patrolling teams to attend to accidents. Such non-dedication can arise due to the emergence of higher priority tasks or damage to existing agents. However, there is very limited literature dealing with handling of non-dedication in decentralized settings. To that end, we provide a general model - to rep- resent problems dealing with cooperative and decentralized planning for + to represent problems dealing with cooperative and decentralized planning for non-dedicated agent teams. We also provide two greedy approaches (an offline one and an offline-online one) that are able to deal with agents leaving the team in an effective and efficient way by exploiting the submodularity property. Finally, - we demon- strate that our approaches are able to obtain more than 90% of optimal + we demonstrate that our approaches are able to obtain more than 90% of optimal solution quality on benchmark problems from the literature. title: Decentralized Planning for Non-dedicated Agent Teams with Submodular Rewards in Uncertain Environments diff --git a/_posts/2018-08-06-atzmon18a.md b/_posts/2018-08-06-atzmon18a.md index f21fb33..bd1aefe 100644 --- a/_posts/2018-08-06-atzmon18a.md +++ b/_posts/2018-08-06-atzmon18a.md @@ -1,18 +1,18 @@ --- abstract: 'In zero-shot learning (ZSL), a classifier is trained to recognize visual - classes without any image samples. Instead, it is given seman- tic information about + classes without any image samples. Instead, it is given semantic information about the class, like a textual description or a set of attributes. Learning from attributes - could benefit from explicitly modeling structure of the attribute space. Un- fortunately, + could benefit from explicitly modeling structure of the attribute space. Unfortunately, learning of general structure from empirical samples is hard with typical dataset sizes. Here we describe LAGO1, a probabilistic model designed to capture natural - soft and- or relations across groups of attributes. We show how this model can be - learned end-to- end with a deep attribute-detection model. The soft group structure - can be learned from data jointly as part of the model, and can also read- ily incorporate - prior knowledge about groups if available. The soft and-or structure suc- ceeds + soft and or relations across groups of attributes. We show how this model can be + learned end-to-end with a deep attribute-detection model. The soft group structure + can be learned from data jointly as part of the model, and can also readily incorporate + prior knowledge about groups if available. The soft and-or structure succeeds to capture meaningful and predictive structures, improving the accuracy of zero-shot learning on two of three benchmarks. Finally, LAGO reveals a unified formulation over two ZSL approaches: DAP (Lampert et al., 2009) and ESZSL (Romera-Paredes & - Torr, 2015). Interestingly, taking only one sin- gleton group for each attribute, + Torr, 2015). Interestingly, taking only one singleton group for each attribute, introduces a new soft-relaxation of DAP, that outperforms DAP by $\tilde$40%.' title: Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning year: '2018' diff --git a/_posts/2018-08-06-babichev18a.md b/_posts/2018-08-06-babichev18a.md index 69f057f..f6c14ad 100644 --- a/_posts/2018-08-06-babichev18a.md +++ b/_posts/2018-08-06-babichev18a.md @@ -1,9 +1,9 @@ --- abstract: Stochastic gradient methods enable learning probabilistic models from large amounts of data. While large step-sizes (learning rates) have shown to be best for - least-squares (e.g., Gaussian noise) once combined with param- eter averaging, these - are not leading to con- vergent algorithms in general. In this pa- per, we consider - generalized linear models, that is, conditional models based on exponen- tial families. + least-squares (e.g., Gaussian noise) once combined with parameter averaging, these + are not leading to convergent algorithms in general. In this paper, we consider + generalized linear models, that is, conditional models based on exponential families. We propose averaging moment parameters instead of natural parameters for constant-step-size stochastic gradient descent. For finite-dimensional models, we show that this can sometimes (and surprisingly) lead to better predictions than the best linear model. diff --git a/_posts/2018-08-06-berg18a.md b/_posts/2018-08-06-berg18a.md index 722c994..1fd769f 100644 --- a/_posts/2018-08-06-berg18a.md +++ b/_posts/2018-08-06-berg18a.md @@ -1,10 +1,10 @@ --- -abstract: Variational inference relies on flexible ap- proximate posterior distributions. - Normaliz- ing flows provide a general recipe to con- struct flexible variational - posteriors. We in- troduce Sylvester normalizing flows, which can be seen as a generalization +abstract: Variational inference relies on flexible approximate posterior distributions. + Normalizing flows provide a general recipe to construct flexible variational + posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single-unit bottleneck from planar flows, making a single transformation much more flexible. We compare - the performance of Sylvester normalizing flows against pla- nar flows and inverse + the performance of Sylvester normalizing flows against planar flows and inverse autoregressive flows and demonstrate that they compare favorably on several datasets. title: Sylvester Normalizing Flows for Variational Inference year: '2018' diff --git a/_posts/2018-08-06-bixler18a.md b/_posts/2018-08-06-bixler18a.md index a9d3944..25f854f 100644 --- a/_posts/2018-08-06-bixler18a.md +++ b/_posts/2018-08-06-bixler18a.md @@ -1,12 +1,12 @@ --- abstract: We propose sparse-matrix belief propagation, which executes loopy belief - propagation in pair- wise Markov random fields by replacing in- dexing over graph - neighborhoods with sparse- matrix operations. This abstraction allows for seamless - integration with optimized sparse lin- ear algebra libraries, including those that - per- form matrix and tensor operations on modern hardware such as graphical processing + propagation in pairwise Markov random fields by replacing indexing over graph + neighborhoods with sparse-matrix operations. This abstraction allows for seamless + integration with optimized sparse linear algebra libraries, including those that + per-form matrix and tensor operations on modern hardware such as graphical processing units (GPUs). The sparse-matrix abstraction allows the implementation of belief propagation in a high-level language (e.g., Python) that is also able to leverage - the power of GPU paralleliza- tion. We demonstrate sparse-matrix belief prop- agation + the power of GPU parallelization. We demonstrate sparse-matrix belief propagation by implementing it in a modern deep learning framework (PyTorch), measuring the resulting massive improvement in running time, and facilitating future integration into deep learning models. diff --git a/_posts/2018-08-06-bloem-reddy18a.md b/_posts/2018-08-06-bloem-reddy18a.md index 23238a7..88bc7c9 100644 --- a/_posts/2018-08-06-bloem-reddy18a.md +++ b/_posts/2018-08-06-bloem-reddy18a.md @@ -1,17 +1,17 @@ --- abstract: Empirical evidence suggests that heavy-tailed degree distributions occurring - in many real net- works are well-approximated by power laws with exponents $\eta$ + in many real networks are well-approximated by power laws with exponents $\eta$ that may take values either less than and greater than two. Models based on various forms of exchangeability are able to capture power laws with $\eta$ < 2, and admit - tractable inference algorithms; we draw on pre- vious results to show that $\eta$ - > 2 cannot be gen- erated by the forms of exchangeability used in existing random - graph models. Preferential at- tachment models generate power law exponents greater + tractable inference algorithms; we draw on previous results to show that $\eta$ + > 2 cannot be generated by the forms of exchangeability used in existing random + graph models. Preferential attachment models generate power law exponents greater than two, but have been of limited use as statistical models due to the inherent difficulty of performing inference in non-exchangeable models. Motivated by this gap, we design and implement inference algorithms for a recently proposed class of models that generates $\eta$ of all possible values. We show that although they - are not exchangeable, these models have prob- abilistic structure amenable to inference. - Our methods make a large class of previously in- tractable models useful for statistical + are not exchangeable, these models have probabilistic structure amenable to inference. + Our methods make a large class of previously intractable models useful for statistical inference. title: Sampling and Inference for Beta Neutral-to-the-Left Models of Sparse Networks year: '2018' diff --git a/_posts/2018-08-06-blom18a.md b/_posts/2018-08-06-blom18a.md index cf398fb..e141fc4 100644 --- a/_posts/2018-08-06-blom18a.md +++ b/_posts/2018-08-06-blom18a.md @@ -1,10 +1,10 @@ --- -abstract: Causal discovery algorithms infer causal re- lations from data based on - several assump- tions, including notably the absence of mea- surement error. However, +abstract: Causal discovery algorithms infer causal relations from data based on + several assumptions, including notably the absence of measurement error. However, this assumption is most likely violated in practical applications, which may result in erroneous, irreproducible results. In this work we show how to obtain an upper - bound for the variance of random mea- surement error from the covariance matrix - of measured variables and how to use this up- per bound as a correction for constraint-based + bound for the variance of random measurement error from the covariance matrix + of measured variables and how to use this upper bound as a correction for constraint-based causal discovery. We demonstrate a practical application of our approach on both simulated data and real-world protein signaling data. title: Causal Discovery in the Presence of Measurement Error diff --git a/_posts/2018-08-06-broka18a.md b/_posts/2018-08-06-broka18a.md index bf887e4..a198bf4 100644 --- a/_posts/2018-08-06-broka18a.md +++ b/_posts/2018-08-06-broka18a.md @@ -1,13 +1,13 @@ --- -abstract: We present a new sampling scheme for approx- imating hard to compute queries - over graphical models, such as computing the partition func- tion. The scheme builds +abstract: We present a new sampling scheme for approximating hard to compute queries + over graphical models, such as computing the partition function. The scheme builds upon exact algorithms that traverse a weighted directed state-space graph representing a global function over a graphical model (e.g., probability distribution). With - the aid of an abstraction function and ran- domization, the state space can be compacted - (or trimmed) to facilitate tractable computa- tion, yielding a Monte Carlo Estimate + the aid of an abstraction function and randomization, the state space can be compacted + (or trimmed) to facilitate tractable computation, yielding a Monte Carlo Estimate that is unbiased. We present the general scheme and analyze its properties analytically - and empiri- cally, investigating two specific ideas for pick- ing abstractions - - targeting reduction of vari- ance or search space size. + and empirically, investigating two specific ideas for picking abstractions - + targeting reduction of variance or search space size. title: Abstraction Sampling in Graphical Models year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-callaway18a.md b/_posts/2018-08-06-callaway18a.md index 8b4cf6f..1cbcc09 100644 --- a/_posts/2018-08-06-callaway18a.md +++ b/_posts/2018-08-06-callaway18a.md @@ -1,16 +1,16 @@ --- -abstract: 'The efficient use of limited computational re- sources is an essential - ingredient of intel- ligence. Selecting computations optimally according to rational - metareasoning would achieve this, but this is computationally in- tractable. Inspired - by psychology and neu- roscience, we propose the first concrete and domain-general - learning algorithm for approx- imating the optimal selection of computations: Bayesian +abstract: 'The efficient use of limited computational resources is an essential + ingredient of intelligence. Selecting computations optimally according to rational + metareasoning would achieve this, but this is computationally intractable. Inspired + by psychology and neuroscience, we propose the first concrete and domain-general + learning algorithm for approximating the optimal selection of computations: Bayesian metalevel policy search (BMPS). We derive this general, sample-efficient search - al- gorithm for a computation-selecting metalevel policy based on the insight that + algorithm for a computation-selecting metalevel policy based on the insight that the value of information lies between the myopic value of information and the value - of perfect in- formation. We evaluate BMPS on three in- creasingly difficult metareasoning - problems: when to terminate computation, how to allo- cate computation between competing + of perfect information. We evaluate BMPS on three increasingly difficult metareasoning + problems: when to terminate computation, how to allocate computation between competing options, and planning. Across all three domains, BMPS achieved near-optimal performance - and com- pared favorably to previously proposed metar- easoning heuristics. Finally, + and compared favorably to previously proposed metareasoning heuristics. Finally, we demonstrate the practical utility of BMPS in an emergency management scenario, even accounting for the overhead of metareasoning.' title: Learning to select computations diff --git a/_posts/2018-08-06-chaudhuri18a.md b/_posts/2018-08-06-chaudhuri18a.md index 2f26560..2d28679 100644 --- a/_posts/2018-08-06-chaudhuri18a.md +++ b/_posts/2018-08-06-chaudhuri18a.md @@ -1,10 +1,10 @@ --- -abstract: 'The stochastic multi-armed bandit is a well- studied abstraction of decision +abstract: 'The stochastic multi-armed bandit is a wellstudied abstraction of decision making in the face of uncertainty. We consider the setting in which the number of bandit arms is much larger than the possible number of pulls, and can even be infinite. With the aim of minimising regret with respect to an optimal arm, existing methods - for this set- ting either assume some structure over the set of arms (Kleinberg - et al., 2008, Ray Chowdhury and Gopalan, 2017), or some property of the re- ward + for this setting either assume some structure over the set of arms (Kleinberg + et al., 2008, Ray Chowdhury and Gopalan, 2017), or some property of the reward distribution (Wang et al., 2008). Invariably, the validity of such assumptions—and therefore the performance of the corresponding methods— depends on instance-specific parameters, which might not be known beforehand. We propose a conceptually simple, @@ -14,7 +14,7 @@ abstract: 'The stochastic multi-armed bandit is a well- studied abstraction of d that achieves sublinear “quantile-regret”, both (1) when it is specified a quantile, and (2) when the quantile can be any (unknown) positive value. The algorithm needs no side information about the arms or about the structure of their reward distributions: - it re- lies on random sampling to reach arms in the top quantile. Experiments show + it relies on random sampling to reach arms in the top quantile. Experiments show that our algorithm outperforms several previous methods (in terms of conventional regret) when the latter are not tuned well, and often even when they are.' title: Quantile-Regret Minimisation in Infinitely Many-Armed Bandits diff --git a/_posts/2018-08-06-chen18a.md b/_posts/2018-08-06-chen18a.md index f2a66db..1530b2f 100644 --- a/_posts/2018-08-06-chen18a.md +++ b/_posts/2018-08-06-chen18a.md @@ -1,19 +1,19 @@ --- abstract: There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient - descent (SVGD) for big-data analysis. A standard SG-MCMC algo- rithm simulates samples - from a discrete-time Markov chain to approximate a target distribu- tion, thus samples - could be highly correlated, an undesired property for SG-MCMC. In con- trary, SVGD + descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples + from a discrete-time Markov chain to approximate a target distribution, thus samples + could be highly correlated, an undesired property for SG-MCMC. In contrary, SVGD directly optimizes a set of particles to approximate a target distribution, and - thus is able to obtain good approximations with rela- tively much fewer samples. - In this paper, we propose a principle particle-optimization frame- work based on + thus is able to obtain good approximations with relatively much fewer samples. + In this paper, we propose a principle particle-optimization framework based on Wasserstein gradient flows to unify SG-MCMC and SVGD, and to allow new algorithms to be developed. Our framework interprets SG-MCMC as particle optimization on the space of probability measures, revealing a strong connection between SG-MCMC and SVGD. The key component of our framework is several particle-approximate techniques to efficiently solve the original partial differential equations on the space of probability measures. Extensive experiments on both synthetic data and deep neural - networks demonstrate the ef- fectiveness and efficiency of our framework for scalable + networks demonstrate the effectiveness and efficiency of our framework for scalable Bayesian sampling. title: A Unified Particle-Optimization Framework for Scalable Bayesian Sampling year: '2018' diff --git a/_posts/2018-08-06-chen18b.md b/_posts/2018-08-06-chen18b.md index eb3d837..7e02b6a 100644 --- a/_posts/2018-08-06-chen18b.md +++ b/_posts/2018-08-06-chen18b.md @@ -1,13 +1,13 @@ --- -abstract: WTA (Winner Take All) hashing has been suc- cessfully applied in many large-scale - vision applications. This hashing scheme was tai- lored to take advantage of the - comparative rea- soning (or order based information), which showed significant accuracy +abstract: WTA (Winner Take All) hashing has been successfully applied in many large-scale + vision applications. This hashing scheme was tailored to take advantage of the + comparative reasoning (or order based information), which showed significant accuracy improvements. In this paper, we identify a subtle issue with WTA, which grows with the sparsity of the datasets. This issue limits the discriminative power of WTA. We then propose a solution to this problem based on the idea of Densification which makes use of 2-universal hash functions in a novel way. Our experiments show that Densified WTA Hashing outperforms Vanilla WTA Hashing both in image retrieval and - clas- sification tasks consistently and significantly. + classification tasks consistently and significantly. title: Densified Winner Take All (WTA) Hashing for Sparse Datasets year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-cheng18a.md b/_posts/2018-08-06-cheng18a.md index e241247..6106984 100644 --- a/_posts/2018-08-06-cheng18a.md +++ b/_posts/2018-08-06-cheng18a.md @@ -1,16 +1,16 @@ --- abstract: Imitation learning (IL) consists of a set of tools that leverage expert - demonstrations to quickly learn policies. However, if the expert is subop- timal, - IL can yield policies with inferior per- formance compared to reinforcement learning - (RL). In this paper, we aim to provide an algo- rithm that combines the best aspects - of RL and IL. We accomplish this by formulating sev- eral popular RL and IL algorithms - in a com- mon mirror descent framework, showing that these algorithms can be viewed + demonstrations to quickly learn policies. However, if the expert is suboptimal, + IL can yield policies with inferior performance compared to reinforcement learning + (RL). In this paper, we aim to provide an algorithm that combines the best aspects + of RL and IL. We accomplish this by formulating several popular RL and IL algorithms + in a common mirror descent framework, showing that these algorithms can be viewed as a variation on a single approach. We then propose LOKI, a strategy for policy - learning that first performs a small but random number of IL iterations be- fore + learning that first performs a small but random number of IL iterations before switching to a policy gradient RL method. We show that if the switching time is - prop- erly randomized, LOKI can learn to outperform a suboptimal expert and converge + properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch. Finally, we evaluate the performance - of LOKI experi- mentally in several simulated environments. + of LOKI experimentally in several simulated environments. title: Fast Policy Learning through Imitation and Reinforcement year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-chou18a.md b/_posts/2018-08-06-chou18a.md index a329f90..4af2557 100644 --- a/_posts/2018-08-06-chou18a.md +++ b/_posts/2018-08-06-chou18a.md @@ -1,12 +1,12 @@ --- abstract: We consider the weighted model counting task which includes important tasks - in graphical models, such as computing the partition func- tion and probability - of evidence as special cases. We propose a novel partition-based bounding al- gorithm + in graphical models, such as computing the partition function and probability + of evidence as special cases. We propose a novel partition-based bounding algorithm that exploits logical structure and gives rise to a set of inequalities from which upper (or lower) bounds can be derived efficiently. The bounds come with optimality guarantees under certain conditions and are oblivious in that they require only limited observations of the structure and parameters of the problem. We experimentally - compare our bounds with the mini-bucket scheme (which is also oblivi- ous) and show + compare our bounds with the mini-bucket scheme (which is also oblivious) and show that our new bounds are often superior and never worse on a wide variety of benchmark networks. title: Dissociation-Based Oblivious Bounds for Weighted Model Counting diff --git a/_posts/2018-08-06-cui18a.md b/_posts/2018-08-06-cui18a.md index cc55115..89e943d 100644 --- a/_posts/2018-08-06-cui18a.md +++ b/_posts/2018-08-06-cui18a.md @@ -1,18 +1,17 @@ --- abstract: A common goal in psychometrics, sociology, and econometrics is to uncover - causal rela- tions among latent variables representing hy- pothetical constructs + causal relations among latent variables representing hypothetical constructs that cannot be measured directly, such as attitude, intelligence, and motivation. - Through measurement models, these constructs are typically linked to mea- surable - indicators, e.g., responses to question- naire items. This paper addresses the prob- - lem of causal structure learning among such la- tent variables and other observed + Through measurement models, these constructs are typically linked to measurable + indicators, e.g., responses to questionnaire items. This paper addresses the problem of causal structure learning among such latent variables and other observed variables. We propose the ‘Copula Factor PC’ algorithm as a novel two-step approach. - It first draws samples of the underlying correlation matrix in a Gaus- sian copula + It first draws samples of the underlying correlation matrix in a Gaussian copula factor model via a Gibbs sampler on rank-based data. These are then translated into - an average correlation matrix and an ef- fective sample size, which are taken as + an average correlation matrix and an effective sample size, which are taken as input to the standard PC algorithm for causal discovery in the second step. We prove - the consistency of our ‘Copula Factor PC’ algorithm, and demon- strate that it outperforms - the PC-MIMBuild al- gorithm and a greedy step-wise approach. We illustrate our method - on a real-world data set about children with Attention Deficit Hyperac- tivity Disorder. + the consistency of our ‘Copula Factor PC’ algorithm, and demonstrate that it outperforms + the PC-MIMBuild algorithm and a greedy step-wise approach. We illustrate our method + on a real-world data set about children with Attention Deficit Hyperactivity Disorder. title: Learning the Causal Structure of Copula Models with Latent Variables year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-davidson18a.md b/_posts/2018-08-06-davidson18a.md index 4d27783..f55c520 100644 --- a/_posts/2018-08-06-davidson18a.md +++ b/_posts/2018-08-06-davidson18a.md @@ -1,13 +1,13 @@ --- abstract: The Variational Auto-Encoder (VAE) is one of the most used unsupervised - machine learn- ing models. But although the default choice of a Gaussian distribution - for both the prior and posterior represents a mathematically con- venient distribution - often leading to competi- tive results, we show that this parameterization fails - to model data with a latent hyperspheri- cal structure. To address this issue we - propose using a von Mises-Fisher (vMF) distribution in- stead, leading to a hyperspherical + machine learning models. But although the default choice of a Gaussian distribution + for both the prior and posterior represents a mathematically convenient distribution + often leading to competitive results, we show that this parameterization fails + to model data with a latent hyperspherical structure. To address this issue we + propose using a von Mises-Fisher (vMF) distribution instead, leading to a hyperspherical latent space. Through a series of experiments we show how such a hyperspherical - VAE, or S-VAE, is more suitable for capturing data with a hyperspheri- cal latent - structure, while outperforming a nor- mal, N-VAE, in low dimensions on other data + VAE, or S-VAE, is more suitable for capturing data with a hyperspherical latent + structure, while outperforming a normal, N-VAE, in low dimensions on other data types. title: Hyperspherical Variational Auto-Encoders year: '2018' diff --git a/_posts/2018-08-06-de-asis18a.md b/_posts/2018-08-06-de-asis18a.md index b15570f..33af525 100644 --- a/_posts/2018-08-06-de-asis18a.md +++ b/_posts/2018-08-06-de-asis18a.md @@ -1,15 +1,15 @@ --- abstract: Multi-step temporal difference (TD) learning is an important approach in reinforcement learning, as it unifies one-step TD learning with Monte Carlo methods - in a way where intermediate algorithms can outperform ei- ther extreme. They address + in a way where intermediate algorithms can outperform either extreme. They address a bias-variance trade off between reliance on current estimates, which could be poor, and incorporating longer sampled reward sequences into the updates. Especially in the off-policy setting, where the agent aims to learn about a policy different - from the one generating its behaviour, the vari- ance in the updates can cause learning - to di- verge as the number of sampled rewards used in the estimates increases. In - this paper, we in- troduce per-decision control variates for multi- step TD algorithms, - and compare them to ex- isting methods. Our results show that includ- ing the control - variates can greatly improve performance on both on and off-policy multi- step temporal + from the one generating its behaviour, the variance in the updates can cause learning + to diverge as the number of sampled rewards used in the estimates increases. In + this paper, we introduce per-decision control variates for multi-step TD algorithms, + and compare them to existing methods. Our results show that including the control + variates can greatly improve performance on both on and off-policy multi-step temporal difference learning tasks. title: Per-decision Multi-step Temporal Difference Learning with Control Variates year: '2018' diff --git a/_posts/2018-08-06-degenne18a.md b/_posts/2018-08-06-degenne18a.md index 0117127..319a6ba 100644 --- a/_posts/2018-08-06-degenne18a.md +++ b/_posts/2018-08-06-degenne18a.md @@ -1,9 +1,9 @@ --- -abstract: We consider the classical stochastic multi- armed bandit but where, from +abstract: We consider the classical stochastic multi-armed bandit but where, from time to time and roughly with frequency $\epsilon$, an extra observation is gathered by the agent for free. We prove that, no matter how small $\epsilon$ is the agent can ensure a regret uniformly bounded in time. More precisely, we construct an algorithm - with a regret smaller than P i log(1/$\epsilon$) $\Delta$i , up to multi- plicative + with a regret smaller than P i log(1/$\epsilon$) $\Delta$i , up to multiplicative constant and log log terms. We also prove a matching lower-bound, stating that no reasonable algorithm can outperform this quantity. title: 'Bandits with Side Observations: Bounded vs. Logarithmic Regret' diff --git a/_posts/2018-08-06-denevi18a.md b/_posts/2018-08-06-denevi18a.md index 61bbdba..f896a47 100644 --- a/_posts/2018-08-06-denevi18a.md +++ b/_posts/2018-08-06-denevi18a.md @@ -1,18 +1,17 @@ --- abstract: In learning-to-learn the goal is to infer a learning algorithm that works - well on a class of tasks sampled from an unknown meta- distribution. In contrast + well on a class of tasks sampled from an unknown meta-distribution. In contrast to previous work on batch learning-to-learn, we consider a scenario where tasks - are presented sequentially and the algorithm needs to adapt incrementally to im- - prove its performance on future tasks. Key to this setting is for the algorithm - to rapidly in- corporate new observations into the model as they arrive, without - keeping them in memory. We focus on the case where the underlying al- gorithm is + are presented sequentially and the algorithm needs to adapt incrementally to improve its performance on future tasks. Key to this setting is for the algorithm + to rapidly incorporate new observations into the model as they arrive, without + keeping them in memory. We focus on the case where the underlying algorithm is Ridge Regression parametrised by a symmetric positive semidefinite matrix. We propose to learn this matrix by applying a stochastic strategy to minimize the empirical error incurred by Ridge Regression on future tasks sampled from the meta-distribution. We study the statistical properties of the proposed algorithm and prove non-asymptotic - bounds on its excess transfer risk, that is, the gener- alization performance on - new tasks from the same meta-distribution. We compare our on- line learning-to-learn - approach with a state-of- the-art batch method, both theoretically and empirically. + bounds on its excess transfer risk, that is, the generalization performance on + new tasks from the same meta-distribution. We compare our online learning-to-learn + approach with a state-of-the-art batch method, both theoretically and empirically. title: Incremental Learning-to-Learn with Statistical Guarantees year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-derman18a.md b/_posts/2018-08-06-derman18a.md index 286f6a2..277d79c 100644 --- a/_posts/2018-08-06-derman18a.md +++ b/_posts/2018-08-06-derman18a.md @@ -1,12 +1,12 @@ --- abstract: Robust Reinforcement Learning aims to derive an optimal behavior that accounts - for model un- certainty in dynamical systems. However, pre- vious studies have shown + for model uncertainty in dynamical systems. However, previous studies have shown that by considering the worst case scenario, robust policies can be overly conservative. Our soft-robust framework is an attempt to overcome this issue. In this paper, we present a novel Soft-Robust Actor- Critic algorithm (SR-AC). It learns an optimal policy with respect to a distribution over an uncertainty set and stays robust to - model uncer- tainty but avoids the conservativeness of robust strategies. We show - the convergence of SR-AC and test the efficiency of our approach on dif- ferent + model uncertainty but avoids the conservativeness of robust strategies. We show + the convergence of SR-AC and test the efficiency of our approach on different domains by comparing it against regular learning methods and their robust formulations. title: Soft-Robust Actor-Critic Policy-Gradient year: '2018' diff --git a/_posts/2018-08-06-ding18a.md b/_posts/2018-08-06-ding18a.md index fa70bab..188eed7 100644 --- a/_posts/2018-08-06-ding18a.md +++ b/_posts/2018-08-06-ding18a.md @@ -1,13 +1,13 @@ --- -abstract: We present the first framework for Gaussian- process-modulated Poisson processes +abstract: We present the first framework for Gaussian-process-modulated Poisson processes when the temporal data appear in the form of panel counts. Panel count data frequently - arise when experimental subjects are observed only at dis- crete time points and - only the numbers of oc- currences of the events between subsequent observation times - are available. The exact occurrence timestamps of the events are un- known. The + arise when experimental subjects are observed only at discrete time points and + only the numbers of occurrences of the events between subsequent observation times + are available. The exact occurrence timestamps of the events are un-known. The method of conducting the efficient variational inference is presented, based on the assumption of a Gaussian-process-modulated intensity function. We derive a tractable - lower bound to alleviate the problems of the in- tractable evidence lower bound - inherent in the variational inference framework. Our algo- rithm outperforms classical + lower bound to alleviate the problems of the intractable evidence lower bound + inherent in the variational inference framework. Our algorithm outperforms classical methods on both synthetic and three real panel count sets. title: Variational Inference for Gaussian Processes with Panel Count Data year: '2018' diff --git a/_posts/2018-08-06-donner18a.md b/_posts/2018-08-06-donner18a.md index 7c45d46..7470a69 100644 --- a/_posts/2018-08-06-donner18a.md +++ b/_posts/2018-08-06-donner18a.md @@ -2,11 +2,11 @@ abstract: We reconsider a nonparametric density model based on Gaussian processes. By augmenting the model with latent P{ó}lya–Gamma random variables and a latent marked Poisson process we obtain a new likelihood which is conjugate to the model’s - Gaussian process prior. The augmented posterior allows for efficient infer- ence + Gaussian process prior. The augmented posterior allows for efficient inference by Gibbs sampling and an approximate variational mean field approach. For the latter we utilise sparse GP approximations to tackle the infinite dimensionality of the - problem. The performance of both algorithms and compar- isons with other density - estimators are demon- strated on artificial and real datasets with up to several + problem. The performance of both algorithms and comparisons with other density + estimators are demonstrated on artificial and real datasets with up to several thousand data points. title: Efficient Bayesian Inference for a Gaussian Process Density Model year: '2018' diff --git a/_posts/2018-08-06-dvijotham18a.md b/_posts/2018-08-06-dvijotham18a.md index d5b89c7..c10c36d 100644 --- a/_posts/2018-08-06-dvijotham18a.md +++ b/_posts/2018-08-06-dvijotham18a.md @@ -1,18 +1,18 @@ --- abstract: This paper addresses the problem of formally verifying desirable properties - of neural net- works, i.e., obtaining provable guarantees that neural networks satisfy + of neural networks, i.e., obtaining provable guarantees that neural networks satisfy specifications relating their inputs and outputs (robustness to bounded norm adversarial - perturbations, for example). Most previous work on this topic was lim- ited in its - applicability by the size of the net- work, network architecture and the complexity + perturbations, for example). Most previous work on this topic was limited in its + applicability by the size of the network, network architecture and the complexity of properties to be verified. In contrast, our framework applies to a general class - of activa- tion functions and specifications on neural net- work inputs and outputs. - We formulate verifi- cation as an optimization problem (seeking to find the largest - violation of the specification) and solve a Lagrangian relaxation of the opti- mization + of activation functions and specifications on neural network inputs and outputs. + We formulate verification as an optimization problem (seeking to find the largest + violation of the specification) and solve a Lagrangian relaxation of the optimization problem to obtain an upper bound on the worst case violation of the specification - be- ing verified. Our approach is anytime i.e. it can be stopped at any time and - a valid bound on the maximum violation can be obtained. We de- velop specialized - verification algorithms with provable tightness guarantees under special as- sumptions - and demonstrate the practical sig- nificance of our general verification approach + being verified. Our approach is anytime i.e. it can be stopped at any time and + a valid bound on the maximum violation can be obtained. We develop specialized + verification algorithms with provable tightness guarantees under special assumptions + and demonstrate the practical significance of our general verification approach on a variety of verification tasks. title: A Dual Approach to Scalable Verification of Deep Networks year: '2018' diff --git a/_posts/2018-08-06-eismann18a.md b/_posts/2018-08-06-eismann18a.md index 5aa2d6b..a287eeb 100644 --- a/_posts/2018-08-06-eismann18a.md +++ b/_posts/2018-08-06-eismann18a.md @@ -1,15 +1,14 @@ --- abstract: Automatic design via Bayesian optimization holds great promise given the constant increase of available data across domains. However, it faces difficulties - from high-dimensional, poten- tially discrete, search spaces. We propose to probabilistically - embed inputs into a lower di- mensional, continuous latent space, where we perform + from high-dimensional, potentially discrete, search spaces. We propose to probabilistically + embed inputs into a lower dimensional, continuous latent space, where we perform gradient-based optimization guided by a Gaussian process. Building on variational - au- toncoders, we use both labeled and unlabeled data to guide the encoding and - increase its ac- curacy. In addition, we propose an adversar- ial extension to render - the latent representa- tion invariant with respect to specific design attributes, - which allows us to transfer these at- tributes across structures. We apply the frame- - work both to a functional-protein dataset and to perform optimization of drag coefficients - di- rectly over high-dimensional shapes without in- corporating domain knowledge + autoncoders, we use both labeled and unlabeled data to guide the encoding and + increase its accuracy. In addition, we propose an adversarial extension to render + the latent representation invariant with respect to specific design attributes, + which allows us to transfer these attributes across structures. We apply the framework both to a functional-protein dataset and to perform optimization of drag coefficients + directly over high-dimensional shapes without incorporating domain knowledge or handcrafted features. title: Bayesian optimization and attribute adjustment year: '2018' diff --git a/_posts/2018-08-06-fiez18a.md b/_posts/2018-08-06-fiez18a.md index 2954625..08d9bb5 100644 --- a/_posts/2018-08-06-fiez18a.md +++ b/_posts/2018-08-06-fiez18a.md @@ -1,13 +1,12 @@ --- -abstract: 'The design of personalized incentives or rec- ommendations to improve user +abstract: 'The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge. - We propose a multi-armed bandit framework for match- ing incentives to users, whose + We propose a multi-armed bandit framework for matching incentives to users, whose preferences are unknown a priori and evolving dynamically in time, in a resource - constrained environ- ment. We design an algorithm that com- bines ideas from three + constrained environment. We design an algorithm that combines ideas from three distinct domains: (i) a greedy matching paradigm, (ii) the upper confidence bound - algorithm (UCB) for ban- dits, and (iii) mixing times from the theory of Markov - chains. For this algorithm, we provide theoretical bounds on the regret and demon- - strate its performance via both synthetic and realistic (matching supply and demand + algorithm (UCB) for bandits, and (iii) mixing times from the theory of Markov + chains. For this algorithm, we provide theoretical bounds on the regret and demonstrate its performance via both synthetic and realistic (matching supply and demand in a bike-sharing platform) examples.' title: Combinatorial Bandits for Incentivizing Agents with Dynamic Preferences year: '2018' diff --git a/_posts/2018-08-06-fitzsimons18a.md b/_posts/2018-08-06-fitzsimons18a.md index c2ec42a..7b39dc8 100644 --- a/_posts/2018-08-06-fitzsimons18a.md +++ b/_posts/2018-08-06-fitzsimons18a.md @@ -1,12 +1,12 @@ --- abstract: The paper begins by introducing the definition and construction of mutually unbiased bases, which are a widely used concept in quantum information processing - but have received lit- tle to no attention in the machine learning and statistics - literature. We demonstrate their use- fulness by using them to create a new sampling + but have received little to no attention in the machine learning and statistics + literature. We demonstrate their usefulness by using them to create a new sampling technique which offers an improvement on the previously well established bounds - of stochas- tic trace estimation. This approach offers a new state of the art single - shot sampling vari- ance while requiring O(log(n)) random bits for x $\in$Rn which - significantly improves on traditional methods such as fixed basis meth- ods, Hutchinson’s + of stochastic trace estimation. This approach offers a new state of the art single + shot sampling variance while requiring O(log(n)) random bits for x $\in$Rn which + significantly improves on traditional methods such as fixed basis methods, Hutchinson’s and Gaussian estimators in terms of the number of random bits required and worst case sample variance. title: Improved Stochastic Trace Estimation using Mutually Unbiased Bases diff --git a/_posts/2018-08-06-forre18a.md b/_posts/2018-08-06-forre18a.md index a474fb8..6fb73f3 100644 --- a/_posts/2018-08-06-forre18a.md +++ b/_posts/2018-08-06-forre18a.md @@ -1,17 +1,17 @@ --- abstract: We address the problem of causal discovery from data, making use of the - recently pro- posed causal modeling framework of modu- lar structural causal models + recently proposed causal modeling framework of modular structural causal models (mSCM) to handle cycles, latent confounders and non-linearities. We introduce $\sigma$-connection - graphs ($\sigma$-CG), a new class of mixed graphs (containing undi- rected, bidirected - and directed edges) with ad- ditional structure, and extend the concept of $\sigma$-separation, + graphs ($\sigma$-CG), a new class of mixed graphs (containing undirected, bidirected + and directed edges) with additional structure, and extend the concept of $\sigma$-separation, the appropriate generalization of the well-known notion of d-separation in this setting, to apply to $\sigma$-CGs. We prove the closedness of $\sigma$-separation - under marginalisa- tion and conditioning and exploit this to im- plement a test + under marginalisation and conditioning and exploit this to implement a test of $\sigma$-separation on a $\sigma$-CG. This then leads us to the first causal - discovery algo- rithm that can handle non-linear functional re- lations, latent - confounders, cyclic causal rela- tionships, and data from different (stochastic) + discovery algorithm that can handle non-linear functional relations, latent + confounders, cyclic causal relationships, and data from different (stochastic) perfect interventions. As a proof of concept, we show on synthetic data how well - the algo- rithm recovers features of the causal graph of modular structural causal + the algorithm recovers features of the causal graph of modular structural causal models. title: Constraint-based Causal Discovery for Non-Linear Structural Causal Models with Cycles and Latent Confounders diff --git a/_posts/2018-08-06-garcia-duran18a.md b/_posts/2018-08-06-garcia-duran18a.md index fc4bb86..1d5688e 100644 --- a/_posts/2018-08-06-garcia-duran18a.md +++ b/_posts/2018-08-06-garcia-duran18a.md @@ -1,13 +1,13 @@ --- -abstract: We present KBLRN, a framework for end-to- end learning of knowledge base - representa- tions from latent, relational, and numerical fea- tures. KBLRN integrates +abstract: We present KBLRN, a framework for end-to-end learning of knowledge base + representations from latent, relational, and numerical features. KBLRN integrates feature types with a novel combination of neural representation learning and probabilistic product of experts models. To the best of our knowledge, KBLRN is the first approach - that learns representa- tions of knowledge bases by integrating la- tent, relational, - and numerical features. We show that instances of KBLRN outperform ex- isting methods + that learns representations of knowledge bases by integrating latent, relational, + and numerical features. We show that instances of KBLRN outperform existing methods on a range of knowledge base completion tasks. We contribute a novel data set enriching - commonly used knowledge base completion benchmarks with numerical fea- tures. The - data sets are available under a per- missive BSD-3 license1. We also investigate + commonly used knowledge base completion benchmarks with numerical features. The + data sets are available under a permissive BSD-3 license1. We also investigate the impact numerical features have on the KB completion performance of KBLRN. title: 'KBlrn: End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features' diff --git a/_posts/2018-08-06-garg18a.md b/_posts/2018-08-06-garg18a.md index ebf88d9..523ac5f 100644 --- a/_posts/2018-08-06-garg18a.md +++ b/_posts/2018-08-06-garg18a.md @@ -1,10 +1,10 @@ --- -abstract: We introduce a new class of sparse multi- prototype classifiers, designed +abstract: We introduce a new class of sparse multi-prototype classifiers, designed to combine the computational advantages of sparse predictors with the non-linear power of prototype-based classification techniques. This combination makes sparse - multi- prototype models especially well-suited for resource constrained computational - plat- forms, such as the IoT devices. We cast our supervised learning problem as - a convex- concave saddle point problem and design a provably-fast algorithm to solve + multi-prototype models especially well-suited for resource constrained computational + platforms, such as the IoT devices. We cast our supervised learning problem as + a convex-concave saddle point problem and design a provably-fast algorithm to solve it. We complement our theoretical analysis with an empirical study that demonstrates the merits of our methodology. title: Sparse Multi-Prototype Classification diff --git a/_posts/2018-08-06-gotovos18a.md b/_posts/2018-08-06-gotovos18a.md index 7c23981..96e6642 100644 --- a/_posts/2018-08-06-gotovos18a.md +++ b/_posts/2018-08-06-gotovos18a.md @@ -1,14 +1,13 @@ --- -abstract: We consider the problem of inference in dis- crete probabilistic models, +abstract: We consider the problem of inference in discrete probabilistic models, that is, distributions over subsets of a finite ground set. These encompass a range of well-known models in machine learning, such as determinantal point processes and Ising models. Locally-moving Markov chain Monte Carlo algorithms, such as the - Gibbs sampler, are commonly used for inference in such models, but their conver- - gence is, at times, prohibitively slow. This is often caused by state-space bottlenecks + Gibbs sampler, are commonly used for inference in such models, but their convergence is, at times, prohibitively slow. This is often caused by state-space bottlenecks that greatly hinder the movement of such samplers. We propose a novel sampling strategy that uses a specific mixture of product distributions to propose global moves and, - thus, acceler- ate convergence. Furthermore, we show how to construct such a mixture - using semigradi- ent information. We illustrate the effective- ness of combining + thus, accelerate convergence. Furthermore, we show how to construct such a mixture + using semigradient information. We illustrate the effectiveness of combining our sampler with existing ones, both theoretically on an example model, as well as practically on three models learned from real-world data sets. title: Discrete Sampling using Semigradient-based Product Mixtures diff --git a/_posts/2018-08-06-gregorova18a.md b/_posts/2018-08-06-gregorova18a.md index c2caf5f..e2caa9f 100644 --- a/_posts/2018-08-06-gregorova18a.md +++ b/_posts/2018-08-06-gregorova18a.md @@ -1,14 +1,14 @@ --- abstract: We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general - nonlinear func- tions not limiting a priori the function space to additive models. - We propose two new regu- larizers based on partial derivatives as nonlin- ear equivalents - of group lasso and elastic net. We formulate the problem within the frame- work + nonlinear functions not limiting a priori the function space to additive models. + We propose two new regularizers based on partial derivatives as nonlinear equivalents + of group lasso and elastic net. We formulate the problem within the framework of learning in reproducing kernel Hilbert spaces and show how the variational problem - can be reformulated into a more practical fi- nite dimensional equivalent. We develop + can be reformulated into a more practical finite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms - of the proxi- mal operators. We explore the empirical prop- erties of our new algorithm - for Nonlinear Vari- able Selection based on Derivatives (NVSD) on a set of experiments + of the proximal operators. We explore the empirical properties of our new algorithm + for Nonlinear Variable Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy. title: Structured nonlinear variable selection diff --git a/_posts/2018-08-06-guarnizo18a.md b/_posts/2018-08-06-guarnizo18a.md index e35345c..eaf3248 100644 --- a/_posts/2018-08-06-guarnizo18a.md +++ b/_posts/2018-08-06-guarnizo18a.md @@ -2,13 +2,13 @@ abstract: A latent force model is a Gaussian process with a covariance function inspired by a differential operator. Such covariance function is obtained by performing convolution integrals between Green’s functions associated to the differential operators, and - covariance functions associated to latent functions. In the classical formula- tion - of latent force models, the covariance func- tions are obtained analytically by - solving a dou- ble integral, leading to expressions that involve numerical solutions - of different types of error functions. In consequence, the covariance ma- trix calculation - is considerably expensive, be- cause it requires the evaluation of one or more of + covariance functions associated to latent functions. In the classical formulation + of latent force models, the covariance functions are obtained analytically by + solving a double integral, leading to expressions that involve numerical solutions + of different types of error functions. In consequence, the covariance matrix calculation + is considerably expensive, because it requires the evaluation of one or more of these error functions. In this paper, we use random Fourier features to approximate - the so- lution of these double integrals obtaining sim- pler analytical expressions + the solution of these double integrals obtaining simpler analytical expressions for such covariance functions. We show experimental results using ordinary differential operators and provide an extension to build general kernel functions for convolved multiple output Gaussian processes. diff --git a/_posts/2018-08-06-hegde18a.md b/_posts/2018-08-06-hegde18a.md index d3e1b9b..82d33ab 100644 --- a/_posts/2018-08-06-hegde18a.md +++ b/_posts/2018-08-06-hegde18a.md @@ -1,16 +1,16 @@ --- -abstract: Zero-inflated datasets, which have an ex- cess of zero outputs, are commonly - en- countered in problems such as climate or rare event modelling. Conventional - ma- chine learning approaches tend to overesti- mate the non-zeros leading to poor - perfor- mance. We propose a novel model family of zero-inflated Gaussian processes +abstract: Zero-inflated datasets, which have an excess of zero outputs, are commonly + encountered in problems such as climate or rare event modelling. Conventional + machine learning approaches tend to overestimate the non-zeros leading to poor + performance. We propose a novel model family of zero-inflated Gaussian processes (ZiGP) for such zero-inflated datasets, produced by sparse kernels through learning - a la- tent probit Gaussian process that can zero out kernel rows and columns whenever - the signal is absent. The ZiGPs are particu- larly useful for making the powerful - Gaus- sian process networks more interpretable. We introduce sparse GP networks + a latent probit Gaussian process that can zero out kernel rows and columns whenever + the signal is absent. The ZiGPs are particularly useful for making the powerful + Gaussian process networks more interpretable. We introduce sparse GP networks where variable-order latent modelling is achieved through sparse mixing signals. - We derive the non-trivial stochastic variational infer- ence tractably for scalable + We derive the non-trivial stochastic variational inference tractably for scalable learning of the sparse kernels in both models. The novel output-sparse approach - improves both pre- diction of zero-inflated data and inter- pretability of latent + improves both prediction of zero-inflated data and interpretability of latent mixing models. title: Variational zero-inflated Gaussian processes with sparse kernels year: '2018' diff --git a/_posts/2018-08-06-hewitt18a.md b/_posts/2018-08-06-hewitt18a.md index 4057382..b05fdcf 100644 --- a/_posts/2018-08-06-hewitt18a.md +++ b/_posts/2018-08-06-hewitt18a.md @@ -1,15 +1,15 @@ --- abstract: Hierarchical Bayesian methods can unify many related tasks (e.g. k-shot classification, conditional and unconditional generation) as inference within a - single generative model. However, when this generative model is ex- pressed as a + single generative model. However, when this generative model is expressed as a powerful neural network such as a PixelCNN, we show that existing learning techniques - typically fail to effectively use la- tent variables. To address this, we develop + typically fail to effectively use latent variables. To address this, we develop a modification of the Variational Autoencoder in which encoded observations are decoded to new elements from the same class. This technique, which we call a Variational - Ho- moencoder (VHE), produces a hierarchical la- tent variable model which better - utilises la- tent variables. We use the VHE framework to learn a hierarchical PixelCNN + Homoencoder (VHE), produces a hierarchical latent variable model which better + utilises latent variables. We use the VHE framework to learn a hierarchical PixelCNN on the Omniglot dataset, which outperforms all existing models on test set likelihood - and achieves strong per- formance on one-shot generation and classifi- cation tasks. + and achieves strong performance on one-shot generation and classification tasks. We additionally validate the VHE on natural images from the YouTube Faces database. Finally, we develop extensions of the model that apply to richer dataset structures such as factorial and hierarchical categories. diff --git a/_posts/2018-08-06-huang18a.md b/_posts/2018-08-06-huang18a.md index e5b3651..88825ad 100644 --- a/_posts/2018-08-06-huang18a.md +++ b/_posts/2018-08-06-huang18a.md @@ -1,15 +1,15 @@ --- abstract: In this paper, we study the unsupervised multi-view graph embedding (UMGE) - prob- lem, which aims to learn graph embedding from multiple perspectives in an - unsupervised manner. However, the vast majority of multi- view learning work focuses + problem, which aims to learn graph embedding from multiple perspectives in an + unsupervised manner. However, the vast majority of multi-view learning work focuses on non-graph data, and surprisingly there are limited work on UMGE. By systematically analyzing different existing methods for UMGE, we discover that cross-view and nonlinearity - play a vital role in efficiently improving graph embedding qual- ity. Motivated + play a vital role in efficiently improving graph embedding quality. Motivated by this concept, we develop an unsupervised Multi-viEw nonlineaR Graph Embedding - (MERGE) approach to model re- lational multi-view consistency. Experimen- tal results - on five benchmark datasets demon- strate that MERGE significantly outperforms the - state-of-the-art baselines in terms of accu- racy in node classification tasks without - sacri- ficing the computational efficiency. + (MERGE) approach to model relational multi-view consistency. Experimental results + on five benchmark datasets demonstrate that MERGE significantly outperforms the + state-of-the-art baselines in terms of accuracy in node classification tasks without + sacrificing the computational efficiency. title: Unsupervised Multi-view Nonlinear Graph Embedding year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-izmailov18a.md b/_posts/2018-08-06-izmailov18a.md index 5143d7c..9cb851c 100644 --- a/_posts/2018-08-06-izmailov18a.md +++ b/_posts/2018-08-06-izmailov18a.md @@ -1,13 +1,13 @@ --- abstract: Deep neural networks are typically trained by optimizing a loss function - with an SGD vari- ant, in conjunction with a decaying learning rate, until convergence. - We show that simple averaging of multiple points along the trajec- tory of SGD, - with a cyclical or constant learn- ing rate, leads to better generalization than + with an SGD variant, in conjunction with a decaying learning rate, until convergence. + We show that simple averaging of multiple points along the trajectory of SGD, + with a cyclical or constant learning rate, leads to better generalization than conventional training. We also show that this Stochastic Weight Averaging (SWA) - procedure finds much broader optima than SGD, and ap- proximates the recent Fast - Geometric Ensem- bling (FGE) approach with a single model. Using SWA we achieve - notable improvement in test accuracy over conventional SGD train- ing on a range - of state-of-the-art residual net- works, PyramidNets, DenseNets, and Shake- Shake + procedure finds much broader optima than SGD, and approximates the recent Fast + Geometric Ensembling (FGE) approach with a single model. Using SWA we achieve + notable improvement in test accuracy over conventional SGD training on a range + of state-of-the-art residual networks, PyramidNets, DenseNets, and Shake- Shake networks on CIFAR-10, CIFAR-100, and ImageNet. In short, SWA is extremely easy to implement, improves generalization, and has almost no computational overhead. title: Averaging Weights Leads to Wider Optima and Better Generalization diff --git a/_posts/2018-08-06-jaber18a.md b/_posts/2018-08-06-jaber18a.md index af7a05f..a1e8c33 100644 --- a/_posts/2018-08-06-jaber18a.md +++ b/_posts/2018-08-06-jaber18a.md @@ -1,18 +1,17 @@ --- abstract: Assessing the magnitude of cause-and-effect relations is one of the central - challenges found throughout the empirical sciences. The prob- lem of identification - of causal effects is con- cerned with determining whether a causal ef- fect can + challenges found throughout the empirical sciences. The problem of identification + of causal effects is concerned with determining whether a causal effect can be computed from a combination of observational data and substantive knowledge about the domain under investigation, which is formally expressed in the form of a causal graph. In many practical settings, however, the knowledge available for the researcher is not strong enough so as to specify a unique causal graph. Another line of investigation - attempts to use observational data to learn a qualita- tive description of the domain + attempts to use observational data to learn a qualitative description of the domain called a Markov equivalence class, which is the collection of causal graphs that - share the same set of ob- served features. In this paper, we marry both approaches - and study the problem of causal identification from an equivalence class, repre- - sented by a partial ancestral graph (PAG). We start by deriving a set of graphical - properties of PAGs that are carried over to its induced sub- graphs. We then develop - an algorithm to com- pute the effect of an arbitrary set of variables on an arbitrary + share the same set of observed features. In this paper, we marry both approaches + and study the problem of causal identification from an equivalence class, represented by a partial ancestral graph (PAG). We start by deriving a set of graphical + properties of PAGs that are carried over to its induced sub-graphs. We then develop + an algorithm to compute the effect of an arbitrary set of variables on an arbitrary outcome set. We show that the algorithm is strictly more powerful than the current state of the art found in the literature. title: Causal Identification under Markov Equivalence diff --git a/_posts/2018-08-06-junges18a.md b/_posts/2018-08-06-junges18a.md index c2a7184..f16ffcf 100644 --- a/_posts/2018-08-06-junges18a.md +++ b/_posts/2018-08-06-junges18a.md @@ -1,11 +1,11 @@ --- -abstract: We study finite-state controllers (FSCs) for par- tially observable Markov - decision processes (POMDPs) that are provably correct with re- spect to given specifications. - The key in- sight is that computing (randomised) FSCs on POMDPs is equivalent to—and - compu- tationally as hard as—synthesis for paramet- ric Markov chains (pMCs). This - correspon- dence allows to use tools for synthesis in pMCs to compute correct-by-construction +abstract: We study finite-state controllers (FSCs) for partially observable Markov + decision processes (POMDPs) that are provably correct with respect to given specifications. + The key insight is that computing (randomised) FSCs on POMDPs is equivalent to—and + computationally as hard as—synthesis for parametric Markov chains (pMCs). This + correspondence allows to use tools for synthesis in pMCs to compute correct-by-construction FSCs on POMDPs for a variety of specifications. Our experimental evaluation shows - comparable per- formance to well-known POMDP solvers. + comparable performance to well-known POMDP solvers. title: Finite-State Controllers of POMDPs using Parameter Synthesis year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-kang18a.md b/_posts/2018-08-06-kang18a.md index f8d5604..41d4e7e 100644 --- a/_posts/2018-08-06-kang18a.md +++ b/_posts/2018-08-06-kang18a.md @@ -1,20 +1,18 @@ --- -abstract: Meta learning algorithms are effective at ob- taining meta models with the - capability of solving new tasks quickly. However, they crit- ically require sufficient +abstract: Meta learning algorithms are effective at obtaining meta models with the + capability of solving new tasks quickly. However, they critically require sufficient tasks for meta model training and the resulted model can only solve new tasks similar - to the training ones. These limitations make them suffer performance de- cline in - presence of insufficiency of training tasks in target domains and task heterogene- - ity—the source (model training) tasks presents different characteristics from target - (model ap- plication) tasks. To overcome these two signif- icant limitations of - existing meta learning al- gorithms, we introduce the cross-domain meta learning - framework and propose a new trans- ferable meta learning (TML) algorithm. TML performs - meta task adaptation jointly with meta model learning, which effectively nar- rows - divergence between source and target tasks and enables transferring source meta- - knowledge to solve target tasks. Thus, the re- sulted transferable meta model can - solve new learning tasks in new domains quickly. We ap- ply the proposed TML to - cross-domain few- shot classification problems and evaluate its performance on multiple - benchmarks. It per- forms significantly better and faster than well- established - meta learning algorithms and fine- tuned domain-adapted models. + to the training ones. These limitations make them suffer performance decline in + presence of insufficiency of training tasks in target domains and task heterogeneity—the source (model training) tasks presents different characteristics from target + (model application) tasks. To overcome these two significant limitations of + existing meta learning algorithms, we introduce the cross-domain meta learning + framework and propose a new transferable meta learning (TML) algorithm. TML performs + meta task adaptation jointly with meta model learning, which effectively narrows + divergence between source and target tasks and enables transferring source meta-knowledge to solve target tasks. Thus, the resulted transferable meta model can + solve new learning tasks in new domains quickly. We apply the proposed TML to + cross-domain few-shot classification problems and evaluate its performance on multiple + benchmarks. It performs significantly better and faster than well-established + meta learning algorithms and fine-tuned domain-adapted models. title: Transferable Meta Learning Across Domains year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-karan18a.md b/_posts/2018-08-06-karan18a.md index c609680..4b564b0 100644 --- a/_posts/2018-08-06-karan18a.md +++ b/_posts/2018-08-06-karan18a.md @@ -1,11 +1,10 @@ --- -abstract: We propose scalable methods to execute count- ing queries in machine learning - applications. To achieve memory and computational effi- ciency, we abstract counting +abstract: We propose scalable methods to execute counting queries in machine learning + applications. To achieve memory and computational efficiency, we abstract counting queries and their context such that the counts can be aggregated as a stream. We demonstrate performance and scalability of the resulting approach on random queries, - and through extensive experimentation using Bayesian networks learning and associ- - ation rule mining. Our methods significantly outperform commonly used ADtrees and - hash tables, and are practical alternatives for process- ing large-scale data. + and through extensive experimentation using Bayesian networks learning and association rule mining. Our methods significantly outperform commonly used ADtrees and + hash tables, and are practical alternatives for processing large-scale data. title: Fast Counting in Machine Learning Applications year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-kim18a.md b/_posts/2018-08-06-kim18a.md index 00abae2..0a346e2 100644 --- a/_posts/2018-08-06-kim18a.md +++ b/_posts/2018-08-06-kim18a.md @@ -1,19 +1,18 @@ --- abstract: 'Gaussian process survival analysis model (GP- SAM) was recently proposed to address key deficiencies of the Cox proportional hazard model, namely the need - to account for uncer- tainty in the hazard function modeling while, at the same - time, relaxing the time-covariates factorized assumption of the Cox model. How- - ever, the existing MCMC inference algorithms for GPSAM have proven to be slow in - prac- tice. In this paper we propose novel and scal- able variational inference + to account for uncertainty in the hazard function modeling while, at the same + time, relaxing the time-covariates factorized assumption of the Cox model. However, the existing MCMC inference algorithms for GPSAM have proven to be slow in + practice. In this paper we propose novel and scalable variational inference algorithms for GP- SAM that reduce the time complexity of the sampling approaches - and improve scalability to large datasets. We accomplish this by em- ploying two + and improve scalability to large datasets. We accomplish this by employing two effective strategies in scalable GP: i) using pseudo inputs and ii) approximation via random feature expansions. In both setups, we derive the full and partial likelihood - formu- lations, typically considered in survival analy- sis settings. The proposed - approaches are eval- uated on two clinical and a divorce-marriage benchmark datasets, + formulations, typically considered in survival analysis settings. The proposed + approaches are evaluated on two clinical and a divorce-marriage benchmark datasets, where we demonstrate improvements in prediction accuracy over the existing survival - analysis methods, while re- ducing the complexity of inference compared to the recent - state-of-the-art MCMC-based al- gorithms.' + analysis methods, while reducing the complexity of inference compared to the recent + state-of-the-art MCMC-based algorithms.' title: Variational Inference for Gaussian Process Models for Survival Analysis year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-krishnan18a.md b/_posts/2018-08-06-krishnan18a.md index 85cfedb..a097b2b 100644 --- a/_posts/2018-08-06-krishnan18a.md +++ b/_posts/2018-08-06-krishnan18a.md @@ -1,17 +1,17 @@ --- -abstract: We propose a new way to answer probabilis- tic queries that span multiple - datapoints. We formalize reasoning about the similarity of dif- ferent datapoints +abstract: We propose a new way to answer probabilistic queries that span multiple + datapoints. We formalize reasoning about the similarity of different datapoints as the evaluation of the Bayes Factor within a hierarchical deep generative model that enforces a separation between the latent variables used for representation learning and those used for reasoning. Under this model, we derive an intuitive estimator for the Bayes Factor that represents similarity as the amount of overlap - in representation space shared by dif- ferent points. The estimator we derive relies + in representation space shared by different points. The estimator we derive relies on a query-conditional latent reasoning network, that parameterizes a distribution over the latent space of the deep generative model. The latent reasoning network - is trained to amortize the posterior-predictive distribution under a hierar- chical - model using supervised data and a max- margin learning algorithm. We explore how + is trained to amortize the posterior-predictive distribution under a hierarchical + model using supervised data and a maxmargin learning algorithm. We explore how the model may be used to focus the data variations captured in the latent space - of the deep genera- tive model and how this may be used to build new algorithms + of the deep generative model and how this may be used to build new algorithms for few-shot learning. title: Max-margin learning with the Bayes factor year: '2018' diff --git a/_posts/2018-08-06-kyrillidis18a.md b/_posts/2018-08-06-kyrillidis18a.md index 96a7d27..946a3fd 100644 --- a/_posts/2018-08-06-kyrillidis18a.md +++ b/_posts/2018-08-06-kyrillidis18a.md @@ -1,11 +1,11 @@ --- abstract: 'We propose practical algorithms for entrywise ‘p-norm low-rank approximation, for p = 1 or p = 1. The proposed framework, which is non-convex and gradient-based, - is easy to implement and typically attains better approx- imations, faster, than + is easy to implement and typically attains better approximations, faster, than state of the art. From a theoretical standpoint, we show that the proposed scheme can attain (1 + ")- OPT approximations. Our algorithms are not hyperparameter-free: - they achieve the desider- ata only assuming algorithm’s hyperparame- ters are known - apriori—or are at least approx- imable. I.e., our theory indicates what problem + they achieve the desiderata only assuming algorithm’s hyperparameters are known + apriori—or are at least approximable. I.e., our theory indicates what problem quantities need to be known, in order to get a good solution within polynomial time, and does not contradict to recent inapproximabilty results, as in [46].' title: Simple and practical algorithms for $\ell_p$-norm low-rank approximation diff --git a/_posts/2018-08-06-lacey18a.md b/_posts/2018-08-06-lacey18a.md index 6b3b42f..802c223 100644 --- a/_posts/2018-08-06-lacey18a.md +++ b/_posts/2018-08-06-lacey18a.md @@ -1,15 +1,13 @@ --- -abstract: Low precision weights, activations, and gradi- ents have been proposed as - a way to improve the computational efficiency and memory foot- print of deep neural +abstract: Low precision weights, activations, and gradients have been proposed as + a way to improve the computational efficiency and memory footprint of deep neural networks. Recently, low precision networks have even shown to be more robust to - adversarial attacks. How- ever, typical implementations of low precision DNNs use - uniform precision across all lay- ers of the network. In this work, we explore whether - a heterogeneous allocation of preci- sion across a network leads to improved per- - formance, and introduce a learning scheme where a DNN stochastically explores multi- - ple precision configurations through learning. This permits a network to learn an - optimal pre- cision configuration. We show on convolu- tional neural networks trained + adversarial attacks. However, typical implementations of low precision DNNs use + uniform precision across all layers of the network. In this work, we explore whether + a heterogeneous allocation of precision across a network leads to improved performance, and introduce a learning scheme where a DNN stochastically explores multiple precision configurations through learning. This permits a network to learn an + optimal precision configuration. We show on convolutional neural networks trained on MNIST and ILSVRC12 that even though these nets learn a uniform or near-uniform - allocation strat- egy respectively, stochastic precision leads to a favourable regularization + allocation strategy respectively, stochastic precision leads to a favourable regularization effect improving generalization. title: Stochastic Layer-Wise Precision in Deep Neural Networks year: '2018' diff --git a/_posts/2018-08-06-lawton18a.md b/_posts/2018-08-06-lawton18a.md index b4f3845..310f8f9 100644 --- a/_posts/2018-08-06-lawton18a.md +++ b/_posts/2018-08-06-lawton18a.md @@ -1,13 +1,13 @@ --- abstract: Coordinate ascent variational inference is an important algorithm for inference - in proba- bilistic models, but it is slow because it updates only a single variable - at a time. Block coordi- nate methods perform inference faster by up- dating blocks - of variables in parallel. How- ever, the speed and convergence of these algo- rithms - depends on how the variables are par- titioned into blocks. In this paper, we give + in probabilistic models, but it is slow because it updates only a single variable + at a time. Block coordinate methods perform inference faster by updating blocks + of variables in parallel. However, the speed and convergence of these algorithms + depends on how the variables are partitioned into blocks. In this paper, we give a convergent parallel algorithm for inference in deep exponential families that doesn’t require the variables to be partitioned into blocks. We achieve this by lower bounding the ELBO by a new objective we call the forest mixture bound (FM - bound) that separates the inference prob- lem for variables within a hidden layer. + bound) that separates the inference problem for variables within a hidden layer. We apply this to the simple case when all random variables are Gaussian and show empirically that the algorithm converges faster for models that are inherently more forest-like. diff --git a/_posts/2018-08-06-le-priol18a.md b/_posts/2018-08-06-le-priol18a.md index 423ecfb..d219363 100644 --- a/_posts/2018-08-06-le-priol18a.md +++ b/_posts/2018-08-06-le-priol18a.md @@ -1,12 +1,12 @@ --- -abstract: This work investigates the training of condi- tional random fields (CRFs) - via the stochas- tic dual coordinate ascent (SDCA) algorithm of Shalev-Shwartz and +abstract: This work investigates the training of conditional random fields (CRFs) + via the stochastic dual coordinate ascent (SDCA) algorithm of Shalev-Shwartz and Zhang (2016). SDCA enjoys a linear convergence rate and a strong empirical performance for binary classification problems. However, it has never been used to train CRFs. Yet it benefits from an “exact” line search with a single marginalization oracle call, unlike previous approaches. In this paper, we adapt SDCA to train CRFs, and we enhance it with an adaptive non-uniform sampling strategy based on block duality - gaps. We perform ex- periments on four standard sequence prediction tasks. SDCA + gaps. We perform experiments on four standard sequence prediction tasks. SDCA demonstrates performances on par with the state of the art, and improves over it on three of the four datasets, which have in common the use of sparse features. title: Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields diff --git a/_posts/2018-08-06-lee18a.md b/_posts/2018-08-06-lee18a.md index 2e3b20b..d96d2e0 100644 --- a/_posts/2018-08-06-lee18a.md +++ b/_posts/2018-08-06-lee18a.md @@ -1,14 +1,14 @@ --- abstract: We introduce a new decomposition method for bounding the maximum expected - utility of in- fluence diagrams. While most current schemes use reductions to the - Marginal Map task over a Bayesian Network, our approach is direct, aim- ing to avoid + utility of influence diagrams. While most current schemes use reductions to the + Marginal Map task over a Bayesian Network, our approach is direct, aiming to avoid the large explosion in the model size that often results by such reductions. In this paper, we extend to influence diagrams the principles of decomposition methods - that were applied earlier to probabilistic inference, uti- lizing an algebraic framework - called valuation algebra which effectively captures both multi- plicative and additive + that were applied earlier to probabilistic inference, utilizing an algebraic framework + called valuation algebra which effectively captures both multiplicative and additive local structures present in influence diagrams. Empirical evaluation on four benchmarks demonstrates the effectiveness of our approach compared to reduction-based approaches - and illustrates significant improve- ments in the upper bounds on maximum ex- pected + and illustrates significant improvements in the upper bounds on maximum expected utility. title: Join Graph Decomposition Bounds for Influence Diagrams year: '2018' diff --git a/_posts/2018-08-06-liu18a.md b/_posts/2018-08-06-liu18a.md index c363740..1f14651 100644 --- a/_posts/2018-08-06-liu18a.md +++ b/_posts/2018-08-06-liu18a.md @@ -5,12 +5,12 @@ abstract: We study multi-armed bandit problems with graph feedback, in which the We show that when the feedback graphs are undirected, the original Thompson Sampling achieves the optimal (within logarithmic factors) regret $\tilde{}$O (p $\beta$0(G)T ) over time horizon T, where $\beta$0(G) is the average independence number of the - latent graphs. To the best of our knowl- edge, this is the first result showing + latent graphs. To the best of our knowledge, this is the first result showing that the original Thompson Sampling is optimal for graphical bandits in the undirected setting. A slightly weaker regret bound of Thompson Sampling in the directed setting - is also pre- sented. To fill this gap, we propose a variant of Thompson Sampling, - that attains the opti- mal regret in the directed setting within a log- arithmic - factor. Both algorithms can be im- plemented efficiently and do not require the + is also presented. To fill this gap, we propose a variant of Thompson Sampling, + that attains the optimal regret in the directed setting within a logarithmic + factor. Both algorithms can be implemented efficiently and do not require the knowledge of the feedback graphs at any time. title: Analysis of Thompson Sampling for Graphical Bandits Without the Graphs year: '2018' diff --git a/_posts/2018-08-06-lou18a.md b/_posts/2018-08-06-lou18a.md index 3c9ae95..f7a24a6 100644 --- a/_posts/2018-08-06-lou18a.md +++ b/_posts/2018-08-06-lou18a.md @@ -1,15 +1,15 @@ --- -abstract: Marginal MAP is a key task in Bayesian in- ference and decision-making, +abstract: Marginal MAP is a key task in Bayesian inference and decision-making, and known to be very challenging in general. In this paper, we present an algorithm - that blends heuristic search and importance sampling to provide any- time finite-sample - bounds for marginal MAP along with predicted MAP solutions. We con- vert bounding - marginal MAP to a surrogate task of bounding a series of summation prob- lems of + that blends heuristic search and importance sampling to provide anytime finite-sample + bounds for marginal MAP along with predicted MAP solutions. We convert bounding + marginal MAP to a surrogate task of bounding a series of summation problems of an augmented graphical model, and then adapt dynamic importance sampling [Lou et al., 2017b], a recent advance in bounding the partition function, to provide finite-sample bounds for the surrogate task. Those bounds are guaranteed to be tight given enough time, and the values of the predicted MAP solutions will converge to the optimum. Our algorithm runs in an anytime/anyspace manner, which gives flexible trade-offs - between memory, time, and solution quality. We demonstrate the effective- ness of + between memory, time, and solution quality. We demonstrate the effectiveness of our approach empirically on multiple challenging benchmarks in comparison with some state-of-the-art search algorithms. title: Finite-sample Bounds for Marginal MAP diff --git a/_posts/2018-08-06-lyu18a.md b/_posts/2018-08-06-lyu18a.md index 2383ed6..afff942 100644 --- a/_posts/2018-08-06-lyu18a.md +++ b/_posts/2018-08-06-lyu18a.md @@ -1,15 +1,15 @@ --- -abstract: Area under ROC (AUC) is an important met- ric for binary classification - and bipartite rank- ing problems. However, it is difficult to di- rectly optimize - AUC as a learning objective, so most existing algorithms are based on optimiz- ing +abstract: Area under ROC (AUC) is an important metric for binary classification + and bipartite ranking problems. However, it is difficult to directly optimize + AUC as a learning objective, so most existing algorithms are based on optimizing a surrogate loss to AUC. One significant drawback of these surrogate losses is that they require pairwise comparisons among training data, which leads to slow running - time and in- creasing local storage for online learning. In this work, we describe + time and increasing local storage for online learning. In this work, we describe a new surrogate loss based on a reformulation of AUC risk, which does not require - pairwise comparison but rank- ings of the predictions. We further show that the + pairwise comparison but rankings of the predictions. We further show that the ranking operation can be avoided, and the learning objective obtained based on this - sur- rogate enjoys linear complexity in time and storage. We perform experiments - to demon- strate the effectiveness of the online and batch algorithms for AUC optimization + surrogate enjoys linear complexity in time and storage. We perform experiments + to demonstrate the effectiveness of the online and batch algorithms for AUC optimization based on the proposed surrogate loss. title: A Univariate Bound of Area Under ROC year: '2018' diff --git a/_posts/2018-08-06-ma18a.md b/_posts/2018-08-06-ma18a.md index 01cb229..343fab5 100644 --- a/_posts/2018-08-06-ma18a.md +++ b/_posts/2018-08-06-ma18a.md @@ -1,16 +1,16 @@ --- abstract: In this paper we introduce a novel holographic memory model for the distributed storage of complex association patterns and apply it to knowledge graphs. In a knowledge - graph, a la- belled link connects a subject node with an ob- ject node, jointly - forming a subject-predicate- objects triple. In the presented work, nodes and links + graph, a labelled link connects a subject node with an object node, jointly + forming a subject-predicateobjects triple. In the presented work, nodes and links have initial random representations, plus holistic representations derived from the initial representations of nodes and links in their local neighbourhoods. A memory trace is represented in the same vector space as the holistic representations themselves. To reduce the interference between stored information, it is required that the initial random vectors should be pairwise quasi-orthogonal. We show that - pairwise quasi-orthogonality can be im- proved by drawing vectors from heavy-tailed + pairwise quasi-orthogonality can be improved by drawing vectors from heavy-tailed distributions, e.g., a Cauchy distribution, and, thus, memory capacity of holistic - representa- tions can significantly be improved. Further- more, we show that, in + representations can significantly be improved. Furthermore, we show that, in combination with a simple neural network, the presented holistic representation approach is superior to other methods for link predictions on knowledge graphs. title: Holistic Representations for Memorization and Inference diff --git a/_posts/2018-08-06-madan18a.md b/_posts/2018-08-06-madan18a.md index 9b18ffa..be7df6b 100644 --- a/_posts/2018-08-06-madan18a.md +++ b/_posts/2018-08-06-madan18a.md @@ -1,14 +1,14 @@ --- -abstract: One popular way for lifted inference in proba- bilistic graphical models - is to first merge sym- metric states into a single cluster (orbit) and then use +abstract: One popular way for lifted inference in probabilistic graphical models + is to first merge symmetric states into a single cluster (orbit) and then use these for downstream inference, via variations of orbital MCMC [Niepert, 2012]. - These orbits are represented compactly us- ing permutations over variables, and - variable- value (VV) pairs, but they can miss several state symmetries in a domain. + These orbits are represented compactly using permutations over variables, and + variablevalue (VV) pairs, but they can miss several state symmetries in a domain. We define the notion of permutations over block-value (BV) pairs, where a block - is a set of variables. BV strictly generalizes VV sym- metries, and can compute - many more sym- metries for increasing block sizes. To opera- tionalize use of BV - permutations in lifted in- ference, we describe 1) an algorithm to com- pute BV - permutations given a block parti- tion of the variables, 2) BV-MCMC, an exten- sion + is a set of variables. BV strictly generalizes VV symmetries, and can compute + many more symmetries for increasing block sizes. To operationalize use of BV + permutations in lifted inference, we describe 1) an algorithm to compute BV + permutations given a block partition of the variables, 2) BV-MCMC, an extension of orbital MCMC that can sample from BV orbits, and 3) a heuristic to suggest good block partitions. Our experiments show that BV-MCMC can mix much faster compared to vanilla MCMC and orbital MCMC. diff --git a/_posts/2018-08-06-marchetti18a.md b/_posts/2018-08-06-marchetti18a.md index 897a739..20d2bfc 100644 --- a/_posts/2018-08-06-marchetti18a.md +++ b/_posts/2018-08-06-marchetti18a.md @@ -1,9 +1,8 @@ --- abstract: We introduce a novel class of adjustment rules for a collection of beliefs. - This is an exten- sion of Lewis’ imaging to absorb probabilistic evidence in generalized - settings. Unlike stan- dard tools for belief revision, our proposal may be used - when information is inconsistent with an agent’s belief base. We show that the func- - tionals we introduce are based on the imagi- nary counterpart of probability kinematics + This is an extension of Lewis’ imaging to absorb probabilistic evidence in generalized + settings. Unlike standard tools for belief revision, our proposal may be used + when information is inconsistent with an agent’s belief base. We show that the functionals we introduce are based on the imaginary counterpart of probability kinematics for standard belief revision, and prove that, under certain conditions, all standard postulates for belief revision are satisfied. title: Imaginary Kinematics diff --git a/_posts/2018-08-06-meisami18a.md b/_posts/2018-08-06-meisami18a.md index 445f2c1..7ec927a 100644 --- a/_posts/2018-08-06-meisami18a.md +++ b/_posts/2018-08-06-meisami18a.md @@ -1,13 +1,12 @@ --- -abstract: We provide the first study on online learn- ing problems under stochastic +abstract: We provide the first study on online learning problems under stochastic constraints that are “soft”, i.e., need to be satisfied with high probability. These constraints are imposed on all or some stages of the time horizon so that the stage - decisions probabilistically satisfy some given safety conditions. The distribu- - tions that govern these conditions are learned through the collected observations. + decisions probabilistically satisfy some given safety conditions. The distributions that govern these conditions are learned through the collected observations. Under a Bayesian framework, we introduce a scheme that provides statistical feasibility guarantees through the time horizon, by using posterior Monte Carlo samples to form - sampled con- straints which leverage the scenario generation approach in chance-constrained - programming. We demonstrate how our scheme can be inte- grated into Thompson sampling + sampled constraints which leverage the scenario generation approach in chance-constrained + programming. We demonstrate how our scheme can be integrated into Thompson sampling and illustrate it with an application in online advertisement. title: Sequential Learning under Probabilistic Constraints year: '2018' diff --git a/_posts/2018-08-06-mogensen18a.md b/_posts/2018-08-06-mogensen18a.md index c3659be..a8d27e0 100644 --- a/_posts/2018-08-06-mogensen18a.md +++ b/_posts/2018-08-06-mogensen18a.md @@ -1,16 +1,16 @@ --- abstract: Many models of dynamical systems have causal interpretations that support - reasoning about the consequences of interventions, suita- bly defined. Furthermore, + reasoning about the consequences of interventions, suitably defined. Furthermore, local independence has been suggested as a useful independence concept for stochastic - dynamical systems. There is, however, no well-developed theore- tical framework - for causal learning based on this notion of independence. We study inde- pendence - models induced by directed graphs (DGs) and provide abstract graphoid proper- ties + dynamical systems. There is, however, no well-developed theoretical framework + for causal learning based on this notion of independence. We study independence + models induced by directed graphs (DGs) and provide abstract graphoid properties that guarantee that an independence model has the global Markov property w.r.t. a DG. We apply these results to It\^{}o diffusions and event processes. For a partially - observed sys- tem, directed mixed graphs (DMGs) represent the marginalized local - independence model, and we develop, under a faithfulness assump- tion, a sound and - complete learning algo- rithm of the directed mixed equivalence graph (DMEG) as - a summary of all Markov equiva- lent DMGs. + observed system, directed mixed graphs (DMGs) represent the marginalized local + independence model, and we develop, under a faithfulness assumption, a sound and + complete learning algorithm of the directed mixed equivalence graph (DMEG) as + a summary of all Markov equivalent DMGs. title: Causal Learning for Partially Observed Stochastic Dynamical Systems year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-monti18a.md b/_posts/2018-08-06-monti18a.md index 017180a..0cbc358 100644 --- a/_posts/2018-08-06-monti18a.md +++ b/_posts/2018-08-06-monti18a.md @@ -1,19 +1,19 @@ --- abstract: Connectivity estimation is challenging in the context of high-dimensional data. A useful preprocessing step is to group variables into clusters, however, - it is not always clear how to do so from the perspective of connectiv- ity estimation. + it is not always clear how to do so from the perspective of connectivity estimation. Another practical challenge is that we may have data from multiple related classes (e.g., multiple subjects or conditions) and wish to incorporate constraints on the - simi- larities across classes. We propose a probabilis- tic model which simultaneously - performs both a grouping of variables (i.e., detecting commu- nity structure) and + similarities across classes. We propose a probabilistic model which simultaneously + performs both a grouping of variables (i.e., detecting community structure) and estimation of connectivities between the groups which correspond to latent variables. - The model is essentially a factor anal- ysis model where the factors are allowed + The model is essentially a factor analysis model where the factors are allowed to have arbitrary correlations, while the factor loading matrix is constrained to express a community structure. The model can be applied on multiple classes so that - the connectivities can be differ- ent between the classes, while the community structure - is the same for all classes. We pro- pose an efficient estimation algorithm based + the connectivities can be different between the classes, while the community structure + is the same for all classes. We propose an efficient estimation algorithm based on score matching, and prove the identifiability of the model. Finally, we present - an extension to directed (causal) connectivities over latent vari- ables. Simulations + an extension to directed (causal) connectivities over latent variables. Simulations and experiments on fMRI data validate the practical utility of the method. title: A unified probabilistic model for learning latent factors and their connectivities from high-dimensional data diff --git a/_posts/2018-08-06-moore18a.md b/_posts/2018-08-06-moore18a.md index 17b2f22..87752ca 100644 --- a/_posts/2018-08-06-moore18a.md +++ b/_posts/2018-08-06-moore18a.md @@ -1,16 +1,15 @@ --- abstract: Domains such as citizen science biodiversity monitoring and real estate - sales are produc- ing spatial data with a continuous response and a vector of covariates - associated with each spatial data point. A common data analy- sis task involves + sales are producing spatial data with a continuous response and a vector of covariates + associated with each spatial data point. A common data analysis task involves finding unusual regions that differ from the surrounding area. Existing techniques - compare regions according to the means of their distributions to measure unusu- - alness. Comparing means is not only vulner- able to outliers, but it is also restrictive + compare regions according to the means of their distributions to measure unusualness. Comparing means is not only vulnerable to outliers, but it is also restrictive as an analyst may want to compare other parts of the probability distributions. - For instance, an analyst interested in unusual areas for high- end homes would be + For instance, an analyst interested in unusual areas for high-end homes would be more interested in the 90th percentile of home sale prices than in the mean. We introduce the Quantile Spatial Scan Statistic (QSSS), which finds unusual regions in spatial data by comparing quantiles of data distributions while accounting for - covariates at each data point. We also develop an exact in- cremental update of + covariates at each data point. We also develop an exact incremental update of the hypothesis test used by the QSSS, which results in a massive speedup over a naive implementation. title: An Efficient Quantile Spatial Scan Statistic for Finding Unusual Regions in diff --git a/_posts/2018-08-06-nabi18a.md b/_posts/2018-08-06-nabi18a.md index cde9b88..18ac8ab 100644 --- a/_posts/2018-08-06-nabi18a.md +++ b/_posts/2018-08-06-nabi18a.md @@ -1,18 +1,18 @@ --- abstract: The goal of personalized decision making is to map a unit’s characteristics - to an action tai- lored to maximize the expected outcome for that unit. Obtaining - high-quality mappings of this type is the goal of the dynamic regime lit- erature. - In healthcare settings, optimizing poli- cies with respect to a particular causal + to an action tailored to maximize the expected outcome for that unit. Obtaining + high-quality mappings of this type is the goal of the dynamic regime literature. + In healthcare settings, optimizing policies with respect to a particular causal pathway may be of interest as well. For example, we may wish to maximize the chemical effect of a drug given data from an observational study where the chemical effect of the drug on the outcome is entangled with the indirect effect mediated by differential - adherence. In such cases, we may wish to optimize the direct ef- fect of a drug, + adherence. In such cases, we may wish to optimize the direct effect of a drug, while keeping the indirect effect to that of some reference treatment. [15] shows - how to combine mediation analysis and dy- namic treatment regime ideas to defines - poli- cies associated with causal pathways and coun- terfactual responses to these - policies. In this paper, we derive a variety of methods for learn- ing high quality - policies of this type from data, in a causal model corresponding to a longitu- dinal - setting of practical importance. We illus- trate our methods via a dataset of HIV + how to combine mediation analysis and dynamic treatment regime ideas to defines + policies associated with causal pathways and counterfactual responses to these + policies. In this paper, we derive a variety of methods for learning high quality + policies of this type from data, in a causal model corresponding to a longitudinal + setting of practical importance. We illustrate our methods via a dataset of HIV patients undergoing therapy, gathered in the Nigerian PEPFAR program. title: Estimation of Personalized Effects Associated With Causal Pathways year: '2018' diff --git a/_posts/2018-08-06-nair18a.md b/_posts/2018-08-06-nair18a.md index c49aa16..f9c8ed5 100644 --- a/_posts/2018-08-06-nair18a.md +++ b/_posts/2018-08-06-nair18a.md @@ -1,19 +1,19 @@ --- abstract: We present a novel reinforcement learning (RL) approach to learning a fast - and highly scalable solver for a two-stage stochastic integer pro- gram in the large-scale - data setting. Mixed inte- ger programming solvers do not scale to large datasets + and highly scalable solver for a two-stage stochastic integer program in the large-scale + data setting. Mixed integer programming solvers do not scale to large datasets for this problem class. Additionally, they solve each instance independently, without any knowledge transfer across instances. We address these limitations with a learnable local search solver that jointly learns two policies, one to generate an initial solution and another to iteratively improve it with local moves. The policies use contextual features for a problem instance as input, which enables learning across instances and generalization to new ones. We also propose learning a policy to compute - a bound on the objective using dual decompo- sition. Benchmark results show that - on test instances our approach rapidly achieves approx- imately 30% to 2000% better + a bound on the objective using dual decomposition. Benchmark results show that + on test instances our approach rapidly achieves approximately 30% to 2000% better objective value, which a state of the art integer programming solver (SCIP) requires more than an order of magnitude more running time to match. Our approach also achieves better solution quality on seven out of eight benchmark problems than standard baselines - such as Tabu Search and Pro- gressive Hedging. + such as Tabu Search and Progressive Hedging. title: Learning Fast Optimizers for Contextual Stochastic Integer Programs year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-palmer18a.md b/_posts/2018-08-06-palmer18a.md index b1203f4..1fcdc0c 100644 --- a/_posts/2018-08-06-palmer18a.md +++ b/_posts/2018-08-06-palmer18a.md @@ -3,16 +3,15 @@ abstract: 'Generative models, while not new, have taken the deep learning field storm. However, the widely used training methods have not exploited the substantial statistical literature concerning parametric distributional testing. Having sound theoretical foundations, these goodness-of-fit tests enable parts of the black box - to be stripped away. In this paper we use the Shapiro-Wilk and propose a new multivari- - ate generalization of Shapiro-Wilk to respec- tively test for univariate and multivariate - nor- mality of the code layer of a generative autoen- coder. By replacing the discriminator - in tradi- tional deep models with the hypothesis tests, we gain several advantages: - objectively evalu- ate whether the encoder is actually embedding data onto a normal + to be stripped away. In this paper we use the Shapiro-Wilk and propose a new multivariate generalization of Shapiro-Wilk to respectively test for univariate and multivariate + normality of the code layer of a generative autoencoder. By replacing the discriminator + in traditional deep models with the hypothesis tests, we gain several advantages: + objectively evaluate whether the encoder is actually embedding data onto a normal manifold, accurately define when convergence happens, explicitly balance between reconstruction and encoding training. Not only does our method produce competitive results, but it does so in a fraction of the time. We highlight the fact that the hypothesis tests used in our model asymptotically lead to the same solution of the - L2-Wasserstein distance metrics used by several generative models to- day.' + L2-Wasserstein distance metrics used by several generative models today.' title: Reforming Generative Autoencoders via Goodness-of-Fit Hypothesis Testing year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-pena18a.md b/_posts/2018-08-06-pena18a.md index 0e2d691..03b0849 100644 --- a/_posts/2018-08-06-pena18a.md +++ b/_posts/2018-08-06-pena18a.md @@ -1,10 +1,10 @@ --- abstract: The essential graph is a distinguished member of a Markov equivalence class - of AMP chain graphs. However, the directed edges in the es- sential graph are not - necessarily strong or in- variant, i.e. they may not be shared by every member of + of AMP chain graphs. However, the directed edges in the essential graph are not + necessarily strong or invariant, i.e. they may not be shared by every member of the equivalence class. Likewise for the undirected edges. In this paper, we develop a procedure for identifying which edges in an essential graph are strong. We also - show how this makes it possible to bound some causal ef- fects when the true chain + show how this makes it possible to bound some causal effects when the true chain graph is unknown. title: Identification of Strong Edges in AMP Chain Graphs year: '2018' diff --git a/_posts/2018-08-06-piatkowski18a.md b/_posts/2018-08-06-piatkowski18a.md index dac7be9..f5efd0e 100644 --- a/_posts/2018-08-06-piatkowski18a.md +++ b/_posts/2018-08-06-piatkowski18a.md @@ -1,17 +1,17 @@ --- abstract: Recent stochastic quadrature techniques for undirected graphical models - rely on near- minimax degree-k polynomial approximations to the model’s potential - function for inferring the partition function. While providing de- sirable statistical - guarantees, typical construc- tions of such approximations are themselves not amenable - to efficient inference. Here, we develop a class of Monte Carlo sampling algo- rithms + rely on nearminimax degree-k polynomial approximations to the model’s potential + function for inferring the partition function. While providing desirable statistical + guarantees, typical constructions of such approximations are themselves not amenable + to efficient inference. Here, we develop a class of Monte Carlo sampling algorithms for efficiently approximating the value of the partition function, as well as the - asso- ciated pseudo-marginals. More precisely, for pairwise models with n vertices + associated pseudo-marginals. More precisely, for pairwise models with n vertices and m edges, the complexity can be reduced from O(dk) to O(k4 + kn + m), where d $\geq$4m is the parameter dimension. We also consider the uses of stochastic quadrature - for the problem of maximum-likelihood (ML) parameter esti- mation. For completely + for the problem of maximum-likelihood (ML) parameter estimation. For completely observed data, our analysis gives rise to a probabilistic bound on the log-likelihood - of the model. Maxi- mizing this bound yields an approximate ML estimate which, in - analogy to the moment- matching of exact ML estimation, can be inter- preted in + of the model. Maximizing this bound yields an approximate ML estimate which, in + analogy to the momentmatching of exact ML estimation, can be interpreted in terms of pseudo-moment-matching. We present experimental results illustrating the behavior of this approximate ML estimator. title: Fast Stochastic Quadrature for Approximate Maximum-Likelihood Estimation diff --git a/_posts/2018-08-06-pinot18a.md b/_posts/2018-08-06-pinot18a.md index a56fc3a..09cda33 100644 --- a/_posts/2018-08-06-pinot18a.md +++ b/_posts/2018-08-06-pinot18a.md @@ -1,11 +1,11 @@ --- -abstract: In this paper, we present the first differen- tially private clustering - method for arbitrary- shaped node clusters in a graph. This algo- rithm takes as - input only an approximate Min- imum Spanning Tree (MST) T released under weight +abstract: In this paper, we present the first differentially private clustering + method for arbitraryshaped node clusters in a graph. This algorithm takes as + input only an approximate Minimum Spanning Tree (MST) T released under weight differential privacy constraints from the graph. Then, the underlying nonconvex - clus- tering partition is successfully recovered from cutting optimal cuts on T - . As opposed to ex- isting methods, our algorithm is theoretically well-motivated. - Experiments support our the- oretical findings. + clustering partition is successfully recovered from cutting optimal cuts on T + . As opposed to existing methods, our algorithm is theoretically well-motivated. + Experiments support our theoretical findings. title: Graph-based Clustering under Differential Privacy year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-rainforth18a.md b/_posts/2018-08-06-rainforth18a.md index 9774db5..3a5fdca 100644 --- a/_posts/2018-08-06-rainforth18a.md +++ b/_posts/2018-08-06-rainforth18a.md @@ -1,15 +1,15 @@ --- abstract: We formalize the notion of nesting probabilistic programming queries and - investigate the result- ing statistical implications. We demonstrate that while - query nesting allows the definition of models which could not otherwise be ex- pressed, - such as those involving agents reason- ing about other agents, existing systems + investigate the resulting statistical implications. We demonstrate that while + query nesting allows the definition of models which could not otherwise be expressed, + such as those involving agents reasoning about other agents, existing systems take approaches which lead to inconsistent estimates. We show how to correct this - by delineating pos- sible ways one might want to nest queries and asserting the - respective conditions required for convergence. We further introduce a new on- line + by delineating possible ways one might want to nest queries and asserting the + respective conditions required for convergence. We further introduce a new online nested Monte Carlo estimator that makes it substantially easier to ensure these conditions are met, thereby providing a simple framework for designing statistically - correct inference en- gines. We prove the correctness of this online estimator and - show that, when using the recom- mended setup, its asymptotic variance is always + correct inference engines. We prove the correctness of this online estimator and + show that, when using the recommended setup, its asymptotic variance is always better than that of the equivalent fixed estimator, while its bias is always within a factor of two. title: Nesting Probabilistic Programs diff --git a/_posts/2018-08-06-ren18a.md b/_posts/2018-08-06-ren18a.md index b2cbba4..a85a0a6 100644 --- a/_posts/2018-08-06-ren18a.md +++ b/_posts/2018-08-06-ren18a.md @@ -1,18 +1,18 @@ --- abstract: We developed a novel statistical method to identify structural differences - between net- works characterized by structural equation models. We propose to reparameterize + between networks characterized by structural equation models. We propose to reparameterize the model to separate the differential structures from common structures, and then design an algorithm with calibration and construction stages to identify these differential - structures. The calibration stage serves to obtain con- sistent prediction by building - the $\ell$2 regular- ized regression of each endogenous variables against pre-screened - exogenous variables, cor- recting for potential endogeneity issue. The construction - stage consistently selects and es- timates both common and differential effects + structures. The calibration stage serves to obtain consistent prediction by building + the $\ell$2 regularized regression of each endogenous variables against pre-screened + exogenous variables, correcting for potential endogeneity issue. The construction + stage consistently selects and estimates both common and differential effects by undertaking $\ell$1 regularized regression of each endogenous variable against - the predicts of other endogenous variables as well as its an- choring exogenous - variables. Our method al- lows easy parallel computation at each stage. Theoretical + the predicts of other endogenous variables as well as its anchoring exogenous + variables. Our method allows easy parallel computation at each stage. Theoretical results are obtained to establish non-asymptotic error bounds of predictions and - estimates at both stages, as well as the con- sistency of identified common and - differential effects. Our studies on synthetic data demon- strated that our proposed + estimates at both stages, as well as the consistency of identified common and + differential effects. Our studies on synthetic data demonstrated that our proposed method performed much better than independently constructing the networks. A real data set is analyzed to illustrate the applicability of our method. title: Differential Analysis of Directed Networks diff --git a/_posts/2018-08-06-rubenstein18a.md b/_posts/2018-08-06-rubenstein18a.md index 750f510..f2c001b 100644 --- a/_posts/2018-08-06-rubenstein18a.md +++ b/_posts/2018-08-06-rubenstein18a.md @@ -1,10 +1,10 @@ --- abstract: Structural Causal Models are widely used in causal modelling, but how they relate to other modelling tools is poorly understood. In this paper we provide a - novel perspective on the re- lationship between Ordinary Differential Equa- tions + novel perspective on the relationship between Ordinary Differential Equations and Structural Causal Models. We show how, under certain conditions, the asymptotic behaviour of an Ordinary Differential Equation under non-constant interventions - can be mod- elled using Dynamic Structural Causal Models. In contrast to earlier + can be modelled using Dynamic Structural Causal Models. In contrast to earlier work, we study not only the effect of interventions on equilibrium states; rather, we model asymptotic behaviour that is dynamic under interventions that vary in time, and include as a special case the study of static equilibria. diff --git a/_posts/2018-08-06-sabharwal18a.md b/_posts/2018-08-06-sabharwal18a.md index c5c098b..47f06eb 100644 --- a/_posts/2018-08-06-sabharwal18a.md +++ b/_posts/2018-08-06-sabharwal18a.md @@ -1,17 +1,16 @@ --- abstract: We propose a new algorithm for computing a constant-factor approximation - of precision- recall (PR) curves for massive noisy datasets produced by generative - models. Assessing va- lidity of items in such datasets requires human annotation, - which is costly and must be mini- mized. Our algorithm, ADASTRAT, is the first data-aware + of precisionrecall (PR) curves for massive noisy datasets produced by generative + models. Assessing validity of items in such datasets requires human annotation, + which is costly and must be minimized. Our algorithm, ADASTRAT, is the first data-aware method for this task. It chooses the next point to query on the PR curve adaptively, based on previous observations. It then selects specific items to annotate using - stratified sam- pling. Under a mild monotonicity assumption, ADASTRAT outputs a - guaranteed approxima- tion of the underlying precision function, while using a number - of annotations that scales very slowly with N, the dataset size. For exam- ple, - when the minimum precision is bounded by a constant, it issues only log log N preci- - sion queries. In general, it has a regret of no more than log log N w.r.t. an oracle - that is- sues queries at data-dependent (unknown) op- timal points. On a scaled-up - NLP dataset of 3.5M items, ADASTRAT achieves a remark- ably close approximation + stratified sampling. Under a mild monotonicity assumption, ADASTRAT outputs a + guaranteed approximation of the underlying precision function, while using a number + of annotations that scales very slowly with N, the dataset size. For example, + when the minimum precision is bounded by a constant, it issues only log log N precision queries. In general, it has a regret of no more than log log N w.r.t. an oracle + that issues queries at data-dependent (unknown) optimal points. On a scaled-up + NLP dataset of 3.5M items, ADASTRAT achieves a remarkably close approximation of the true precision function using only 18 precision queries, 13x fewer than best previous approaches. title: Adaptive Stratified Sampling for Precision-Recall Estimation diff --git a/_posts/2018-08-06-saemundsson18a.md b/_posts/2018-08-06-saemundsson18a.md index 5e2200a..207dbeb 100644 --- a/_posts/2018-08-06-saemundsson18a.md +++ b/_posts/2018-08-06-saemundsson18a.md @@ -1,13 +1,12 @@ --- abstract: Learning from small data sets is critical in many practical applications - where data col- lection is time consuming or expensive, e.g., robotics, animal experiments + where data collection is time consuming or expensive, e.g., robotics, animal experiments or drug design. Meta learning is one way to increase the data efficiency of learning - algorithms by general- izing learned concepts from a set of training tasks to unseen, - but related, tasks. Often, this relationship between tasks is hard coded or re- - lies in some other way on human expertise. In this paper, we frame meta learning - as a hi- erarchical latent variable model and infer the relationship between tasks - automatically from data. We apply our framework in a model- based reinforcement - learning setting and show that our meta-learning model effectively gen- eralizes + algorithms by generalizing learned concepts from a set of training tasks to unseen, + but related, tasks. Often, this relationship between tasks is hard coded or relies in some other way on human expertise. In this paper, we frame meta learning + as a hierarchical latent variable model and infer the relationship between tasks + automatically from data. We apply our framework in a model-based reinforcement + learning setting and show that our meta-learning model effectively generalizes to novel tasks by identifying how new tasks relate to prior ones from minimal data. This results in up to a 60% reduction in the average interaction time needed to solve tasks compared to strong baselines. diff --git a/_posts/2018-08-06-saha18a.md b/_posts/2018-08-06-saha18a.md index bb33a1f..3769d21 100644 --- a/_posts/2018-08-06-saha18a.md +++ b/_posts/2018-08-06-saha18a.md @@ -2,14 +2,14 @@ abstract: 'We introduce Battling-Bandits – an online learning framework where given a set of n arms, the learner needs to select a subset of k $\geq$2 arms in each round and subsequently observes a stochastic feedback indicating the winner of the - round. This framework generalizes the stan- dard Dueling-Bandit framework which + round. This framework generalizes the standard Dueling-Bandit framework which applies to several practical scenarios such as medical treatment preferences, recommender - systems, search engine optimization etc., where it is eas- ier and more effective + systems, search engine optimization etc., where it is easier and more effective to collect feedback for multiple options simultaneously. We develop a novel class of pairwise-subset choice model, for modelling the subset-wise winner feedback and propose three algorithms - Battling-Doubler, Battling-MultiSBM and Battling-Duel: While the first two are designed for a special class of linear-link based choice - models, the third one applies to a much general class of pairwise- subset choice + models, the third one applies to a much general class of pairwisesubset choice models with Condorcet winner. We also analyzed their regret guarantees and show the optimality of Battling-Duel proving a matching regret lower bound of $\Omega$(n log T), which (perhaps surprisingly) shows that the flexibility of playing size-k diff --git a/_posts/2018-08-06-sajed18a.md b/_posts/2018-08-06-sajed18a.md index ee7d549..85196ae 100644 --- a/_posts/2018-08-06-sajed18a.md +++ b/_posts/2018-08-06-sajed18a.md @@ -1,19 +1,18 @@ --- -abstract: Estimating the value function for a fixed pol- icy is a fundamental problem - in reinforcement learning. Policy evaluation algorithms—to es- timate value functions—continue - to be devel- oped, to improve convergence rates, improve stability and handle variability, - particularly for off-policy learning. To understand the prop- erties of these algorithms, - the experimenter needs high-confidence estimates of the accu- racy of the learned - value functions. For en- vironments with small, finite state-spaces, like chains, +abstract: Estimating the value function for a fixed policy is a fundamental problem + in reinforcement learning. Policy evaluation algorithms—to estimate value functions—continue + to be developed, to improve convergence rates, improve stability and handle variability, + particularly for off-policy learning. To understand the properties of these algorithms, + the experimenter needs high-confidence estimates of the accuracy of the learned + value functions. For environments with small, finite state-spaces, like chains, the true value function can be easily computed, to compute accuracy. For large, or continuous state-spaces, however, this is no longer feasible. In this paper, we address the largely open problem of how to obtain these high-confidence estimates, - for general state- spaces. We provide a high-confidence bound on an empirical estimate + for general state-spaces. We provide a high-confidence bound on an empirical estimate of the value error to the true value error. We use this bound to design an offline sampling algorithm, which stores the required quantities to repeatedly compute value - error estimates for any learned value function. We provide experiments in- vestigating - the number of samples required by this offline algorithm in simple benchmark re- - inforcement learning domains, and highlight that there are still many open questions + error estimates for any learned value function. We provide experiments investigating + the number of samples required by this offline algorithm in simple benchmark reinforcement learning domains, and highlight that there are still many open questions to be solved for this important problem. title: High-confidence error estimates for learned value functions year: '2018' diff --git a/_posts/2018-08-06-salah18a.md b/_posts/2018-08-06-salah18a.md index 8be523b..916655b 100644 --- a/_posts/2018-08-06-salah18a.md +++ b/_posts/2018-08-06-salah18a.md @@ -1,16 +1,16 @@ --- -abstract: We present Probabilistic Collaborative Repre- sentation Learning (PCRL), +abstract: We present Probabilistic Collaborative Representation Learning (PCRL), a new generative model of user preferences and item contexts. The latter builds - on the assumption that rela- tionships among items within contexts (e.g., browsing - session, shopping cart, etc.) may un- derlie various aspects that guide the choices - people make. Intuitively, PCRL seeks repre- sentations of items reflecting various - regulari- ties between them that might be useful at ex- plaining user preferences. + on the assumption that relationships among items within contexts (e.g., browsing + session, shopping cart, etc.) may underlie various aspects that guide the choices + people make. Intuitively, PCRL seeks representations of items reflecting various + regularities between them that might be useful at explaining user preferences. Formally, it relies on Bayesian Poisson Factorization to model user-item interactions, - and uses a multilayered latent variable architecture to learn represen- tations + and uses a multilayered latent variable architecture to learn representations of items from their contexts. PCRL seamlessly integrates both tasks within a joint - framework. However, inference and learn- ing under the proposed model are challenging - due to several sources of intractability. Rely- ing on the recent advances in approximate - in- ference/learning, we derive an efficient varia- tional algorithm to estimate + framework. However, inference and learning under the proposed model are challenging + due to several sources of intractability. Relying on the recent advances in approximate + inference/learning, we derive an efficient variational algorithm to estimate our model from observations. We further conduct experiments on several real-world datasets to showcase the benefits of the proposed model. title: Probabilistic Collaborative Representation Learning for Personalized Item Recommendation diff --git a/_posts/2018-08-06-seong18a.md b/_posts/2018-08-06-seong18a.md index d4f5142..5c85657 100644 --- a/_posts/2018-08-06-seong18a.md +++ b/_posts/2018-08-06-seong18a.md @@ -1,16 +1,16 @@ --- -abstract: Whereas optimizing deep neural networks us- ing stochastic gradient descent +abstract: Whereas optimizing deep neural networks using stochastic gradient descent has shown great performances in practice, the rule for setting step size (i.e. learning - rate) of gradient de- scent is not well studied. Although it appears that some intriguing + rate) of gradient descent is not well studied. Although it appears that some intriguing learning rate rules such as ADAM (Kingma and Ba, 2014) have since been developed, - they concentrated on improv- ing convergence, not on improving generaliza- tion - capabilities. Recently, the improved gen- eralization property of the flat minima - was re- visited, and this research guides us towards promising solutions to many - current optimiza- tion problems. In this paper, we analyze the flatness of loss - surfaces through the lens of ro- bustness to input perturbations and advocate that + they concentrated on improving convergence, not on improving generalization + capabilities. Recently, the improved generalization property of the flat minima + was revisited, and this research guides us towards promising solutions to many + current optimization problems. In this paper, we analyze the flatness of loss + surfaces through the lens of robustness to input perturbations and advocate that gradient descent should be guided to reach flatter region of loss surfaces to achieve - gen- eralization. Finally, we suggest a learning rate rule for escaping sharp regions - of loss surfaces, and we demonstrate the capacity of our ap- proach by performing + generalization. Finally, we suggest a learning rate rule for escaping sharp regions + of loss surfaces, and we demonstrate the capacity of our approach by performing numerous experiments. title: Towards Flatter Loss Surface via Nonmonotonic Learning Rate Scheduling year: '2018' diff --git a/_posts/2018-08-06-sharma18a.md b/_posts/2018-08-06-sharma18a.md index c4d02e2..80adae1 100644 --- a/_posts/2018-08-06-sharma18a.md +++ b/_posts/2018-08-06-sharma18a.md @@ -1,18 +1,18 @@ --- -abstract: 'Lifted inference reduces the complexity of in- ference in relational probabilistic +abstract: 'Lifted inference reduces the complexity of inference in relational probabilistic models by identifying groups of constants (or atoms) which behave symmetric to each other. A number of techniques have been proposed in the literature for lifting marginal as well MAP inference. We present the first application of lifting rules for marginal-MAP (MMAP), an important inference problem in models having latent (random) variables. - Our main contribu- tion is two fold: (1) we define a new equiv- alence class of - (logical) variables, called Sin- gle Occurrence for MAX (SOM), and show that solution + Our main contribution is two fold: (1) we define a new equivalence class of + (logical) variables, called Single Occurrence for MAX (SOM), and show that solution lies at extreme with respect to the SOM variables, i.e., predicate groundings differing only in the instantiation of the SOM variables take the same truth value (2) we - de- fine a sub-class SOM-R (SOM Reduce) and exploit properties of extreme assignments + define a sub-class SOM-R (SOM Reduce) and exploit properties of extreme assignments to show that MMAP inference can be performed by reducing the domain of SOM-R variables to a single constant. We refer to our lifting technique as the SOM-R rule for lifted - MMAP. Combined with existing rules such as decom- poser and binomial, this results - in a power- ful framework for lifted MMAP. Experiments on three benchmark domains + MMAP. Combined with existing rules such as decomposer and binomial, this results + in a powerful framework for lifted MMAP. Experiments on three benchmark domains show significant gains in both time and memory compared to ground inference as well as lifted approaches not using SOM-R.' title: Lifted Marginal MAP Inference diff --git a/_posts/2018-08-06-sherstan18a.md b/_posts/2018-08-06-sherstan18a.md index 3d6726f..0585eb5 100644 --- a/_posts/2018-08-06-sherstan18a.md +++ b/_posts/2018-08-06-sherstan18a.md @@ -1,20 +1,20 @@ --- abstract: Temporal-difference (TD) learning methods are widely used in reinforcement learning to estimate the expected return for each state, without a model, because - of their significant advantages in computational and data effi- ciency. For many + of their significant advantages in computational and data efficiency. For many applications involving risk mitigation, it would also be useful to estimate the variance of the return by TD methods. In this paper, we describe a way of doing this that is substantially simpler than those proposed by Tamar, Di Castro, and Mannor in 2012, or those proposed by White and White in 2016. We show that two TD - learners operating in series can learn expectation and variance esti- mates. The + learners operating in series can learn expectation and variance estimates. The trick is to use the square of the TD error of the expectation learner as the reward - of the variance learner, and the square of the ex- pectation learner’s discount + of the variance learner, and the square of the expectation learner’s discount rate as the discount rate of the variance learner. With these two modifications, the variance learning problem becomes a conventional TD learning problem to which - standard theoretical results can be ap- plied. Our formal results are limited to - the ta- ble lookup case, for which our method is still novel, but the extension - to function approxi- mation is immediate, and we provide some em- pirical results - for the linear function approx- imation case. Our experimental results show that + standard theoretical results can be applied. Our formal results are limited to + the table lookup case, for which our method is still novel, but the extension + to function approximation is immediate, and we provide some empirical results + for the linear function approximation case. Our experimental results show that our direct method behaves just as well as a comparable indirect method, but is generally more robust. title: Comparing Direct and Indirect Temporal-Difference Methods for Estimating the diff --git a/_posts/2018-08-06-shilton18a.md b/_posts/2018-08-06-shilton18a.md index 0f5c983..843f8cb 100644 --- a/_posts/2018-08-06-shilton18a.md +++ b/_posts/2018-08-06-shilton18a.md @@ -1,12 +1,12 @@ --- -abstract: In Bayesian Multi-Objective optimisation, ex- pected hypervolume improvement +abstract: In Bayesian Multi-Objective optimisation, expected hypervolume improvement is often used to measure the goodness of candidate solutions. However when there - are many ob- jectives the calculation of expected hyper- volume improvement can - become computa- tionally prohibitive. An alternative approach measures the goodness + are many objectives the calculation of expected hypervolume improvement can + become computationally prohibitive. An alternative approach measures the goodness of a candidate based on the distance of that candidate from the Pareto front in objective space. In this paper we present a novel distance-based Bayesian Many-Objective optimisation algorithm. We demonstrate the efficacy of our algorithm on three problems, - namely the DTLZ2 bench- mark problem, a hyper-parameter selection problem, and high-temperature + namely the DTLZ2 benchmark problem, a hyper-parameter selection problem, and high-temperature creep-resistant alloy design. title: Multi-Target Optimisation via Bayesian Optimisation and Linear Programming year: '2018' diff --git a/_posts/2018-08-06-shpakova18a.md b/_posts/2018-08-06-shpakova18a.md index c5de2a4..d278b3f 100644 --- a/_posts/2018-08-06-shpakova18a.md +++ b/_posts/2018-08-06-shpakova18a.md @@ -2,13 +2,12 @@ abstract: We consider the structured-output prediction problem through probabilistic approaches and generalize the “perturb-and-MAP” framework to more challenging weighted Hamming losses, which are crucial in applications. While in principle our approach - is a straightforward marginalization, it requires solving many re- lated MAP inference - problems. We show that for log-supermodular pairwise models these op- erations can - be performed efficiently using the machinery of dynamic graph cuts. We also pro- - pose to use double stochastic gradient descent, both on the data and on the perturbations, + is a straightforward marginalization, it requires solving many related MAP inference + problems. We show that for log-supermodular pairwise models these operations can + be performed efficiently using the machinery of dynamic graph cuts. We also propose to use double stochastic gradient descent, both on the data and on the perturbations, for efficient learning. Our framework can naturally take weak supervision (e.g., partial labels) into account. We conduct a set of experiments on medium-scale character - recognition and image segmentation, showing the benefits of our algo- rithms. + recognition and image segmentation, showing the benefits of our algorithms. title: Marginal Weighted Maximum Log-likelihood for Efficient Learning of Perturb-and-Map models year: '2018' diff --git a/_posts/2018-08-06-shpitser18a.md b/_posts/2018-08-06-shpitser18a.md index 89d3a28..6a8e67d 100644 --- a/_posts/2018-08-06-shpitser18a.md +++ b/_posts/2018-08-06-shpitser18a.md @@ -1,18 +1,18 @@ --- abstract: Unlike classical causal inference, where the goal is to estimate average - causal effects within a population, in settings such as per- sonalized medicine, - the goal is to map a unit’s characteristics to a treatment tailored to maxi- mize - the expected outcome for that unit. Ob- taining high-quality mappings of this type - is the goal of the dynamic treatment regime liter- ature. In healthcare settings, - optimizing poli- cies with respect to a particular causal pathway is often of interest + causal effects within a population, in settings such as personalized medicine, + the goal is to map a unit’s characteristics to a treatment tailored to maximize + the expected outcome for that unit. Obtaining high-quality mappings of this type + is the goal of the dynamic treatment regime literature. In healthcare settings, + optimizing policies with respect to a particular causal pathway is often of interest as well. In the context of average treatment effects, estimation of effects associated with causal pathways is considered in the mediation analysis literature. In this - paper, we combine mediation analy- sis and dynamic treatment regime ideas and consider + paper, we combine mediation analysis and dynamic treatment regime ideas and consider how unit characteristics may be used to tailor a treatment strategy that maximizes - an effect along specified sets of causal path- ways. In particular, we define counterfactual - responses to such policies, give a general iden- tification algorithm for these + an effect along specified sets of causal pathways. In particular, we define counterfactual + responses to such policies, give a general identification algorithm for these counterfactuals, and prove completeness of the algorithm for unrestricted policies. - A corollary of our re- sults is that the identification algorithm for re- sponses + A corollary of our results is that the identification algorithm for responses to policies given in [16] is complete for arbitrary policies. title: Identification of Personalized Effects Associated With Causal Pathways year: '2018' diff --git a/_posts/2018-08-06-shpitser18b.md b/_posts/2018-08-06-shpitser18b.md index 66bba8c..baa8e2e 100644 --- a/_posts/2018-08-06-shpitser18b.md +++ b/_posts/2018-08-06-shpitser18b.md @@ -1,19 +1,19 @@ --- -abstract: The conditional independence structure in- duced on the observed marginal +abstract: The conditional independence structure induced on the observed marginal distribution by a hidden variable directed acyclic graph (DAG) may be represented - by a graphical model rep- resented by mixed graphs called maximal an- cestral graphs - (MAGs). This model has a num- ber of desirable properties, in particular the set + by a graphical model represented by mixed graphs called maximal ancestral graphs + (MAGs). This model has a number of desirable properties, in particular the set of Gaussian distributions can be parameterized by viewing the graph as a path diagram. - Mod- els represented by MAGs have been used for causal discovery [22], and identification - theory for causal effects [28]. In addition to ordinary conditional indepen- dence + Models represented by MAGs have been used for causal discovery [22], and identification + theory for causal effects [28]. In addition to ordinary conditional independence constraints, hidden variable DAGs also induce generalized independence constraints. These constraints form the nested Markov property [20]. We first show that acyclic linear SEMs obey this property. Further we show that a natural parameterization - for all Gaussian dis- tributions obeying the nested Markov property arises from - a generalization of maximal ances- tral graphs that we call maximal arid graphs + for all Gaussian distributions obeying the nested Markov property arises from + a generalization of maximal ancestral graphs that we call maximal arid graphs (MArG). We show that every nested Markov model can be associated with a MArG; viewed as a path diagram this MArG parametrizes the Gaussian nested Markov model. This - leads di- rectly to methods for ML fitting and computing BIC scores for Gaussian + leads directly to methods for ML fitting and computing BIC scores for Gaussian nested models. title: Acyclic Linear SEMs Obey the Nested Markov Property year: '2018' diff --git a/_posts/2018-08-06-smith18a.md b/_posts/2018-08-06-smith18a.md index 6339df7..d9690e9 100644 --- a/_posts/2018-08-06-smith18a.md +++ b/_posts/2018-08-06-smith18a.md @@ -1,15 +1,15 @@ --- abstract: Measuring uncertainty is a promising technique for detecting adversarial - examples, crafted in- puts on which the model predicts an incorrect class with high + examples, crafted inputs on which the model predicts an incorrect class with high confidence. There are various measures of uncertainty, including predictive entropy and mutual information, each capturing distinct types of uncertainty. We study these - measures, and shed light on why mutual infor- mation seems to be effective at the - task of adver- sarial example detection. We highlight failure modes for MC dropout, + measures, and shed light on why mutual information seems to be effective at the + task of adversarial example detection. We highlight failure modes for MC dropout, a widely used approach for estimating uncertainty in deep models. This leads to - an improved understanding of the draw- backs of current methods, and a proposal - to im- prove the quality of uncertainty estimates using probabilistic model ensembles. - We give illustra- tive experiments using MNIST to demonstrate the intuition underlying - the different measures of uncertainty, as well as experiments on a real- world Kaggle + an improved understanding of the drawbacks of current methods, and a proposal + to improve the quality of uncertainty estimates using probabilistic model ensembles. + We give illustrative experiments using MNIST to demonstrate the intuition underlying + the different measures of uncertainty, as well as experiments on a real-world Kaggle dogs vs cats classification dataset. title: Understanding Measures of Uncertainty for Adversarial Example Detection year: '2018' diff --git a/_posts/2018-08-06-subbaswamy18a.md b/_posts/2018-08-06-subbaswamy18a.md index efd2d54..dfa1d00 100644 --- a/_posts/2018-08-06-subbaswamy18a.md +++ b/_posts/2018-08-06-subbaswamy18a.md @@ -1,15 +1,15 @@ --- abstract: Predictive models can fail to generalize from training to deployment environments - because of dataset shift, posing a threat to model re- liability in practice. As + because of dataset shift, posing a threat to model reliability in practice. As opposed to previous methods which use samples from the target distribution to reactively correct dataset shift, we propose using graphical knowledge of the causal mechanisms - relating variables in a pre- diction problem to proactively remove variables that + relating variables in a prediction problem to proactively remove variables that participate in spurious associations with the prediction target, allowing models - to gen- eralize across datasets. To accomplish this, we augment the causal graph - with latent counter- factual variables that account for the underlying causal mechanisms, - and show how we can es- timate these variables. In our experiments we demonstrate + to generalize across datasets. To accomplish this, we augment the causal graph + with latent counterfactual variables that account for the underlying causal mechanisms, + and show how we can estimate these variables. In our experiments we demonstrate that models using good estimates of the latent variables instead of the observed - variables transfer better from training to tar- get domains with minimal accuracy + variables transfer better from training to target domains with minimal accuracy loss in the training domain. title: 'Counterfactual Normalization: Proactively Addressing Dataset Shift Using Causal Mechanisms' diff --git a/_posts/2018-08-06-tan18a.md b/_posts/2018-08-06-tan18a.md index 168d119..c3212ad 100644 --- a/_posts/2018-08-06-tan18a.md +++ b/_posts/2018-08-06-tan18a.md @@ -1,14 +1,13 @@ --- -abstract: Linear mixed models (LMMs) are used exten- sively to model observations - that are not in- dependent. Parameter estimation for LMMs can be computationally +abstract: Linear mixed models (LMMs) are used extensively to model observations + that are not in-dependent. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational - complexity which depends at least linearly on the dimension p of the co- variates, - and often use heuristics that do not offer theoretical guarantees. We present scal- - able algorithms for learning high-dimensional LMMs with sublinear computational - complex- ity dependence on p. Key to our approach are novel dual estimators which + complexity which depends at least linearly on the dimension p of the covariates, + and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational + complexity dependence on p. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on - the subsampled random- ized Hadamard transform. We provide theo- retical guarantees - for our learning algorithms, demonstrating the robustness of parameter es- timation. + the subsampled randomized Hadamard transform. We provide theoretical guarantees + for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data. title: Scalable Algorithms for Learning High-Dimensional Linear Mixed Models year: '2018' diff --git a/_posts/2018-08-06-tan18b.md b/_posts/2018-08-06-tan18b.md index 367be8e..a8e4135 100644 --- a/_posts/2018-08-06-tan18b.md +++ b/_posts/2018-08-06-tan18b.md @@ -1,21 +1,21 @@ --- abstract: Temporal events in the real world often exhibit reinforcing dynamics, where earlier events trigger follow-up activity in the near future. A canonical example - of modeling such dy- namics is the Hawkes process (HP). However, previous HP models + of modeling such dynamics is the Hawkes process (HP). However, previous HP models do not capture the rich dynamics of real-world activity—which can be driven by multiple - latent triggering factors shared by past and future events, with the la- tent features + latent triggering factors shared by past and future events, with the latent features themselves exhibiting temporal dependency structures. For instance, rather than view a new document just as a response to other documents in the recent past, it - is impor- tant to account for the factor-structure under- lying all previous documents. + is important to account for the factor-structure underlying all previous documents. This structure itself is not fixed, with the influence of earlier documents decaying with time. To this end, we propose a novel Bayesian nonparametric stochastic point - process model, the Indian Buf- fet Hawkes Processes (IBHP), to learn multiple latent + process model, the Indian Buffet Hawkes Processes (IBHP), to learn multiple latent triggering factors underlying streaming document/message data. The IBP facilitates the inclusion of multiple triggering factors in the HP, and the HP allows for modeling - latent factor evolution in the IBP. We develop a learn- ing algorithm for the IBHP - based on Sequen- tial Monte Carlo and demonstrate the effective- ness of the model. - In both synthetic and real data experiments, our model achieves equiv- alent or - higher likelihood and provides inter- pretable topics and shows their dynamics. + latent factor evolution in the IBP. We develop a learning algorithm for the IBHP + based on Sequential Monte Carlo and demonstrate the effectiveness of the model. + In both synthetic and real data experiments, our model achieves equivalent or + higher likelihood and provides interpretable topics and shows their dynamics. title: The Indian Buffet Hawkes Process to Model Evolving Latent Influences year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-turner18a.md b/_posts/2018-08-06-turner18a.md index db78139..cf340c5 100644 --- a/_posts/2018-08-06-turner18a.md +++ b/_posts/2018-08-06-turner18a.md @@ -3,13 +3,13 @@ abstract: We present a data-driven benchmark system to evaluate the performance new MCMC samplers. Taking inspiration from the COCO benchmark in optimization, we view this benchmark as having critical importance to machine learning and statistics given the rate at which new samplers are proposed. The common hand-crafted examples - to test new samplers are unsatisfactory; we take a meta- learning-like approach + to test new samplers are unsatisfactory; we take a meta-learning-like approach to generate realistic benchmark examples from a large corpus of data sets and models. Surrogates of posteriors found in real problems are created using highly flexible - density models including modern neu- ral network models. We provide new insights + density models including modern neural network models. We provide new insights into the real effective sample size of various samplers per unit time and the estimation - effi- ciency of the samplers per sample. Addition- ally, we provide a meta-analysis - to assess the predictive utility of various MCMC diagnos- tics and perform a nonparametric + efficiency of the samplers per sample. Additionally, we provide a meta-analysis + to assess the predictive utility of various MCMC diagnostics and perform a nonparametric regression to combine them. title: How well does your sampler really work? year: '2018' diff --git a/_posts/2018-08-06-wang18a.md b/_posts/2018-08-06-wang18a.md index 677ea78..93e1642 100644 --- a/_posts/2018-08-06-wang18a.md +++ b/_posts/2018-08-06-wang18a.md @@ -1,20 +1,19 @@ --- abstract: 'Learning nonlinear dynamics from diffusion data is a challenging problem - since the individ- uals observed may be different at different time points, generally - following an aggregate be- haviour. Existing work cannot handle the tasks well since - they model such dynamics either di- rectly on observations or enforce the availabil- - ity of complete longitudinal individual-level trajectories. However, in most of - the practical applications, these requirements are unrealis- tic: the evolving dynamics + since the individuals observed may be different at different time points, generally + following an aggregate behaviour. Existing work cannot handle the tasks well since + they model such dynamics either directly on observations or enforce the availability of complete longitudinal individual-level trajectories. However, in most of + the practical applications, these requirements are unrealistic: the evolving dynamics may be too complex to be modeled directly on observations, and individual-level - trajectories may not be avail- able due to technical limitations, experimental costs + trajectories may not be available due to technical limitations, experimental costs and/or privacy issues. To address these challenges, we formulate a model of diffusion dynamics as the hidden stochastic process via the introduction of hidden variables - for flexi- bility, and learn the hidden dynamics directly on aggregate observations - without any require- ment for individual-level trajectories. We pro- pose a dynamic - generative model with Wasser- stein distance for LEarninG dEep hidden Non- linear - Dynamics (LEGEND) and prove its the- oretical guarantees as well. Experiments on - a range of synthetic and real-world datasets il- lustrate that LEGEND has very strong - perfor- mance compared to state-of-the-art baselines.' + for flexibility, and learn the hidden dynamics directly on aggregate observations + without any requirement for individual-level trajectories. We propose a dynamic + generative model with Wasserstein distance for LEarninG dEep hidden Non-linear + Dynamics (LEGEND) and prove its theoretical guarantees as well. Experiments on + a range of synthetic and real-world datasets illustrate that LEGEND has very strong + performance compared to state-of-the-art baselines.' title: Learning Deep Hidden Nonlinear Dynamics from Aggregate Data year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-wang18b.md b/_posts/2018-08-06-wang18b.md index 9b2b5f2..f004e0a 100644 --- a/_posts/2018-08-06-wang18b.md +++ b/_posts/2018-08-06-wang18b.md @@ -1,14 +1,13 @@ --- -abstract: 'We revisit the problem of linear regression un- der a differential privacy - constraint. By con- solidating existing pieces in the literature, we clarify the - correct dependence of the feature, la- bel and coefficient domains in the optimization +abstract: 'We revisit the problem of linear regression under a differential privacy + constraint. By consolidating existing pieces in the literature, we clarify the + correct dependence of the feature, label and coefficient domains in the optimization error and estimation error, hence revealing the delicate price of differential privacy - in statis- tical estimation and statistical learning. More- over, we propose simple + in statistical estimation and statistical learning. Moreover, we propose simple modifications of two existing DP algorithms: (a) posterior sampling, (b) sufficient - statistics perturbation, and show that they can be upgraded into adaptive algo- - rithms that are able to exploit data-dependent quantities and behave nearly optimally + statistics perturbation, and show that they can be upgraded into adaptive algorithms that are able to exploit data-dependent quantities and behave nearly optimally for every instance. Extensive experiments are conducted on both simulated data and - real data, which conclude that both ADAOPS and ADASSP out- perform the existing + real data, which conclude that both ADAOPS and ADASSP outperform the existing techniques on nearly all 36 data sets that we test on.' title: 'Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain' diff --git a/_posts/2018-08-06-wang18c.md b/_posts/2018-08-06-wang18c.md index 268d305..462a2af 100644 --- a/_posts/2018-08-06-wang18c.md +++ b/_posts/2018-08-06-wang18c.md @@ -1,15 +1,15 @@ --- -abstract: Advances in deep learning have led to substan- tial increases in prediction +abstract: Advances in deep learning have led to substantial increases in prediction accuracy but have been accompanied by increases in the cost of rendering predictions. - We conjecture that for a majority of real-world inputs, the recent ad- vances in + We conjecture that for a majority of real-world inputs, the recent advances in deep learning have created models that effectively “over-think” on simple inputs. In this paper we revisit the classic question of building model cascades that primarily leverage class asymmetry to reduce cost. We introduce the “I Don’t Know” (IDK) prediction - cascades framework, a general framework to systemat- ically compose a set of pre-trained - models to accelerate inference without a loss in predic- tion accuracy. We propose + cascades framework, a general framework to systematically compose a set of pre-trained + models to accelerate inference without a loss in prediction accuracy. We propose two search based methods for constructing cascades as well as a new cost-aware objective - within this frame- work. The proposed IDK cascade framework can be easily adopted - in the existing model serving systems without additional model re- training. We + within this framework. The proposed IDK cascade framework can be easily adopted + in the existing model serving systems without additional model retraining. We evaluate the proposed techniques on a range of benchmarks to demonstrate the effectiveness of the proposed framework. title: 'IDK Cascades: Fast Deep Learning by Learning not to Overthink' diff --git a/_posts/2018-08-06-yu18a.md b/_posts/2018-08-06-yu18a.md index b12042b..b59dea8 100644 --- a/_posts/2018-08-06-yu18a.md +++ b/_posts/2018-08-06-yu18a.md @@ -1,16 +1,16 @@ --- -abstract: Inspired by heavy-tailed distributions in prac- tical scenarios, we investigate +abstract: Inspired by heavy-tailed distributions in practical scenarios, we investigate the problem on pure exploration of Multi-Armed Bandits (MAB) with heavy-tailed payoffs by breaking the assumption of payoffs with sub-Gaussian noises in MAB, and assuming - that stochastic payoffs from bandits are with finite p-th mo- ments, where p $\in$(1, - +$\infty$). The main contri- butions in this paper are three-fold. First, we technically - analyze tail probabilities of empir- ical average and truncated empirical average - (TEA) for estimating expected payoffs in se- quential decisions with heavy-tailed + that stochastic payoffs from bandits are with finite p-th moments, where p $\in$(1, + +$\infty$). The main contributions in this paper are three-fold. First, we technically + analyze tail probabilities of empirical average and truncated empirical average + (TEA) for estimating expected payoffs in sequential decisions with heavy-tailed noises via martingales. Second, we propose two effective bandit algorithms based - on different prior in- formation (i.e., fixed confidence or fixed bud- get) for + on different prior information (i.e., fixed confidence or fixed budget) for pure exploration of MAB generating payoffs with finite p-th moments. Third, we derive theoretical guarantees for the proposed two bandit algorithms, and demonstrate the - ef- fectiveness of two algorithms in pure explo- ration of MAB with heavy-tailed + effectiveness of two algorithms in pure exploration of MAB with heavy-tailed payoffs in synthetic data and real-world financial data. title: Pure Exploration of Multi-Armed Bandits with Heavy-Tailed Payoffs year: '2018' diff --git a/_posts/2018-08-06-yuan18a.md b/_posts/2018-08-06-yuan18a.md index 0ee3eb8..86c6aae 100644 --- a/_posts/2018-08-06-yuan18a.md +++ b/_posts/2018-08-06-yuan18a.md @@ -1,20 +1,19 @@ --- abstract: Learning sparse features from only positive and unlabeled (PU) data is a - fundamental task for problems of several domains, such as natural lan- guage processing - (NLP), computer vision (CV), information retrieval (IR). Considering the nu- merous + fundamental task for problems of several domains, such as natural language processing + (NLP), computer vision (CV), information retrieval (IR). Considering the numerous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) - to in- crease computational efficiency. However, sam- pling a fraction of unlabeled + to increase computational efficiency. However, sampling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead - to non-optimal prediction perfor- mance. To address this, we present a fast and + to non-optimal prediction performance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples - without sam- pling. By leveraging sparsity in PU data, we ac- celerate fBGD by several - magnitudes, making its time complexity the same level as the NS- based stochastic - gradient descent method. Mean- while, we observe that the standard batch gradi- - ent method suffers from gradient instability is- sues due to the sparsity property. - Driven by a theoretical analysis for this potential cause, an in- tuitive solution - arises naturally. To verify its effi- cacy, we perform experiments on multiple tasks + without sampling. By leveraging sparsity in PU data, we accelerate fBGD by several + magnitudes, making its time complexity the same level as the NS-based stochastic + gradient descent method. Meanwhile, we observe that the standard batch gradient method suffers from gradient instability issues due to the sparsity property. + Driven by a theoretical analysis for this potential cause, an intuitive solution + arises naturally. To verify its efficacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based - mod- els on all tasks with comparable efficiency. + models on all tasks with comparable efficiency. title: "$f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD" year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-zhang18a.md b/_posts/2018-08-06-zhang18a.md index 162e5e2..72b8af1 100644 --- a/_posts/2018-08-06-zhang18a.md +++ b/_posts/2018-08-06-zhang18a.md @@ -1,12 +1,12 @@ --- abstract: We propose a new network architecture, Gated Attention Networks (GaAN), - for learning on graphs. Unlike the traditional multi-head at- tention mechanism, + for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network - to control each attention head’s importance. We demonstrate the effective- ness - of GaAN on the inductive node classi- fication problem on large graphs. Moreover, + to control each attention head’s importance. We demonstrate the effectiveness + of GaAN on the inductive node classification problem on large graphs. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) - to address the traffic speed forecasting prob- lem. Extensive experiments on three - real- world datasets show that our GaAN framework achieves state-of-the-art results + to address the traffic speed forecasting problem. Extensive experiments on three + real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks. title: 'GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs' year: '2018' diff --git a/_posts/2018-08-06-zhang18b.md b/_posts/2018-08-06-zhang18b.md index 4621600..4d3e18a 100644 --- a/_posts/2018-08-06-zhang18b.md +++ b/_posts/2018-08-06-zhang18b.md @@ -1,14 +1,14 @@ --- -abstract: One of the fundamental tasks in causal infer- ence is to decompose the observed +abstract: One of the fundamental tasks in causal inference is to decompose the observed association between a decision X and an outcome Y into its most basic structural mechanisms. In this paper, we introduce counterfactual measures for effects along - with a specific mechanism, represented as a path from X to Y in an ar- bitrary structural + with a specific mechanism, represented as a path from X to Y in an arbitrary structural causal model. We derive a novel non-parametric decomposition formula that expresses - the covariance of X and Y as a sum over unblocked paths from X to Y con- tained - in an arbitrary causal model. This for- mula allows a fine-grained path analysis - with- out requiring a commitment to any particular parametric form, and can be seen - as a gen- eralization of Wright’s decomposition method in linear systems (1923,1932) - and Pearl’s non- parametric mediation formula (2001). + the covariance of X and Y as a sum over unblocked paths from X to Y contained + in an arbitrary causal model. This formula allows a fine-grained path analysis + without requiring a commitment to any particular parametric form, and can be seen + as a generalization of Wright’s decomposition method in linear systems (1923,1932) + and Pearl’s non-parametric mediation formula (2001). title: Non-Parametric Path Analysis in Structural Causal Models year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-zhang18c.md b/_posts/2018-08-06-zhang18c.md index 3527594..90e1fa3 100644 --- a/_posts/2018-08-06-zhang18c.md +++ b/_posts/2018-08-06-zhang18c.md @@ -1,16 +1,16 @@ --- abstract: Causal discovery methods aim to recover the causal process that generated - purely observa- tional data. Despite its successes on a number of real problems, - the presence of measurement error in the observed data can produce seri- ous mistakes + purely observational data. Despite its successes on a number of real problems, + the presence of measurement error in the observed data can produce serious mistakes in the output of various causal discovery methods. Given the ubiquity of measurement error caused by instruments or proxies used in the measuring process, this problem - is one of the main obstacles to reli- able causal discovery. It is still unknown + is one of the main obstacles to reliable causal discovery. It is still unknown to what extent the causal structure of relevant variables can be identified in principle. This study aims to take a step towards filling that void. We assume that the underlining - pro- cess or the measurement-error free variables follows a linear, non-Guassian + process or the measurement-error free variables follows a linear, non-Guassian causal model, and show that the so-called ordered group decomposition of the causal - model, which con- tains major causal information, is identifiable. The causal structure - identifiability is further improved with different types of sparsity con- straints + model, which contains major causal information, is identifiable. The causal structure + identifiability is further improved with different types of sparsity constraints on the causal structure. Finally, we give rather mild conditions under which the whole causal structure is fully identifiable. title: 'Causal Discovery with Linear Non-Gaussian Models under Measurement Error: diff --git a/_posts/2018-08-06-zhao18a.md b/_posts/2018-08-06-zhao18a.md index 985bff2..6db2f44 100644 --- a/_posts/2018-08-06-zhao18a.md +++ b/_posts/2018-08-06-zhao18a.md @@ -1,16 +1,16 @@ --- abstract: Symmetric nonnegative matrix factorization has found abundant applications - in various do- mains by providing a symmetric low-rank de- composition of nonnegative + in various domains by providing a symmetric low-rank decomposition of nonnegative matrices. In this paper we propose a Frank-Wolfe (FW) solver to optimize the symmetric - nonnegative matrix factorization problem under a simplicial con- straint, which - has recently been proposed for probabilistic clustering. Compared with exist- ing - solutions, this algorithm is simple to imple- ment, and has no hyperparameters to - be tuned. Building on the recent advances of FW algo- rithms in nonconvex optimization, + nonnegative matrix factorization problem under a simplicial constraint, which + has recently been proposed for probabilistic clustering. Compared with existing + solutions, this algorithm is simple to implement, and has no hyperparameters to + be tuned. Building on the recent advances of FW algorithms in nonconvex optimization, we prove an O(1/$\varepsilon$2) convergence rate to $\varepsilon$-approximate KKT - points, via a tight bound $\Theta$(n2) on the cur- vature constant, which matches + points, via a tight bound $\Theta$(n2) on the curvature constant, which matches the best known result in unconstrained nonconvex setting using gradient methods. - Numerical results demon- strate the effectiveness of our algorithm. As a side contribution, - we construct a simple nons- mooth convex problem where the FW algorithm fails to + Numerical results demonstrate the effectiveness of our algorithm. As a side contribution, + we construct a simple nonsmooth convex problem where the FW algorithm fails to converge to the optimum. This result raises an interesting question about necessary conditions of the success of the FW algorithm on convex problems. title: Frank-Wolfe Optimization for Symmetric-NMF under Simplicial Constraint diff --git a/_posts/2018-08-06-zhao18b.md b/_posts/2018-08-06-zhao18b.md index 2854a98..c40e4ed 100644 --- a/_posts/2018-08-06-zhao18b.md +++ b/_posts/2018-08-06-zhao18b.md @@ -4,11 +4,11 @@ abstract: We propose a cost-effective framework for preference elicitation and a computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the framework with experiments on Amazon Mechanical Turk, which we use to estimate the cost of answering - different types of elicitation ques- tions. We compare the prediction accuracy of - our framework when adopting various infor- mation criteria that evaluate the expected - infor- mation gain from a question. Our experiments show carefully designed information + different types of elicitation questions. We compare the prediction accuracy of + our framework when adopting various information criteria that evaluate the expected + information gain from a question. Our experiments show carefully designed information criteria are much more efficient, i.e., they arrive at the correct answer using - fewer queries, than ran- domly asking questions given the budget con- straint. + fewer queries, than randomly asking questions given the budget constraint. title: A Cost-Effective Framework for Preference Elicitation and Aggregation year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-zhao18c.md b/_posts/2018-08-06-zhao18c.md index 3bd3459..f304824 100644 --- a/_posts/2018-08-06-zhao18c.md +++ b/_posts/2018-08-06-zhao18c.md @@ -1,17 +1,16 @@ --- abstract: A large number of objectives have been proposed to train latent variable generative models. We show that many of them are Lagrangian dual functions of the - same primal optimization prob- lem. The primal problem optimizes the mutual information - between latent and visible variables, subject to the constraints of accurately model- - ing the data distribution and performing correct amortized inference. By choosing + same primal optimization problem. The primal problem optimizes the mutual information + between latent and visible variables, subject to the constraints of accurately modeling the data distribution and performing correct amortized inference. By choosing to maximize or minimize mutual information, and choosing different Lagrange multipliers, - we obtain differ- ent objectives including InfoGAN, ALI/BiGAN, ALICE, CycleGAN, - beta-VAE, adversarial au- toencoders, AVB, AS-VAE and InfoVAE. Based on this observation, - we provide an exhaustive characterization of the statistical and computa- tional - trade-offs made by all the training objec- tives in this class of Lagrangian duals. + we obtain different objectives including InfoGAN, ALI/BiGAN, ALICE, CycleGAN, + beta-VAE, adversarial autoencoders, AVB, AS-VAE and InfoVAE. Based on this observation, + we provide an exhaustive characterization of the statistical and computational + trade-offs made by all the training objectives in this class of Lagrangian duals. Next, we propose a dual optimization method where we optimize model parameters as - well as the La- grange multipliers. This method achieves Pareto optimal solutions - in terms of optimizing informa- tion and satisfying the constraints. + well as the Lagrange multipliers. This method achieves Pareto optimal solutions + in terms of optimizing information and satisfying the constraints. title: A Lagrangian Perspective on Latent Variable Generative Models year: '2018' layout: inproceedings diff --git a/_posts/2018-08-06-zheng18a.md b/_posts/2018-08-06-zheng18a.md index c5ff4fe..451bb13 100644 --- a/_posts/2018-08-06-zheng18a.md +++ b/_posts/2018-08-06-zheng18a.md @@ -1,20 +1,20 @@ --- abstract: We consider partially-specified optimization problems where the goal is to actively, but efficiently, acquire missing information about the problem in order - to solve it. An algo- rithm designer wishes to solve a linear pro- gram (LP), max - cT x s.t. Ax $\leq$b, x $\geq$0, but does not initially know some of the pa- rameters. + to solve it. An algorithm designer wishes to solve a linear program (LP), max + cT x s.t. Ax $\leq$b, x $\geq$0, but does not initially know some of the parameters. The algorithm can iteratively choose an unknown parameter and gather information in the form of a noisy sample centered at the parameter’s (unknown) value. The goal - is to find an approximately feasible and optimal so- lution to the underlying LP - with high proba- bility while drawing a small number of sam- ples. We focus on two - cases. (1) When the parameters b of the constraints are initially un- known, we - propose an efficient algorithm com- bining techniques from the ellipsoid method + is to find an approximately feasible and optimal solution to the underlying LP + with high probability while drawing a small number of samples. We focus on two + cases. (1) When the parameters b of the constraints are initially un-known, we + propose an efficient algorithm combining techniques from the ellipsoid method for LP and confidence-bound approaches from bandit algorithms. The algorithm adaptively gathers information about constraints only as needed in order to make progress. We give sample complexity bounds for the algorithm and demonstrate its improvement - over a naive approach via simulation. (2) When the param- eters c of the objective + over a naive approach via simulation. (2) When the parameters c of the objective are initially unknown, we take an information-theoretic approach and give roughly - matching upper and lower sam- ple complexity bounds, with an (inefficient) successive-elimination + matching upper and lower sample complexity bounds, with an (inefficient) successive-elimination algorithm. title: Active Information Acquisition for Linear Optimization year: '2018' diff --git a/_posts/2018-08-06-zheng18b.md b/_posts/2018-08-06-zheng18b.md index 9cd8f4f..b9c7cba 100644 --- a/_posts/2018-08-06-zheng18b.md +++ b/_posts/2018-08-06-zheng18b.md @@ -1,19 +1,19 @@ --- abstract: 'We propose a framework for the completely unsupervised learning of latent - object prop- erties from their interactions: the perception- prediction network - (PPN). Consisting of a per- ception module that extracts representations of latent + object properties from their interactions: the perceptionprediction network + (PPN). Consisting of a perception module that extracts representations of latent object properties and a prediction module that uses those extracted properties to simulate system dynamics, the PPN can be trained in an end-to-end fashion purely from samples of object dynamics. The representations of latent object properties learned by PPNs not only are sufficient to accurately simulate the dynamics of systems - comprised of previously unseen ob- jects, but also can be translated directly into - human-interpretable properties (e.g. mass, co- efficient of restitution) in an entirely - unsuper- vised manner. Crucially, PPNs also generalize to novel scenarios: their + comprised of previously unseen objects, but also can be translated directly into + human-interpretable properties (e.g. mass, coefficient of restitution) in an entirely + unsupervised manner. Crucially, PPNs also generalize to novel scenarios: their gradient-based training can be applied to many dynamical systems and their graph-based - structure functions over sys- tems comprised of different numbers of objects. Our - results demonstrate the efficacy of graph- based neural architectures in object-centric - in- ference and prediction tasks, and our model has the potential to discover relevant - object proper- ties in systems that are not yet well understood.' + structure functions over systems comprised of different numbers of objects. Our + results demonstrate the efficacy of graph-based neural architectures in object-centric + inference and prediction tasks, and our model has the potential to discover relevant + object properties in systems that are not yet well understood.' title: Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks year: '2018' diff --git a/_posts/2018-08-06-zhou18a.md b/_posts/2018-08-06-zhou18a.md index fb763ca..59a5d49 100644 --- a/_posts/2018-08-06-zhou18a.md +++ b/_posts/2018-08-06-zhou18a.md @@ -1,13 +1,13 @@ --- abstract: The goal of many machine learning tasks is to learn a model that has small population risk. While mini-batch stochastic gradient descent (SGD) and variants - are popular approaches for achieving this goal, it is hard to pre- scribe a clear + are popular approaches for achieving this goal, it is hard to prescribe a clear stopping criterion and to establish high probability convergence bounds to the population risk. In this paper, we introduce Stable Gradient Descent which validates stochastic - gra- dient computations by splitting data into training and validation sets and - reuses samples using a differential pri- vate mechanism. StGD comes with a natural + gradient computations by splitting data into training and validation sets and + reuses samples using a differential private mechanism. StGD comes with a natural upper bound on the number of iterations and has high-probability convergence to - the population risk. Ex- perimental results illustrate that StGD is empirically + the population risk. Experimental results illustrate that StGD is empirically competitive and often better than SGD and GD. title: Stable Gradient Descent year: '2018' diff --git a/_posts/2018-08-06-zhu18a.md b/_posts/2018-08-06-zhu18a.md index 7fc6637..6021497 100644 --- a/_posts/2018-08-06-zhu18a.md +++ b/_posts/2018-08-06-zhu18a.md @@ -1,9 +1,9 @@ --- -abstract: Estimating the dynamic connectivity struc- ture among a system of entities - has gar- nered much attention in recent years. While usual methods are designed +abstract: Estimating the dynamic connectivity structure among a system of entities + has garnered much attention in recent years. While usual methods are designed to take advantage of temporal consistency to overcome noise, they conflict with - the detectability of anoma- lies. We propose Clustered Fused Graphi- cal Lasso (CFGL), - a method using precom- puted clustering information to improve the signal detectability + the detectability of anomalies. We propose Clustered Fused Graphical Lasso (CFGL), + a method using precomputed clustering information to improve the signal detectability as compared to typical Fused Graphical Lasso methods. We evaluate our method in both simulated and real-world datasets and conclude that, in many cases, CFGL can significantly improve the sensitivity to signals without a significant negative diff --git a/_posts/2018-08-06-zou18a.md b/_posts/2018-08-06-zou18a.md index 80f470c..d24c975 100644 --- a/_posts/2018-08-06-zou18a.md +++ b/_posts/2018-08-06-zou18a.md @@ -1,15 +1,15 @@ --- abstract: Stochastic variance-reduced gradient Langevin dynamics (SVRG-LD) was recently - proposed to improve the performance of stochastic gra- dient Langevin dynamics (SGLD) - by reduc- ing the variance of the stochastic gradient. In this paper, we propose + proposed to improve the performance of stochastic gradient Langevin dynamics (SGLD) + by reducing the variance of the stochastic gradient. In this paper, we propose a variant of SVRG-LD, namely SVRG-LD+, which replaces the full gradient in each epoch with a subsampled one. We provide a nonasymptotic analysis of the convergence of SVRG-LD+ in 2-Wasserstein distance, and show that SVRG-LD+ enjoys a lower gradient - complexity1 than SVRG-LD, when the sample size is large or the target ac- curacy + complexity1 than SVRG-LD, when the sample size is large or the target accuracy requirement is moderate. Our analysis directly implies a sharper convergence rate - for SVRG-LD, which improves the existing con- vergence rate by a factor of $\kappa$1/6n1/6, + for SVRG-LD, which improves the existing convergence rate by a factor of $\kappa$1/6n1/6, where $\kappa$ is the condition number of the log-density function and n is the - sample size. Experiments on both synthetic and real-world datasets vali- date our + sample size. Experiments on both synthetic and real-world datasets validate our theoretical results. title: Subsampled Stochastic Variance-Reduced Gradient Langevin Dynamics year: '2018' From 0e305a8dc425fefccdbe76c594e506bf980cd2aa Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 10:39:07 +0100 Subject: [PATCH 2/7] Re-fix wrap hyphens using dictionary join check. Keep short non-words hyphenated (e.g. abc-mart) while still joining ordinary line-wraps. Regenerated from gh-pages originals. --- _posts/2018-08-06-eismann18a.md | 2 +- _posts/2018-08-06-hewitt18a.md | 2 +- _posts/2018-08-06-krishnan18a.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/_posts/2018-08-06-eismann18a.md b/_posts/2018-08-06-eismann18a.md index a287eeb..f1ba37b 100644 --- a/_posts/2018-08-06-eismann18a.md +++ b/_posts/2018-08-06-eismann18a.md @@ -4,7 +4,7 @@ abstract: Automatic design via Bayesian optimization holds great promise given t from high-dimensional, potentially discrete, search spaces. We propose to probabilistically embed inputs into a lower dimensional, continuous latent space, where we perform gradient-based optimization guided by a Gaussian process. Building on variational - autoncoders, we use both labeled and unlabeled data to guide the encoding and + au-toncoders, we use both labeled and unlabeled data to guide the encoding and increase its accuracy. In addition, we propose an adversarial extension to render the latent representation invariant with respect to specific design attributes, which allows us to transfer these attributes across structures. We apply the framework both to a functional-protein dataset and to perform optimization of drag coefficients diff --git a/_posts/2018-08-06-hewitt18a.md b/_posts/2018-08-06-hewitt18a.md index b05fdcf..0be84db 100644 --- a/_posts/2018-08-06-hewitt18a.md +++ b/_posts/2018-08-06-hewitt18a.md @@ -6,7 +6,7 @@ abstract: Hierarchical Bayesian methods can unify many related tasks (e.g. k-sho typically fail to effectively use latent variables. To address this, we develop a modification of the Variational Autoencoder in which encoded observations are decoded to new elements from the same class. This technique, which we call a Variational - Homoencoder (VHE), produces a hierarchical latent variable model which better + Ho-moencoder (VHE), produces a hierarchical latent variable model which better utilises latent variables. We use the VHE framework to learn a hierarchical PixelCNN on the Omniglot dataset, which outperforms all existing models on test set likelihood and achieves strong performance on one-shot generation and classification tasks. diff --git a/_posts/2018-08-06-krishnan18a.md b/_posts/2018-08-06-krishnan18a.md index a097b2b..8ce7d9a 100644 --- a/_posts/2018-08-06-krishnan18a.md +++ b/_posts/2018-08-06-krishnan18a.md @@ -9,7 +9,7 @@ abstract: We propose a new way to answer probabilistic queries that span multipl on a query-conditional latent reasoning network, that parameterizes a distribution over the latent space of the deep generative model. The latent reasoning network is trained to amortize the posterior-predictive distribution under a hierarchical - model using supervised data and a maxmargin learning algorithm. We explore how + model using supervised data and a max-margin learning algorithm. We explore how the model may be used to focus the data variations captured in the latent space of the deep generative model and how this may be used to build new algorithms for few-shot learning. From 170013b7a864da259dcbe09fb99c72b775511b58 Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 16:59:14 +0100 Subject: [PATCH 3/7] Update _posts/2018-08-06-chaudhuri18a.md --- _posts/2018-08-06-chaudhuri18a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2018-08-06-chaudhuri18a.md b/_posts/2018-08-06-chaudhuri18a.md index 2d28679..d5e9ae2 100644 --- a/_posts/2018-08-06-chaudhuri18a.md +++ b/_posts/2018-08-06-chaudhuri18a.md @@ -1,5 +1,5 @@ --- -abstract: 'The stochastic multi-armed bandit is a wellstudied abstraction of decision +abstract: 'The stochastic multi-armed bandit is a well studied abstraction of decision making in the face of uncertainty. We consider the setting in which the number of bandit arms is much larger than the possible number of pulls, and can even be infinite. With the aim of minimising regret with respect to an optimal arm, existing methods From 06c0ed4d8d85c439f32c29873185b337d265c777 Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 16:59:24 +0100 Subject: [PATCH 4/7] Update _posts/2018-08-06-kim18a.md --- _posts/2018-08-06-kim18a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2018-08-06-kim18a.md b/_posts/2018-08-06-kim18a.md index 0a346e2..371b0da 100644 --- a/_posts/2018-08-06-kim18a.md +++ b/_posts/2018-08-06-kim18a.md @@ -4,7 +4,7 @@ abstract: 'Gaussian process survival analysis model (GP- SAM) was recently propo to account for uncertainty in the hazard function modeling while, at the same time, relaxing the time-covariates factorized assumption of the Cox model. However, the existing MCMC inference algorithms for GPSAM have proven to be slow in practice. In this paper we propose novel and scalable variational inference - algorithms for GP- SAM that reduce the time complexity of the sampling approaches + algorithms for GPSAM that reduce the time complexity of the sampling approaches and improve scalability to large datasets. We accomplish this by employing two effective strategies in scalable GP: i) using pseudo inputs and ii) approximation via random feature expansions. In both setups, we derive the full and partial likelihood From 72d3a86752fd52d9389859836b6385a50b5ee70e Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 16:59:32 +0100 Subject: [PATCH 5/7] Update _posts/2018-08-06-piatkowski18a.md --- _posts/2018-08-06-piatkowski18a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2018-08-06-piatkowski18a.md b/_posts/2018-08-06-piatkowski18a.md index f5efd0e..984affa 100644 --- a/_posts/2018-08-06-piatkowski18a.md +++ b/_posts/2018-08-06-piatkowski18a.md @@ -7,7 +7,7 @@ abstract: Recent stochastic quadrature techniques for undirected graphical model for efficiently approximating the value of the partition function, as well as the associated pseudo-marginals. More precisely, for pairwise models with n vertices and m edges, the complexity can be reduced from O(dk) to O(k4 + kn + m), where d - $\geq$4m is the parameter dimension. We also consider the uses of stochastic quadrature + \geq 4m$ is the parameter dimension. We also consider the uses of stochastic quadrature for the problem of maximum-likelihood (ML) parameter estimation. For completely observed data, our analysis gives rise to a probabilistic bound on the log-likelihood of the model. Maximizing this bound yields an approximate ML estimate which, in From 36f8e6e78346d362e67fb8347c48557c9a56e783 Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 16:59:40 +0100 Subject: [PATCH 6/7] Update _posts/2018-08-06-zhao18a.md --- _posts/2018-08-06-zhao18a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2018-08-06-zhao18a.md b/_posts/2018-08-06-zhao18a.md index 6db2f44..53ce092 100644 --- a/_posts/2018-08-06-zhao18a.md +++ b/_posts/2018-08-06-zhao18a.md @@ -7,7 +7,7 @@ abstract: Symmetric nonnegative matrix factorization has found abundant applicat solutions, this algorithm is simple to implement, and has no hyperparameters to be tuned. Building on the recent advances of FW algorithms in nonconvex optimization, we prove an O(1/$\varepsilon$2) convergence rate to $\varepsilon$-approximate KKT - points, via a tight bound $\Theta$(n2) on the curvature constant, which matches + points, via a tight bound $\Theta(n^2)$ on the curvature constant, which matches the best known result in unconstrained nonconvex setting using gradient methods. Numerical results demonstrate the effectiveness of our algorithm. As a side contribution, we construct a simple nonsmooth convex problem where the FW algorithm fails to From 2457a0bf0978faac726bf1a027bf673c240e1bbe Mon Sep 17 00:00:00 2001 From: Neil Lawrence Date: Tue, 6 Oct 2026 16:59:51 +0100 Subject: [PATCH 7/7] Update _posts/2018-08-06-zou18a.md --- _posts/2018-08-06-zou18a.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2018-08-06-zou18a.md b/_posts/2018-08-06-zou18a.md index d24c975..59d317d 100644 --- a/_posts/2018-08-06-zou18a.md +++ b/_posts/2018-08-06-zou18a.md @@ -7,7 +7,7 @@ abstract: Stochastic variance-reduced gradient Langevin dynamics (SVRG-LD) was r of SVRG-LD+ in 2-Wasserstein distance, and show that SVRG-LD+ enjoys a lower gradient complexity1 than SVRG-LD, when the sample size is large or the target accuracy requirement is moderate. Our analysis directly implies a sharper convergence rate - for SVRG-LD, which improves the existing convergence rate by a factor of $\kappa$1/6n1/6, + for SVRG-LD, which improves the existing convergence rate by a factor of $\kappa^{1/6}n^{1/6}$, where $\kappa$ is the condition number of the log-density function and n is the sample size. Experiments on both synthetic and real-world datasets validate our theoretical results.