• This document presents a, (mostly) chronologically ordered, bibliography of scientific publications on the superiorization methodology and perturbation resilience of algorithms which is compiled and continuously updated by us at: http://math.haifa.ac.il/yair/bib-superiorization-censor.html. Since the beginings of this topic we try to trace the work that has been published about it since its inception. To the best of our knowledge this bibliography represents all available publications on this topic to date, and while the URL is continuously updated we will revise this document and bring it up to date on arXiv approximately once a year. Abstracts of the cited works, and some links and downloadable files of preprints or reprints are available on the above mentioned Internet page. If you know of a related scientific work in any form that should be included here kindly write to me on: yair@math.haifa.ac.il with full bibliographic details, a DOI if available, and a PDF copy of the work if possible. The Internet page was initiated on March 7, 2015, and has been last updated on March 11, 2019.
  • Given two disjoint convex polyhedra, we look for a best approximation pair relative to them, i.e., a pair of points, one in each polyhedron, attaining the minimum distance between the sets. Cheney and Goldstein showed that alternating projections onto the two sets, starting from an arbitrary point, generate a sequence whose two interlaced subsequences converge to a best approximation pair. We propose a process based on projections onto the half-spaces defining the two polyhedra, which are more negotiable than projections on the polyhedra themselves. A central component in the proposed process is the Halpern--Lions--Wittmann--Bauschke algorithm for approaching the projection of a given point onto a convex set.
  • The convex feasibility problem (CFP) is to find a feasible point in the intersection of finitely many convex and closed sets. If the intersection is empty then the CFP is inconsistent and a feasible point does not exist. However, algorithmic research of inconsistent CFPs exists and is mainly focused on two directions. One is oriented toward defining other solution concepts that will apply, such as proximity function minimization wherein a proximity function measures in some way the total violation of all constraints. The second direction investigates the behavior of algorithms that are designed to solve a consistent CFP when applied to inconsistent problems. This direction is fueled by situations wherein one lacks a priori information about the consistency or inconsistency of the CFP or does not wish to invest computational resources to get hold of such knowledge prior to running his algorithm. In this paper we bring under one roof and telegraphically review some recent works on inconsistent CFPs.
  • Superiorization reduces, not necessarily minimizes, the value of a target function while seeking constraints-compatibility. This is done by taking a solely feasibility-seeking algorithm, analyzing its perturbations resilience, and proactively perturbing its iterates accordingly to steer them toward a feasible point with reduced value of the target function. When the perturbation steps are computationally efficient, this enables generation of a superior result with essentially the same computational cost as that of the original feasibility-seeking algorithm. In this work, we refine previous formulations of the superiorization method to create a more general framework, enabling target function reduction steps that do not require partial derivatives of the target function. In perturbations that use partial derivatives the step-sizes in the perturbation phase of the superiorization method are chosen independently from the choice of the nonascent directions. This is no longer true when component-wise perturbations are employed. In that case, the step-sizes must be linked to the choice of the nonascent direction in every step. Besides presenting and validating these notions, we give a computational demonstration of superiorization with component-wise perturbations for a problem of computerized tomography image reconstruction.
  • Previous work showed that total variation superiorization (TVS) improves reconstructed image quality in proton computed tomography (pCT). The structure of the TVS algorithm has evolved since then and this work investigated if this new algorithmic structure provides additional benefits to pCT image quality. Structural and parametric changes introduced to the original TVS algorithm included: (1) inclusion or exclusion of TV reduction requirement, (2) a variable number, $N$, of TV perturbation steps per feasibility-seeking iteration, and (3) introduction of a perturbation kernel $0<\alpha<1$. The structural change of excluding the TV reduction requirement check tended to have a beneficial effect for $3\le N\le 6$ and allows full parallelization of the TVS algorithm. Repeated perturbations per feasibility-seeking iterations reduced total variation (TV) and material dependent standard deviations for $3\le N\le 6$. The perturbation kernel $\alpha$, equivalent to $\alpha=0.5$ in the original TVS algorithm, reduced TV and standard deviations as $\alpha$ was increased beyond $\alpha=0.5$, but negatively impacted reconstructed relative stopping power (RSP) values for $\alpha>0.75$. The reductions in TV and standard deviations allowed feasibility-seeking with a larger relaxation parameter $\lambda$ than previously used, without the corresponding increases in standard deviations experienced with the original TVS algorithm. This work demonstrates that the modifications related to the evolution of the original TVS algorithm provide benefits in terms of both pCT image quality and computational efficiency for appropriately chosen parameter values.
  • The Douglas-Rachford (DR) algorithm is an iterative procedure that uses sequential reflections onto convex sets and which has become popular for convex feasibility problems. In this paper we propose a structural generalization that allows to use $r$-sets-DR operators in a cyclic fashion. We prove convergence and illustrate the advantage of such operators with $r>2$ over the classical $2$-sets-DR operators in a cyclic algorithm.
  • Convex feasibility problems require to find a point in the intersection of a finite family of convex sets. We propose to solve such problems by performing set-enlargements and applying a new kind of projection operators called valiant projectors. A valiant projector onto a convex set implements a special relaxation strategy, proposed by Goffin in 1971, that dictates the move toward the projection according to the distance from the set. Contrary to past realizations of this strategy, our valiant projection operator implements the strategy in a continuous fashion. We study properties of valiant projectors and prove convergence of our new valiant projections method. These results include as a special case and extend the 1985 automatic relaxation method of Censor.
  • Linear superiorization (abbreviated: LinSup) considers linear programming (LP) problems wherein the constraints as well as the objective function are linear. It allows to steer the iterates of a feasibility-seeking iterative process toward feasible points that have lower (not necessarily minimal) values of the objective function than points that would have been reached by the same feasiblity-seeking iterative process without superiorization. Using a feasibility-seeking iterative process that converges even if the linear feasible set is empty, LinSup generates an iterative sequence that converges to a point that minimizes a proximity function which measures the linear constraints violation. In addition, due to LinSup's repeated objective function reduction steps such a point will most probably have a reduced objective function value. We present an exploratory experimental result that illustrates the behavior of LinSup on an infeasible LP problem.
  • Linear superiorization considers linear programming problems but instead of attempting to solve them with linear optimization methods it employs perturbation resilient feasibility-seeking algorithms and steers them toward reduced (not necessarily minimal) target function values. The two questions that we set out to explore experimentally are (i) Does linear superiorization provide a feasible point whose linear target function value is lower than that obtained by running the same feasibility-seeking algorithm without superiorization under identical conditions? and (ii) How does linear superiorization fare in comparison with the Simplex method for solving linear programming problems? Based on our computational experiments presented here, the answers to these two questions are: "yes" and "very well", respectively.
  • The implicit convex feasibility problem attempts to find a point in the intersection of a finite family of convex sets, some of which are not explicitly determined but may vary. We develop simultaneous and sequential projection methods capable of handling such problems and demonstrate their applicability to image denoising in a specific medical imaging situation. By allowing the variable sets to undergo scaling, shifting and rotation, this work generalizes previous results wherein the implicit convex feasibility problem was used for cooperative wireless sensor network positioning where sets are balls and their centers were implicit.
  • In this paper we study new algorithmic structures with Douglas- Rachford (DR) operators to solve convex feasibility problems. We propose to embed the basic two-set-DR algorithmic operator into the String-Averaging Projections (SAP) and into the Block-Iterative Pro- jection (BIP) algorithmic structures, thereby creating new DR algo- rithmic schemes that include the recently proposed cyclic Douglas- Rachford algorithm and the averaged DR algorithm as special cases. We further propose and investigate a new multiple-set-DR algorithmic operator. Convergence of all these algorithmic schemes is studied by using properties of strongly quasi-nonexpansive operators and firmly nonexpansive operators.
  • This is a review paper on some of the physics, modeling, and iterative algorithms in proton computed tomography (pCT) image reconstruction. The primary challenge in pCT image reconstruction lies in the degraded spatial resolution resulting from multiple Coulomb scattering within the imaged object. Analytical models such as the most likely path (MLP) have been proposed to predict the scattered trajectory from measurements of individual proton location and direction before and after the object. Iterative algorithms provide a flexible tool with which to incorporate these models into image reconstruction. The modeling leads to a large and sparse linear system of equations that can efficiently be solved by projection methods-based iterative algorithms. Such algorithms perform projections of the iterates onto the hyperlanes that are represented by the linear equations of the system. They perform these projections in possibly various algorithmic structures, such as block-iterative projections (BIP), string-averaging projections (SAP). These algorithmic schemes allow flexibility of choosing blocks, strings, and other parameters. They also cater for parallel implementations which are apt to further save clock time in computations. Experimental results are presented which compare some of those algorithmic options.
  • We review the superiorization methodology, which can be thought of, in some cases, as lying between feasibility-seeking and constrained minimization. It is not quite trying to solve the full fledged constrained minimization problem; rather, the task is to find a feasible point which is superior (with respect to an objective function value) to one returned by a feasibility-seeking only algorithm. We distinguish between two research directions in the superiorization methodology that nourish from the same general principle: Weak superiorization and strong superiorization and clarify their nature.
  • Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are particularly suitable for parallel computing, and they demonstrate nice convergence properties and/or good initial behavior patterns. This class of algorithms has witnessed great progress in recent years and its member algorithms have been applied with success to many scientific, technological, and mathematical problems. This annotated bibliography includes books and review papers on, or related to, projection methods that we know about, use, and like. If you know of books or review papers that should be added to this list please contact us.
  • We consider the superiorization methodology, which can be thought of as lying between feasibility-seeking and constrained minimization. It is not quite trying to solve the full fledged constrained minimization problem; rather, the task is to find a feasible point which is superior (with respect to the objective function value) to one returned by a feasibility-seeking only algorithm. Our main result reveals new information about the mathematical behavior of the superiorization methodology. We deal with a constrained minimization problem with a feasible region, which is the intersection of finitely many closed convex constraint sets, and use the dynamic string-averaging projection method, with variable strings and variable weights, as a feasibility-seeking algorithm. We show that any sequence, generated by the superiorized version of a dynamic string-averaging projection algorithm, not only converges to a feasible point but, additionally, either its limit point solves the constrained minimization problem or the sequence is strictly Fej\'er monotone with respect to a subset of the solution set of the original problem.
  • The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection of the given subsets of the feasibility problem, despite certain perturbations which are allowed in each iterative step. Unlike previous works on solving the convex feasibility problem, the involved functions, which induce the feasibility problem's subsets, need not be convex. Instead, we allow them to belong to a wider and richer class of functions satisfying a weaker condition, that we call "zero-convexity". This class, which is introduced and discussed here, holds a promise to solve optimization problems in various areas, especially in non-smooth and non-convex optimization. The relevance of this study to approximate minimization and to the recent superiorization methodology for constrained optimization is explained.
  • Proton computed tomography (pCT) is a novel imaging modality developed for patients receiving proton radiation therapy. The purpose of this work was to investigate hull-detection algorithms used for preconditioning of the large and sparse linear system of equations that needs to be solved for pCT image reconstruction. The hull-detection algorithms investigated here included silhouette/space carving (SC), modified silhouette/space carving (MSC), and space modeling (SM). Each was compared to the cone-beam version of filtered backprojection (FBP) used for hull-detection. Data for testing these algorithms included simulated data sets of a digital head phantom and an experimental data set of a pediatric head phantom obtained with a pCT scanner prototype at Loma Linda University Medical Center. SC was the fastest algorithm, exceeding the speed of FBP by more than 100 times. FBP was most sensitive to the presence of noise. Ongoing work will focus on optimizing threshold parameters in order to define a fast and efficient method for hull-detection in pCT image reconstruction.
  • We propose a distributed positioning algorithm to estimate the unknown positions of a number of target nodes, given distance measurements between target nodes and between target nodes and a number of reference nodes at known positions. Based on a geometric interpretation, we formulate the positioning problem as an implicit convex feasibility problem in which some of the sets depend on the unknown target positions, and apply a parallel projection onto convex sets approach to estimate the unknown target node positions. The proposed technique is suitable for parallel implementation in which every target node in parallel can update its position and share the estimate of its location with other targets. We mathematically prove convergence of the proposed algorithm. Simulation results reveal enhanced performance for the proposed approach compared to available techniques based on projections, especially for sparse networks.
  • The projected subgradient method for constrained minimization repeatedly interlaces subgradient steps for the objective function with projections onto the feasible region, which is the intersection of closed and convex constraints sets, to regain feasibility. The latter poses a computational difficulty and, therefore, the projected subgradient method is applicable only when the feasible region is "simple to project onto". In contrast to this, in the superiorization methodology a feasibility-seeking algorithm leads the overall process and objective function steps are interlaced into it. This makes a difference because the feasibility-seeking algorithm employs projections onto the individual constraints sets and not onto the entire feasible region. We present the two approaches side-by-side and demonstrate their performance on a problem of computerized tomography image reconstruction, posed as a constrained minimization problem aiming at finding a constraint-compatible solution that has a reduced value of the total variation of the reconstructed image.
  • In constraining iterative processes, the algorithmic operator of the iterative process is pre-multiplied by a constraining operator at each iterative step. This enables the constrained algorithm, besides solving the original problem, also to find a solution that incorporates some prior knowledge about the solution. This approach has been useful in image restoration and other image processing situations when a single constraining operator was used. In the field of image reconstruction from projections a priori information about the original image, such as smoothness or that it belongs to a certain closed convex set, may be used to improve the reconstruction quality. We study here constraining of iterative processes by a family of operators rather than by a single operator.
  • We consider the convex feasibility problem (CFP) in Hilbert space and concentrate on the study of string-averaging projection (SAP) methods for the CFP, analyzing their convergence and their perturbation resilience. In the past, SAP methods were formulated with a single predetermined set of strings and a single predetermined set of weights. Here we extend the scope of the family of SAP methods to allow iteration-index-dependent variable strings and weights and term such methods dynamic string-averaging projection (DSAP) methods. The bounded perturbation resilience of DSAP methods is relevant and important for their possible use in the framework of the recently developed superiorization heuristic methodology for constrained minimization problems.
  • We consider sequential iterative processes for the common fixed point problem of families of cutter operators on a Hilbert space. These are operators that have the property that, for any point x\inH, the hyperplane through Tx whose normal is x-Tx always "cuts" the space into two half-spaces one of which contains the point x while the other contains the (assumed nonempty) fixed point set of T. We define and study generalized relaxations and extrapolation of cutter operators and construct extrapolated cyclic cutter operators. In this framework we investigate the Dos Santos local acceleration method in a unified manner and adopt it to a composition of cutters. For these we conduct convergence analysis of successive iteration algorithms.
  • We introduce and study the Split Common Null Point Problem (SCNPP) for set-valued maximal monotone mappings in Hilbert spaces. This problem generalizes our Split Variational Inequality Problem (SVIP) [Y. Censor, A. Gibali and S. Reich, Algorithms for the split variational inequality problem, Numerical Algorithms 59 (2012), 301--323]. The SCNPP with only two set-valued mappings entails finding a zero of a maximal monotone mapping in one space, the image of which under a given bounded linear transformation is a zero of another maximal monotone mapping. We present four iterative algorithms that solve such problems in Hilbert spaces, and establish weak convergence for one and strong convergence for the other three.
  • Modifying von Neumann's alternating projections algorithm, we obtain an alternating method for solving the recently introduced Common Solutions to Variational Inequalities Problem (CSVIP). For simplicity, we mainly confine our attention to the two-set CSVIP, which entails finding common solutions to two unrelated variational inequalities in Hilbert space.
  • We propose a prototypical Split Inverse Problem (SIP) and a new variational problem, called the Split Variational Inequality Problem (SVIP), which is a SIP. It entails finding a solution of one inverse problem (e.g., a Variational Inequality Problem (VIP)), the image of which under a given bounded linear transformation is a solution of another inverse problem such as a VIP. We construct iterative algorithms that solve such problems, under reasonable conditions, in Hilbert space and then discuss special cases, some of which are new even in Euclidean space.