
The recent popularity of deep neural networks (DNNs) has generated a lot of
research interest in performing DNNrelated computation efficiently. However,
the primary focus is usually very narrow and limited to (i) inference  i.e.
how to efficiently execute already trained models and (ii) image classification
networks as the primary benchmark for evaluation.
Our primary goal in this work is to break this myopic view by (i) proposing a
new benchmark for DNN training, called TBD (TBD is short for Training Benchmark
for DNNs), that uses a representative set of DNN models that cover a wide range
of machine learning applications: image classification, machine translation,
speech recognition, object detection, adversarial networks, reinforcement
learning, and (ii) by performing an extensive performance analysis of training
these different applications on three major deep learning frameworks
(TensorFlow, MXNet, CNTK) across different hardware configurations (singleGPU,
multiGPU, and multimachine). TBD currently covers six major application
domains and eight different stateoftheart models.
We present a new toolchain for performance analysis for these models that
combines the targeted usage of existing performance analysis tools, careful
selection of new and existing metrics and methodologies to analyze the results,
and utilization of domain specific characteristics of DNN training. We also
build a new set of tools for memory profiling in all three major frameworks;
much needed tools that can finally shed some light on precisely how much memory
is consumed by different data structures (weights, activations, gradients,
workspace) in DNN training. By using our tools and methodologies, we make
several important observations and recommendations on where the future research
and optimization of DNN training should be focused.

We study the asymmetry in the twopoint crosscorrelation function of two
populations of galaxies focusing in particular on the relativistic effects that
include the gravitational redshift. We derive the crosscorrelation function on
small and large scales using two different approaches: General Relativistic and
Newtonian perturbation theory. Following recent work by Bonvin et al.,
Gaztanaga et al. and Croft, we calculate the dipole and the shell estimator
with the two procedures and we compare our results. We find that while General
Relativistic Perturbation Theory (GRPT) is able to make predictions of
relativistic effects on very large, obviously linear scales (r > 50 Mpc/h), the
presence of nonlinearities physically occurring on much smaller scales (down
to those describing galactic potential wells) can strongly affect the asymmetry
estimators. These can lead to cancellations of the relativistic terms, and sign
changes in the estimators on scales up to r ~ 50 Mpc/h. On the other hand, with
an appropriate nonlinear gravitational potential, the results obtained using
Newtonian theory can successfully describe the asymmetry on smaller, nonlinear
scales (r < 20 Mpc/h) where gravitational redshift is the dominant term. On
larger scales the asymmetry is much smaller in magnitude, and measurement is
not within reach of current observations. This is in agreement with the
observational results obtained by Gaztnaga et al. and the first detection of
relativistic effects (on (r < 20 Mpc/h) scales) by Alam et al.

General relativistic effects have long been predicted to subtly influence the
observed largescale structure of the universe. The current generation of
galaxy redshift surveys have reached a size where detection of such effects is
becoming feasible. In this paper, we report the first detection of the redshift
asymmetry from the crosscorrelation function of two galaxy populations which
is consistent with relativistic effects. The dataset is taken from the Sloan
Digital Sky Survey DR12 CMASS galaxy sample, and we detect the asymmetry at the
$2.7\sigma$ level by applying a shellaveraged estimator to the
crosscorrelation function. Our measurement dominates at scales around $10$
h$^{1}$Mpc, larger than those over which the gravitational redshift profile
has been recently measured in galaxy clusters, but smaller than scales for
which linear perturbation theory is likely to be accurate. The detection
significance varies by 0.5$\sigma$ with the details of our measurement and
tests for systematic effects. We have also devised two null tests to check for
various survey systematics and show that both results are consistent with the
null hypothesis. We measure the dipole moment of the crosscorrelation
function, and from this the asymmetry is also detected, at the $2.8 \sigma$
level. The amplitude and scaledependence of the clustering asymmetries are
approximately consistent with the expectations of General Relativity and a
biased galaxy population, within large uncertainties. We explore theoretical
predictions using numerical simulations in a companion paper.

In a galaxy redshift survey the objects to be targeted for spectra are
selected from a photometrically observed sample. The observed magnitudes and
colours of galaxies in this parent sample will be affected by their peculiar
velocities, through relativistic Doppler and relativistic beaming effects. In
this paper we compute the resulting expected changes in galaxy photometry. The
magnitudes of the relativistic effects are a function of redshift, stellar
mass, galaxy velocity and velocity direction. We focus on the CMASS sample from
the Sloan Digital Sky Survey (SDSS), Baryon Oscillation Spectroscopic Survey
(BOSS), which is selected on the basis of colour and magnitude. We find that
0.10\% of the sample ($\sim 585$ galaxies) has been scattered into the targeted
region of colourmagnitude space by relativistic effects, and conversely 0.09\%
of the sample ($\sim 532$ galaxies) has been scattered out. Observational
consequences of these effects include an asymmetry in clustering statistics,
which we explore in a companion paper. Here we compute a set of weights which
can be used to remove the effect of modulations introduced into the density
field inferred from a galaxy sample. We conclude by investigating the possible
effects of these relativistic modulation on large scale clustering of the
galaxy sample.

Large redshift surveys of galaxies and clusters are providing the first
opportunities to search for distortions in the observed pattern of largescale
structure due to such effects as gravitational redshift. We focus on nonlinear
scales and apply a quasiNewtonian approach using Nbody simulations to predict
the small asymmetries in the crosscorrelation function of two galaxy different
populations. Following recent work by Bonvin et al., Zhao and Peacock and
Kaiser on galaxy clusters, we include effects which enter at the same order as
gravitational redshift: the transverse Doppler effect, lightcone effects,
relativistic beaming, luminosity distance perturbation and wideangle effects.
We find that all these effects cause asymmetries in the crosscorrelation
functions. Quantifying these asymmetries, we find that the total effect is
dominated by the gravitational redshift and luminosity distance perturbation at
small and large scales, respectively. By adding additional subresolution
modelling of galaxy structure to the largescale structure information, we find
that the signal is significantly increased, indicating that structure on the
smallest scales is important and should be included. We report on comparison of
our simulation results with measurements from the SDSS/BOSS galaxy redshift
survey in a companion paper.