The first observation of the rare decay$B^0_s \to \phi\pi^+\pi^-$ and evidence for $B^0 \to \phi\pi^+\pi^-$ are reported, using $pp$ collision data recorded by the LHCb detector at centre-of-mass energies $\sqrt{s} = 7$ and 8~TeV, corresponding to an integrated luminosity of $3{\mbox{\,fb}^{-1}}$. The branching fractions in the $\pi^+\pi^-$ invariant mass range $400<m(\pi^+\pi^-)<1600{\mathrm{\,Me\kern -0.1em V\!/}c^2}$ are $[3.48\pm 0.23\pm 0.17\pm 0.35]\times 10^{-6}$ and $[1.82\pm 0.25\pm 0.41\pm 0.14]\times 10^{-7}$ for $B^0_s \to \phi\pi^+\pi^-$ and $B^0 \to \phi\pi^+\pi^-$ respectively, where the uncertainties are statistical, systematic and from the normalisation mode $B^0_s \to \phi\phi $. A combined analysis of the $\pi^+\pi^-$ mass spectrum and the decay angles of the final-state particles identifies the exclusive decays $B^0_s \to \phi f_0(980) $, $B_s^0 \to \phi f_2(1270) $, and $B^0_s \to \phi\rho^0$ with branching fractions of $[1.12\pm 0.16^{+0.09}_{-0.08}\pm 0.11]\times 10^{-6}$, $[0.61\pm 0.13^{+0.12}_{-0.05}\pm 0.06]\times 10^{-6}$ and $[2.7\pm 0.7\pm 0.2\pm 0.2]\times 10^{-7}$, respectively.