RNA polymerase can cleave a phosphodiester bond at the 3' end of a nascent RNA in the presence of pyrophosphate producing NTP. Pyrophosphorolysis has been characterized during elongation steps of transcription where its rate is significantly slower than the forward rate of NMP addition. In contrast, we report here that pyrophosphorolysis can occur in a millisecond time scale during the transition of Escherichia coli RNA polymerase from initiation to elongation at the psbA2 promoter. This rapid pyrophosphorolysis occurs during productive RNA synthesis as opposed to abortive RNA synthesis. Dissociation of σ70 or RNA extension beyond nine nucleotides dramatically reduces the rate of pyrophosphorolysis. We argue that the rapid pyrophosphorolysis allows iterative cycles of cleavage and re-synthesis of the 3' phosphodiester bond by the productive complexes in the early stage of transcription. This iterative process may provide an opportunity for the σ70 to dissociate from the RNA exit channel of the enzyme, enabling RNA to extend through the channel.