When the prisoner is remanded, the execution of the judgment shall not be suspended by a petition for appeal or by a writ of error, or for the purpose of applying for such writ. When he is ordered to be discharged, and the execution of the judgment is suspended for the purpose of petitioning for appeal to the Court of Appeals or applying for a writ of error from the Supreme Court, the court making such suspending order may admit the prisoner to bail until the expiration of the time allowed for filing a petition for appeal or applying for the writ of error, or, in case the petition for appeal is filed or the writ of error is allowed, until the decision of the Court of Appeals or the Supreme Court thereon is duly certified.