Toxin-antitoxin (TA) systems are widespread antiphage defense elements in bacteria that may impede successful phage therapy. Phage-encoded inhibitors of these systems have been discovered that enhance phage infection capacity. We used fragment-based design with deep-learning scoring functions to design peptide inhibitors of the toxin RelE. Our peptides extend a fragment of the native RelB antitoxin and are sufficient to inhibit RelE toxicity. Successful inhibitors share a highly conserved binding mode that mimics the native antitoxin but have diverse sequences, with alternative contacts used to form the peptide-protein interface. Designed peptides show different interaction specificities toward RelE family proteins, distinct from the wild-type RelB antitoxin, and inhibit the antiphage defense activity of multiple orthologous RelBE systems. This work showcases the use of structure-based computational design to generate peptide binders that serve as potential counter-defense elements.