A new architecture called EG-TransUNet has been proposed to improve the performance of biomedical image segmentation in order to meet the precision requirements of medical imaging segmentation tasks. This architecture uses an attention-based Transformer during the encoder and decoder stages to improve feature discrimination at the level of spatial detail and semantic location. The results of the experiments showed that the proposed method achieved better performance than the baseline models.