Skip to main content

Research Repository

Advanced Search

TMA-Net: A Transformer-based Multi-scale Attention Network for Surgical Instrument Segmentation

Yang, Lei; Wang, Hongyong; Gu, Yuge; Bian, Guibin; Liu, Yanhong; Yu, Hongnian

Authors

Lei Yang

Hongyong Wang

Yuge Gu

Guibin Bian

Yanhong Liu



Abstract

The ability to accurately and automatically segment surgical instruments is one of the important prerequisites for reasonable and stable operation of surgical robots. The utilization of deep learning in medical image segmentation has gained widespread recognition in recent years, leading to the proposition of multiple network models designed for the segmentation of diverse medical images, among which the most effective one is U-Net and its variants. Nevertheless, these existing networks also have various drawbacks, such as limited contextual representation capability, insufficient local feature processing, etc. In order to solve the above problems so that more accurate surgical instrument segmentation performance can be obtained, a transformer-based multi-scale attention network is proposed, referred to as TMA-Net, for surgical instrument segmentation from endoscopic images to serve robot-assisted surgery. To enable more accurate extraction of image features, a dual-branch encoder structure is proposed to obtain stronger contexts. Further, to address the problem that the simple skip connection is insufficient for local feature processing, an attention feature fusion (AFF) module and an additive attention and concatenation (AAC) module are proposed for effective feature learning to filter out the irrelevant information in the low-level features. Furthermore, a multi-scale context fusion (MCF) block is introduced to enhance the local feature maps and capture multi-scale contextual information. The efficacy of proposed TMA-Net is demonstrated through experimentation on publicly available surgical instrument segmentation datasets, including Endovis2017 and UW-Sinus-Surgery-C/L. The results show that proposed TMA-Net outperforms existing methods in terms of surgical instrument segmentation accuracy.

Journal Article Type Article
Online Publication Date Apr 25, 2023
Publication Date 2023-05
Deposit Date May 1, 2023
Journal IEEE Transactions on Medical Robotics and Bionics
Print ISSN 2576-3202
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 5
Issue 2
Pages 323-334
DOI https://doi.org/10.1109/tmrb.2023.3269856
Keywords Deep Network, Surgical Instrument Segmentation, Transformer, Attention Mechanism