Image　segmentation　tasks　aim　to　separate　the　image　into　masks　that　represent　different　objects　or　regions,　where　deep-learning-based　methods　have　become　mainstream.　In　the　common　practice,　researchers　utilize　large-scale　datasets　including　images　along　with　their　annotations　to　train　their　models,　and　evaluate　the　predictions　with　evaluation　metrics.　However,　to　our　knowledge,　no　metrics　have　been　proposed　to　assess　the　quality　of　the　segmentation　annotations,　which　will　bring　benefits　to　both　the　labeling　and　experimental　process.　In　this　paper,　we　fill　this　research　gap　and　propose　the　first　no-reference　segmentation　annotation　quality　assessment　named　SAQ.　Based　on　our　observation,　we　utilize　the　normal　gradients　of　pixels　on　the　annotation　contours　to　represent　the　degree　of　fitting　the　real　contours,　which　reflect　the　annotation　accuracy.　To　alleviate　the　image　differences,　we　adopt　the　gradient　ranking　score　rather　than　directly　using　the　gradient　value.　The　multi-scale　strategy　is　introduced　to　accommodate　annotations　of　objects　with　different　structures.　Extensive　experiments　on　datasets　for　various　segmentation　tasks　have　demonstrated　the　rationality　of　our　proposed　SAQ,　and　the　assessment　results　of　their　annotation　quality　can　serve　as　significant　references　for　researchers.　©　2024　IEEE.