It　is　critical　to　use　scientific　methods　to　track　the　performance　degradation　of　in-service　buildings　over　time　and　avoid　accidents.　In　recent　years,　both　supervised　and　unsupervised　learning　methods　have　yielded　positive　results　in　structural　health　monitoring　(SHM).　Supervised　learning　approaches　require　data　from　the　entire　structure　and　various　damage　scenarios　for　training.　However,　it　is　impractical　to　obtain　adequate　training　data　from　various　damage　situations　in　service　facilities.　In　addition,　most　known　unsupervised　approaches　for　training　only　take　response　data　from　the　entire　structure.　In　these　situations,　contaminated　data　containing　both　undamaged　and　damaged　samples,　typical　in　real-world　applications,　prevent　the　models　from　fitting　undamaged　data,　resulting　in　performance　loss.　This　work　provides　an　unsupervised　technique　for　detecting　structural　damage　for　the　reasons　stated　above.　This　approach　trains　on　contaminated　data,　with　the　anomaly　score　of　the　data　serving　as　the　model＇s　output.　First,　we　devised　a　score-guided　regularization　approach　for　damage　detection　to　expand　the　score　difference　between　undamaged　and　damaged　data.　Then,　multi-task　learning　is　incorporated　to　make　parameter　adjustment　easier.　The　experimental　phase　II　of　the　SHM　benchmark　data　and　data　from　the　Qatar　University　grandstand　simulator　are　used　to　validate　this　strategy.　The　suggested　algorithm　has　the　most　excellent　mean　AUC　of　0.708　and　0.998　on　the　two　datasets　compared　to　the　classical　algorithm.