The　pavement　defect　detection　is　challenging　due　to　the　diverse　defects　and　their　unpredictable　formations.　Current　methods　often　struggle　to　perform　well　in　situations　with　complex　pavement　backgrounds　and　weak　textures,　a　problem　that　arises　from　the　interference　of　various　pavement　information.　We　proposed　an　unsupervised　pavement　defect　detection　method　with　a　multiscale　gradient　selection-based　coupled-hypersphere　adaptation　(MGSCA)　to　alleviate　these　challenges.　Specifically,　the　proposed　method　first　constructs　a　lightweight　and　representative　feature-capturing　memory　bank　to　effectively　capture　features　of　complex　pavement　backgrounds　and　reduce　the　inference　time.　Utilizing　these　representative　features　as　hypersphere　centers,　we　propose　a　hypersphere-coupled　adaptive　module　that　trains　an　axis-based　self-attention　descriptor　to　reduce　the　data　bias　of　the　pretrained　network　and　improve　the　defect　detection　from　the　nondefective　pavement　backgrounds.　On　the　Crack500　dataset,　the　proposed　method　achieved　0.930　area　under　the　ROC　curve　(AUROC)　in　defect　detection.　For　defect　localization,　the　proposed　method　achieved　0.925　AUROC,　0.791　per-region-overlap　(PRO),　and　0.744　DICE.　Compared　with　several　state-of-the-art　methods,　the　results　on　three　datasets　demonstrate　the　effectiveness　of　our　proposed　method　in　different　scenarios.