Self-similarity-based　deep　Single　Image　SuperResolution　(SISR)　methods　have　gained　popularity　in　recent　years,　especially　with　the　integration　of　Non-Local　Attention　(NLA)　in　deep　SISR.　However,　NLA　suffers　from　the　drawback　of　mixing　relevant　and　irrelevant　features,　as　it　computes　the　response　of　each　query　by　aggregating　information　from　all　non-local　features.　In　this　work,　we　propose　a　novel　approach　to　exploit　self-similarity　more　effectively　using　FlyHash,　which　is　inspired　by　the　fruit　fly　olfactory　circuit　and　has　exhibited　outstanding　performance　in　approximate　similarity　search.　By　limiting　the　association　area　of　non-local　features　with　FlyHash,　we　develop　the　Decoupled　Non-Local　Attention　(DNLA)　method,　which　tackles　the　difficulties　of　modeling　a　large　amount　of　irrelevant　non-local　features　while　considerably　lowering　the　computational　complexity　from　quadratic　to　nearly　linear.　We　verify　the　effectiveness　of　our　DNLA　approach　with　comprehensive　ablation　studies　to　show　its　capability　of　capturing　nonlocal　information　for　deep　SISR.　Furthermore,　we　build　a　deep　Decoupled　Non-Local　Attention　Network　(DNLAN)　with　DNLA,　which　attains　excellent　results　in　both　objective　evaluation　and　subjective　perception　for　SISR.