Nowadays,　the　impressive　performance　of　deep　neural　networks　(DNNs)　greatly　advances　the　development　of　Internet　of　Things　(IoT)　in　diverse　scenarios.　However,　the　exceptional　vulnerability　of　DNNs　to　adversarial　attack　leads　IoT　devices　to　be　exposed　to　potential　security　issues.　Up　to　now,　since　adversarial　training　empirically　remains　robust　against　gradient-based　adversarial　attacks,　it　is　believed　to　be　the　most　effective　defense　method.　In　this　article,　we　find　that　adversarial　examples　generated　by　gradient-based　adversarial　attacks　tend　to　be　less　imperceptible　induced　by　the　gradient-based　optimization　methods　(adopted　in　the　attacks)　being　difficult　on　searching　the　most　effective　adversarial　examples　(i.e.,　the　global　extreme　points),　which　may　lead　to　an　inaccurate　estimation　for　the　effectiveness　of　the　adversarial　training.　To　overcome　the　inherent　defect　of　gradient-based　adversarial　attacks,　we　propose　a　novel　adversarial　attack　named　nongradient　attack　(NGA),　of　which　search　strategy　is　effective　but　no　longer　depends　on　gradients　to　enhance　the　threat　of　adversarial　examples.　In　detail,　NGA　first　initializes　the　adversarial　examples　outside,　rather　than　inside,　of　decision　boundary　to　make　them　misclassified　by　the　model　and　then,　under　without　violation　of　misclassified　condition,　adjusts　the　adversarial　examples　toward　the　crafted　direction　to　close　the　original　examples.　Extensive　experiments　show　that　NGA　significantly　outperforms　the　state-of-the-art　adversarial　attacks　on　attack　success　rate　(ASR)　by　2%-7%.　Moreover,　we　propose　a　new　evaluation　metric,　i.e.,　composite　criterion　(CC)　based　on　both　ASR　and　accuracy,　to　better　measure　the　effectiveness　of　adversarial　training.　In　the　experiments,　CC　has　shown　to　be　a　more　comprehensive　yet　appropriate　evaluation　metric.　©　2014　IEEE.