This　article　addresses　the　cooperative　control　of　behavior-based　nonholonomic　mobile　vehicles　(NMVs).　Null-space-based　behavioral　control　(NSBC)　has　limitations　in　the　particle　model　and　mission　centralization.　A　novel　underactuated　reinforcement　learning　behavioral　control　(URLBC)　approach　is　proposed　in　this　article　to　address　these　limitations.　An　underactuated　mission　paradigm　is　prepared　to　replace　the　particle　model　and　prevent　nonholonomic　constraint　violations　of　the　behavioral　command.　Subsequently,　a　distributed　reinforcement　learning　mission　supervisor　is　developed　to　determine　behavioral　priorities　without　any　centralized　unit,　intelligently　generating　behavioral　commands　while　achieving　expected　priority　switching.　Furthermore,　an　identifier–actor–critic　behavioral　controller　is　formulated　to　track　behavioral　commands　without　exact　models,　thus　effectively　reducing　control　costs　and　model　dependencies.　Simulation　shows　that　URLBC　has　fewer　unsatisfactory　priority　switching　instances　and　lower　cumulative　control　costs　than　existing　NSBC　methods.　URLBC　is　implemented　on　JetAuto　robots　to　validate　its　practicality.　©　1982-2012　IEEE.