Autonomous　vehicle　steering　control　is　sensitive　to　the　vehicle　driving　speed　and　traditional　model-based　approaches　are　limited　by　the　accuracy　of　the　control　model　in　various　driving　speed　scenarios.　To　address　these　challenges,　this　study　proposes　a　model-free　control　strategy　based　on　deep　reinforcement　learning　(DRL).　In　this　strategy,　the　improved　double　deep　Q-learning　network　(DDQN)　with　varied　agents　is　employed　for　steering　control　to　minimize　the　tracking　errors　across　varying　speeds.　According　to　the　kinematic　characteristics　of　the　vehicle,　a　dynamic　action　space　is　applied　to　enhance　the　tracking　capability　at　high　speeds.　Furthermore,　to　ensure　the　output　of　the　agent　is　more　stable,　a　velocity　adaptive　reward　function　is　designed　by　incorporating　an　action　penalty　factor.　The　performance　of　the　proposed　strategy　is　evaluated　through　simulation　and　experimental　comparisons　with　other　existing　algorithms　at　a　double-lane　change　maneuver.　The　results　demonstrate　that　the　DDQN-based　strategy　can　effectively　adapt　to　various　vehicle　speeds　and　perform　the　tracking　task　more　accurately　and　stably.　Finally,　the　feasibility　of　this　strategy　is　verified　using　an　actual　prototype　vehicle.　©　2024　Elsevier　Ltd