The　integration　of　ultracapacitors　(UCs)　into　hybrid　energy　storage　systems　is　a　solution　to　mitigate　battery　degradation.　Traditional　strategies　focus　on　fuel　cell　and　battery　power　regulation　while　treating　UC　management　as　a　passive　element,　resulting　in　suboptimal　UC　utilization.　To　optimize　the　energy　utilization　of　UCs,　this　article　proposes　an　active　state　control　strategy　within　the　hybrid　system.　Initially,　leveraging　the　battery　severity　factor,　the　optimal　power　split　strategy　for　HESS　is　proposed　for　a　reference　state-of-charge　(SOC)　of　UC.　Subsequently,　a　driving　pattern　severity　factor　is　designed,　and　an　online　self-learning　Markov　predictor　is　employed　to　quantify　the　operational　state　of　vehicle.　To　provide　optimal　reference　SOC　guidance　to　HESS　in　real　time,　a　reinforcement　learning　algorithm　featuring　an　experience　replay　mechanism　is　developed.　Utilizing　pretrained　agents　that　integrate　vehicle　driving　state　abstraction　parameters,　the　system　generates　the　reference　SOC　of　UC,　enabling　the　optimal　battery-UC　power　split　in　real　time.　Both　numerical　and　semiphysical　validations　confirm　the　efficacy　of　the　proposed　strategy　in　enhancing　the　power　output　ratio　of　UC,　optimizing　energy　storage　space　utilization,　and　reducing　the　battery　severity　factor,　consequently　improving　overall　battery　lifespan.