Deep　reinforcement　learning　has　achieved　great　success　in　laser-based　collision　avoidance　work　because　the　laser　can　sense　accurate　depth　information　without　too　much　redundant　data,　which　can　maintain　the　robustness　of　the　algorithm　when　it　is　migrated　from　the　simulation　environment　to　the　real　world.　However,　high-cost　laser　devices　are　not　only　difficult　to　apply　on　a　large　scale　but　also　have　poor　robustness　to　irregular　objects,　e.g.,　tables,　chairs,　shelves,　etc.　In　this　paper,　we　propose　a　vision-based　collision　avoidance　framework　to　solve　the　challenging　problem.　Our　method　attempts　to　estimate　the　depth　and　incorporate　the　semantic　information　from　RGB　data　to　obtain　a　new　form　of　data,　pseudo-laser　data,　which　combines　the　advantages　of　visual　information　and　laser　information.　Compared　to　traditional　laser　data　that　only　contains　the　one-dimensional　distance　information　captured　at　a　certain　height,　our　proposed　pseudo-laser　data　encodes　the　depth　information　and　semantic　information　within　the　image,　which　makes　our　method　more　effective　for　irregular　obstacles.　Besides,　we　adaptively　add　noise　to　the　laser　data　during　the　training　stage　to　increase　the　robustness　of　our　model　in　the　real　world,　due　to　the　estimated　depth　information　is　not　accurate.　Experimental　results　show　that　our　framework　achieves　state-of-the-art　performance　in　several　unseen　virtual　and　real-world　scenarios.　©　2021　IEEE.