In　this　paper,　we　propose　a　one-step　scheme　for　implementing　the　nonadiabatic　holonomic　SWAP　gate　with　Rydberg　atoms.　By　applying　invariant-based　reverse　engineering　to　design　the　effective　Hamiltonian　of　the　system,　a　suitable　evolution　path　for　implementing　nonadiabatic　holonomic　quantum　computation　is　found.　In　addition,　the　systematic-error-sensitivity　nullified　optimal　control　method　is　considered　in　the　parameter　selections,　so　that　the　scheme　is　insensitive　to　the　systematic　error　of　pulses.　We　also　estimate　the　effects　of　random　noise,　the　random　initial　phase　of　the　pulses,　the　Doppler　shift,　and　decoherence　on　the　scheme.　The　numerical　results　show　that　the　scheme　exhibits　fairly　good　performance　against　these　negative　factors.　Finally,　we　generalize　the　scheme　to　realize　the　non-Clifford　SWAP　gates.　Therefore,　this　scheme　can　provide　a　feasible　framework　for　implementing　high-fidelity　and　robust　SWAP　gates　and　non-Clifford　SWAP　gates　with　Rydberg　atoms.