In　this　paper,　we　propose　a　protocol　to　realize　non-adiabatic　holonomic　quantum　computation　(NHQC)　of　cavity　modes　via　invariant-based　reverse　engineering.　Coupling　cavity　modes　with　an　auxiliary　atom　trapped　in　a　cavity,　we　derive　effective　Hamiltonians　with　the　help　of　laser　pulses.　Based　on　the　derived　Hamiltonians,　invariant-based　reverse　engineering　is　used　to　find　proper　evolution　paths　for　NHQC.　Moreover,　the　systematic-error-sensitivity　nullified　optimal　control　method　is　considered　in　the　parameter　selections,　making　the　protocol　insensitive　to　the　influence　of　systematic　errors　of　pulses.　We　also　estimate　the　imperfections　induced　by　random　noise　and　decoherence.　Numerical　results　show　that　the　protocol　holds　robustness　against　these　imperfections.　Therefore,　the　protocol　may　provide　useful　perspectives　to　quantum　computation　with　optical　qubits　in　cavity　quantum　electrodynamics　systems.　This　article　is　part　of　the　theme　issue　＇Shortcuts　to　adiabaticity:　theoretical,　experimental　and　interdisciplinary　perspectives＇.　©　2022　The　Author(s).