The　reconstruction　of　buildings　using　inhomogeneous　and　unstructured　point　clouds　is　a　challenging　task　for　photogrammetry　and　computer　vision　research　communities.　A　new　approach　for　3D　building　surface　modeling,　based　on　closed　constraints,　is　proposed.　First,　a　region　growth　algorithm　is　applied　to　fit　the　input　point　clouds　by　a　set　of　candidate　planes.　Then,　additional　candidate　planes　are　generated　from　the　initial　planes　according　to　a　rigid　transformation　followed　by　expanding　the　original　primitive　set　to　the　candidate　model　set　through　generation　rules.　Furthermore,　an　energy　function　is　employed　to　combine　the　data　fitting　errors　with　the　structural　constraints　at　the　model　selection　stage.　Finally,　the　3D　building　surface　model　is　generated　from　the　candidate　set　through　energy　minimization.　More　precisely　speaking,　we　adopt　the　surface　optimization　scheme　that　enforces　the　3D　polygonal　surfaces　of　the　building　to　be　consistent　with　a　priori　geometric　structures.　Our　approach　was　assessed　using　multi-source　datasets　with　different　densities,　noise　levels　covering　diverse　and　complex　structures.　The　experimental　results　demonstrated　that　the　proposed　approach　achieves　better　accuracy　and　robustness　than　those　of　several　state-of-the-art　methods.