The　processing　and　analysis　of　remotely　sensed　imagery　(RSI)　is　a　research　hotspot　in　the　information　field,　and　building　extraction　and　change　detection　are　some　of　the　difficult　problems.　In　order　to　make　the　maximum　use　of　the　effective　characteristics　and　design　independently　the　algorithm　of　feature　extraction,　an　approach　to　building　extraction　and　change　detection　from　RSI　based　on　layered　architecture　containing　pixel　layer,　object　layer　and　configuration　layer　is　proposed.　In　the　pixel　layer,　the　input　image　is　over-segmented　and　under-segmented,　respectively,　by　a　quantity-controllable　algorithm　using　super-pixel　segmentation　to　obtain　the　segmentation　object　sets,　with　which　the　input　image　is　decomposed　into　shadow　layer,　homogeneity　layer　and　edge　layer,　where　the　buildings　are　extracted　based　on　the　spatial　relationship　between　the　feature　areas　and　segmentation　objects.　In　the　object　layer,　for　preserving　the　accurate　contour　of　the　buildings,　a　new　segmentation　method　based　on　the　traditional　graph-cut　theory　and　mathematical　morphology　is　introduced,　and　then,　the　buildings　extracted　from　each　layer　are　merged.　Finally,　in　the　configuration　layer,　the　change　information　is　detected　using　spatial　relationship　of　buildings　between　the　old　image　and　the　new　one.　The　experimental　results　reveal　that　the　building　contour　is　extracted　accurately,　and　three　types　of　change　including　the　newly　built,　the　demolished　and　the　reconstructed　buildings　can　be　detected;　in　addition,　there　is　no　strict　requirement　for　registration　accuracy.　For　the　test　images,　the　overall　performance　F1　of　the　building　extraction　is　over　85,　and　the　precision　and　recall　of　the　change　detection　are　both　higher　than　90%.　©　2019,　Springer-Verlag　GmbH　Germany,　part　of　Springer　Nature.