A　large　number　of　real-world　systems　generate　graphs　that　are　structured　data　aligned　with　nodes　and　edges.　Graphs　are　usually　dynamic　in　many　scenarios,　where　nodes　or　edges　keep　evolving　over　time.　Recently,　graph　representation　learning　(GRL)　has　received　great　success　in　network　analysis,　which　aims　to　produce　informative　and　representative　features　or　low-dimensional　embeddings　by　exploring　node　attributes　and　network　topology.　Most　state-of-the-art　models　for　dynamic　GRL　are　composed　of　a　static　representation　learning　model　and　a　recurrent　neural　network　(RNN).　The　former　generates　the　representations　of　a　graph　or　nodes　from　one　static　graph　at　a　discrete　time　step,　while　the　latter　captures　the　temporal　correlation　between　adjacent　graphs.　However,　the　two-stage　design　ignores　the　temporal　dynamics　between　contiguous　graphs　during　the　learning　processing　of　graph　representations.　To　alleviate　this　problem,　this　article　proposes　a　representation　learning　model　for　dynamic　graphs,　called　DynGNN.　Differently,　it　is　a　single-stage　model　that　embeds　an　RNN　into　a　graph　neural　network　to　produce　better　representations　in　a　compact　form.　This　takes　the　fusion　of　temporal　and　topology　correlations　into　account　from　low-level　to　high-level　feature　learning,　enabling　the　model　to　capture　more　fine-grained　evolving　patterns.　From　the　experimental　results　on　both　synthetic　and　real-world　networks,　the　proposed　DynGNN　yields　significant　improvements　in　multiple　tasks　compared　to　the　state-of-the-art　counterparts.