Multi-objective　evolutionary　algorithms　have　become　the　most　important　method　to　deal　with　multi-objective　optimization　problems　(MOP).　To　improve　the　performance　of　particle　swarm　optimization　(PSO)　in　addressing　MOPs,　a　multi-objective　PSO　based　on　temporal-difference　learning　(TDLMOPSO)　is　proposed　in　this　paper.　The　iteration　process　of　TDLMOPSO　is　transformed　into　a　Markov　decision　process,　particles　are　treated　as　agents,　each　agent　has　a　personal　archive,　the　states　are　designed　for　the　connection　of　actions,　the　actions　of　particles　contain　all　necessary　behavior　of　them:　basic　movement,　jump　out　of　local　optimum,　and　local　search,　and　the　rewards　depend　on　the　relationship　between　particles＇　positions　and　their　personal　archives.　Besides,　the　external　archive　deletion　strategy　and　the　leader　selection　strategy　are　redesigned　based　on　the　unsupervised　learning　algorithm　to　enhance　the　diversity　of　solutions　in　the　external　archive.　The　effectiveness　of　TDLMOPSO　is　verified　by　applying　it　with　other　seven　advanced　multi-objective　algorithms　in　MOP　benchmark　test　suites.　Furthermore,　the　time　complexity　and　parameter　sensitivity　of　TDLMOPSO　are　analyzed.