Multi-agent reinforcement learning (MARL) is a crucial part of multi-agent system research, demonstrating remarkable effectiveness in complex collaborative tasks. However, in scenarios requiring long-term decision-making, multi-agent systems often underperform due to the difficulty in estimating long-term returns and accurately modeling environmental uncertainties. To this end, this study proposes a multi-agent memory-reinforcement learning model based on quantile regression. The model not only selectively utilizes historical decision-making experience to assist long-term decision-making but also employs quantile functions to model the return distribution, thereby effectively capturing return uncertainties. The model comprises three components, including a memory indexing module, an implicit quantile decision network, and a value distribution decomposition module. Specifically, the memory indexing module generates intrinsic reward by adopting historical decision-making experience to enhance the agents’ full utilization of existing experience. The implicit quantile decision network models reward distribution via quantile regression, providing powerful support for long-term decision-making. The value distribution decomposition module decomposes overall return distributions into the distribution of an individual agent to support single-agent strategy learning. Extensive experiments conducted in StarCraft II environments demonstrate that the proposed method enhances the performance of agents in long-term decision-making tasks, with fast convergence rates.