Reinforcement learning (RL) can free automated vehicles (AVs) from the car-following constraints and provide more possible explorations for mixed behavior. This study uses deep RL as AVs’ longitudinal control and designs a multi-level objectives framework for AVs’ trajectory decision-making based on multi-agent DRL. The saturated signalized intersection is taken as the research object to seek the upper limit of traffic efficiency and realize the specific target control. The simulation results demonstrate the convergence of the proposed framework in complex scenarios. When prioritizing throughputs as the primary objective and emissions as the secondary objective, both indicators exhibit a linear growth pattern with increasing market penetration rate (MPR). Compared with MPR is 0%, the throughputs can be increased by 69.2% when MPR is 100%. Compared with linear adaptive cruise control (LACC) under the same MPR, the emissions can also be reduced by up to 78.8%. Under the control of the fixed throughputs, compared with LACC, the emission benefits grow nearly linearly as MPR increases, it can reach 79.4% at 80% MPR. This study employs experimental results to analyze the behavioral changes of mixed flow and the mechanism of mixed autonomy to improve traffic efficiency. The proposed method is flexible and serves as a valuable tool for exploring and studying the behavior of mixed flow behavior and the patterns of mixed autonomy.