In recent years, edge computing has emerged as a promising paradigm for providing flexible and reliable services for Internet of things (IoT) applications. User requests can be offloaded and processed in real time at the edge of a network. However, considering the limited storage and computing resources of IoT devices, certain services requested by users may not be configured on current edge servers. In this setting, user requests should be offloaded to adjacent edge servers or requested edge servers should be configured by migrating certain services from the former, further reducing the service access delay of user requests and the energy consumption of IoT devices in such networks. To address this issue, in this study, we model this dynamic task offloading and service migration optimization problem as the multiple dimensional Markov decision process and propose a deep q-learning network (DQN) algorithm to achieve fast decision-making, an approximate optimal task offloading, and service migration solution. Experimental results show that our algorithm performs better than existing baseline approaches in terms of reducing the service access delay of user requests and the energy consumption of IoT devices in edge networks.