AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (1.1 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

TIFAflow: Enhancing Traffic Archiving System with Flow Granularity for Forensic Analysis in Network Security

Research Institute of Information Technology and Tsinghua National Laboratory for Information Science and Technology (TNList), Tsinghua University, Beijing 100084, China
Department of Automation, Research Institute of Information Technology and Tsinghua National Laboratory for Information Science and Technology (TNList), Tsinghua University, Beijing 100084, China
Department of Computer Science of Purdue University, West Lafayette, IN 47907, USA
Department of Electronic Engineering and Tsinghua National Laboratory for Information Science and Technology (TNList), Tsinghua University, Beijing 100084, China
Department of Computer Science and Technology, Research Institute of Information Technology and Tsinghua National Laboratory for Information Science and Technology (TNList), Tsinghua University, Beijing 100084, China
Show Author Information

Abstract

The archiving of Internet traffic is an essential function for retrospective network event analysis and forensic computer communication. The state-of-the-art approach for network monitoring and analysis involves storage and analysis of network flow statistic. However, this approach loses much valuable information within the Internet traffic. With the advancement of commodity hardware, in particular the volume of storage devices and the speed of interconnect technologies used in network adapter cards and multi-core processors, it is now possible to capture 10 Gbps and beyond real-time network traffic using a commodity computer, such as n2disk. Also with the advancement of distributed file system (such as Hadoop, ZFS, etc.) and open cloud computing platform (such as OpenStack, CloudStack, and Eucalyptus, etc.), it is practical to store such large volume of traffic data and fully in-depth analyse the inside communication within an acceptable latency. In this paper, based on well-known TimeMachine, we present TIFAflow, the design and implementation of a novel system for archiving and querying network flows. Firstly, we enhance the traffic archiving system named TImemachine+FAstbit (TIFA) with flow granularity, i.e., supply the system with flow table and flow module. Secondly, based on real network traces, we conduct performance comparison experiments of TIFAflow with other implementations such as common database solution, TimeMachine and TIFA system. Finally, based on comparison results, we demonstrate that TIFAflow has a higher performance improvement in storing and querying performance than TimeMachine and TIFA, both in time and space metrics.

Keywords

network securitytraffic archivalforensic analysisphishing attackbitmap databasehadoop distributed file systemcloud computingNoSQL

References

[1]
B. Wardman, G. Shukla, and G. Warner, Identifying vulnerable websites by analysis of common strings in phishing URLs, in eCrime Researchers Summit, eCRIME ’09, Tacoma, USA, 2009.
Crossref
[2]
S. Li and R. Schmitz, A novel anti-phishing framework based on honeypots, in eCrime Researchers Summit, eCRIME’09, Tacoma, USA, 2009.
[3]
R. Layton, P. Watters, and R. Dazeley, Automatically determining phishing campaigns using the USCAP methodology, in eCrime Researchers Summit (eCrime), 2010.
Crossref
[4]
S. Sheng, B. Wardman, G. Warner, L. Cranor, J. Hong, and C. Zhang, An empirical analysis of phishing blacklists, in Proc. Sixth Conference on Email and AntiSpam (CEAS 2009), Mountain View, California, USA, 2009.
[5]
Z. Chen, C. Lin, U. Ni, D. H. Ruan, B. Zheng, Y. X. Jiang, X. H. Deng, Y. Wang, A. A. Luo, B. Zhu, Y. Yue, and F. Y. Ren, AntiWorm NPU-based parallel bloom filters for TCP/IP content processing in Giga-Ethernet LAN, in Local Computer Networks, 30th Anniversary, Sydney, Australia, 2005, pp. 748-755.
Crossref
[6]
D. H. Ruan, Z. Chen, C. Lin, J. Ni, and P. D. Unysundn, Handling high speed traffic measurement using network processors, in Communication Technology, ICCT ’06, Guilin, 2006, pp. 1-5.
[7]
S. H. Huang, Z. Chen, A. A. Luo, X. Jiang, K. Wang, H. Zheng, and X. H. Peng, Proxy-based security audit system for remote desktop access, in Computer Communications and Networks, ICCCN 2009, San Francisco, USA, 2006, pp. 1095-2055.
[8]
C. Cranor, T. Johnson, and O. Spatscheck, Gigascope: A stream database for network applications, in 2003 ACM SIGMOD International Conference on Management of Data, New York, USA, 2003, pp. 647-651.
Crossref
[9]
M. Sullivan and A. Heybey, Tribeca: A system for managing large databases of network traffic, in Proceedings of the USENIX Annual Technical Conference (NO 98), ew Orleans, Louisiana, June 1998.
[10]
L. Deri, V. Lorenzetti, and S. Mortimer, Collection and exploration of large data monitoring sets using bitmap databases, in Traffic Monitoring and Analysis(TMA), Jan 2010, pp. 73-86.
Crossref
[11]
P. Desnoyers and P. Shenoy, Hyperion: High volume stream archival for retrospective querying, in USENIX Annual Technical Conference, Santa Clara, USA, 2007.
[12]
S. Kornexl, V. Paxson, H. Dreger, A. Feldmann, and R. Sommer, Building a time machine for efficient recording and retrieval of high-volume network traffic, in Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement, Berkeley, CA, USA, 2005, p. 23.
Crossref
[13]
G. Maier, R. Sommer, H. Dreger, A. Feldmann, V. Paxson, and F. Schneider, Enriching network security analysis with time travel, in Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication, New York, USA, 2008, pp. 183-194.
Crossref
[14]
L. Deri, A. Cardigliano, and F. Fusco, 10 Gbit line rate packet-to-disk using n2disk, in Proc. of 2013 Traffic Monitoring and Analysis Workshop, Turin, Italy, 2013.
Crossref
[15]
TimeMachine project in Bro, http://tracker.bro.org/time-machine/, 2013.
[16]
J. Li, S. Ding, M. Xu, F. Han, X. Guan, and Z. Chen, TIFA: Enabling real-time querying and storage of massive stream data, in Networking and Distributed Computing (ICNDC), 2011 Second International Conference on IEEE, Beijing, China, 2011.
Crossref
[17]
K. Wu, E. W. Bethel, J. Chen, H. Childs, E. Cormier-Michel, C. Geddes, J. Gu, H. Hagen, B. Hamann, W. Koegler, J. Lauret, J. Meredith, P. Messmer, E. Otoo, V. Perevoztchikov, A. Poskanzer, , O. Rbel, A. Shoshani, A. Sim, K. Stockinger, G. Weber, and W.-M. Zhang, FastBit: Interactively searching massive data, in Proc. of SciDAC 2009, 2009.
Crossref
[18]
B. Claise, Cisco systems netflow services export version 9, RFC 3954, 2004.
[19]
P. Phaal, S. Panchen, and N. McKee, InMon corporations sflow: A method for monitoring traffic in switched and routed networks, RFC 3176, 2001, pp. 1-31.
[20]
P. Haag, Watch your flows with nfsen and nfdump, in 50th RIPE Meeting, Stockholm, Sweden, 2005.
[21]
M. Fullmer and S. Roming, The OSU flowtools packetage and cisco netFlow logs, in Proc. of 19th Intl. Conference on Scientific and Statistical Database Management, Banff, Canada, 2007.
[22]
NEye, An open source netflow collector, http://neye.unsupported.info, 2004.
[23]
J. P. Navarro, B. Nickless, and L. Winkler, Combining cisco NetFlow exports with relational database technology for usage statistics, intrusion detection, and network forensics, in Proceedings of the 14th Large Installation Systems Administration Conference (LISA 2000), 2000, pp. 285-290.
[24]
F. Fusco, X. Dimitropoulos, and M. Vlachos, PcapIndex: An index for network packet traces with legacy compatibility, ACM SIGCOMM Computer Communication Review, vol. 42, no. 1, pp. 47-53, January 2012.
Crossref Google Scholar
[25]
F. Fusco, M. Vlachos, X. Dimitropoulos, and L. Deri, Indexing million of packets per second using GPUs, in Proc. of the 13th ACM SIGCOMM Conference on Internet Measurement, IMC 2013, 2013.
Crossref
[26]
F. Fusco and L. Deri, High speed network traffic analysis with commodity multi-core systems, in Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, New York, USA, 2010, pp. 218-224.
Crossref
[27]
Cavium Network Processor OCTEON 58XX, http://www.cavium.com/OCTEON-Plus_CN58XX.html, 2013.
[28]
J. Meng, X. Chen, Z. Chen, C. Lin, B. Mu, and L. Ruan, Towards high-performance ipsec on cavium OCTEON platform, in Trusted Systems, 2011, pp. 37-46.
Crossref
[29]
Tilera manycore network processor, http://www.tilera.com/products/processors/TILEPRO64, 2013.
[30]
S. Ding, Z. Chen, and Z. Liu, Parallelizing FIB lookup in content centric networking, in Networking and Distributed Computing (ICNDC), Hangzhou, China, 2012, pp. 6-10.
Crossref
[31]
V. Paxson, Bro: A system for detecting network intruders in real-time, Computer Networks, vol. 31, no. 23, pp. 2435-2463, 1998.
Crossref Google Scholar
[32]
F. Fusco, M. P Stoecklin, and M. Vlachos, NET-FLi: On-the-fly compression, archiving and indexing of streaming network traffic, Proceedings of the VLDB Endowment, vol. 3, no. 1-2, pp. 1382-1393, 2010.
Crossref Google Scholar
[33]
Z. Chen, X. Shi, L. Y. Raan, F. Xie, and J. Li, High speed traffic archiving system for flow granularity storage and querying, in Computer Communications and Networks (ICCCN), Munich, Germany, 2012, pp. 1-5.
Crossref
[34]
X. Chen, B. Mu, and Z. Chen, NetSecu: A collaborative network security platform for in-network security, in Proc. the 3rd International Conference on Communications and Mobile Computing (CMC), Qingdao, China, 2011, pp. 59-64.
Crossref
[35]
B. P. Mu, X. M. Chen, and Z. Chen, A collaborative network security management system in metropolitan area network, in Communications and Mobile Computing (CMC), 2011 Third International Conference, Qingdao, China, 2011, pp. 45-50.
Crossref
[36]
F. C. Deng, A. A. Luo, Y. K. Zhang, Z. Chen, X. H. Peng, X, Jiang, and D. S. Peng, TNC-UTM: A holistic solution to secure enterprise networks, in Young Computer Scientists, ICYCS 2008, Hunan, China, 2008, pp. 2240-2245
Crossref
[37]
Z. Chen, F. Y. Han, J. W. Cao, and S. Shen, Cloud computing-based forensic analysis for collaborative network security management system, Tsinghua Science and Technology, vol. 18, no. 1, pp. 40-50, 2013.
Crossref Google Scholar
[38]
F. Han, Z. Chen, H. Xu, and Y. Liang, Garlic: A distributed botnets suppression system, in Distributed Computing Systems Workshops (ICDCSW), Macau, China, 2012, pp. 634-639.
Crossref
[39]
A. Luo, L. Chuang, Z. Chen, X. Peng, and P. D. Ungsunan, TNC-compatible NAC system implemented on network processor, in Local Computer Networks, Dublin, Ireland, 2007, pp. 1096-1075.
Crossref
[40]
Z. Ying, F. Deng, Z. Chen, Y. Xue, and C. Lin, UTM-CM: A practical control mechanism solution for UTM system, in Communications and Mobile Computing (CMC), Shenzhen, China, 2010, pp. 86-90.
[41]
L. Deri and F. Fusco, MicroCloud-based network traffic monitoring, in Proc. of the Intern. Symposium on Integrated Network Management, IM 2013, 2013.
[42]
T. Y. Li, F. Y. Han, S. Ding, and Z. Chen, LARX: Large-scale anti-phishing by retrospective data-exploring based on a cloud computing platform, in Computer Communications and Networks (ICCCN), 2011 Proceedings of 20th International Conference on IEEE, Maui, USA, 2011, pp. 1-5.
Crossref
Tsinghua Science and Technology
Volume 18 Issue 4,
August 2013
Pages 406-417
DOI: 10.1109/TST.2013.6574679
Cite this article:
Chen Z, Ruan L, Cao J, et al. TIFAflow: Enhancing Traffic Archiving System with Flow Granularity for Forensic Analysis in Network Security. Tsinghua Science and Technology, 2013, 18(4): 406-417. https://doi.org/10.1109/TST.2013.6574679

557

Views

8

Downloads

10

Crossref

N/A

Web of Science

16

Scopus

0

CSCD

Altmetrics
Received: 15 July 2013
Accepted: 15 July 2013
Published: 05 August 2013
© The author(s) 2013
Return