It’s easy to capture packets with Wireshark, the world’s

    most popular network sniffer, whether off the wire or

    from the air. But how do you use those packets to

    understand what’s happening on your network?

    Updated to cover Wireshark 2.x, the third edition

    of Practical Packet Analysis will teach you to make

    sense of your packet captures so that you can better

    troubleshoot network problems. You’ll find added

    coverage of IPv6 and SMTP, a new chapter on the

    powerful command line packet analyzers tcpdump

    and TShark, and an appendix on how to read and

    reference packet values using a packet map.

    Practical Packet Analysis will show you how to:

    · Monitor your network in real time and tap live

    network communications

    · Build customized capture and display filters

    · Use packet analysis to troubleshoot and resolve

    common network problems, like loss of connectivity,DNS issues, and slow speeds

    · Explore modern exploits and malware at the packet

    level

    · Extract files sent across a network from packet

    captures

    · Graph traffic patterns to visualize the data flowing

    across your network

    · Use advanced Wireshark features to understand

    confusing captures

    · Build statistics and reports to help you better explain

    technical network information to non-techies

    No matter what your level of experience is, Practical

    Packet Analysis will show you how to use Wireshark to

    make sense of any network and get things done.

    ABOU T THE AU THOR

    Chris Sanders is a computer security consultant,researcher, and educator. He is the author of Applied

    Network Security Monitoring and blogs regularly at

    ChrisSanders.org. Chris uses packet analysis daily to

    catch bad guys and find evil.

    Download the capture files

    used in this book from

    nostarch.compacketanalysis3

    SHELVE IN:

    NETWORKINGSECURITY

    49.95 (57.95 CDN)

    www.nostarch.com

    THE F INE ST IN GEEK ENT ERTAINMENT?

    FSC LOGO

    The author’s royalties from this book

    will be donated to the Rural Technology Fund

    (http:ruraltechfund.org).

    COVERS WIRESHARK 2.X

    “ I L I E F LAT .”

    Thi s book uses a durable binding that won’t snap shut.

    D O N’ T J U S T S T A R E

    A T C A P T U R E D

    PA C K E T S .

    A N A LY Z E T H EM.

    DO N’ T J U S T S T A R E

    A T C A P T U R E D

    PA C K E T S .

    A N A LY Z E T H EM.

    PR ACTICAL

    PACKE T ANALYSIS

    PR ACTICAL

    PACKE T ANALYSIS U SIN G W I R E SH A R K T O S O L V E R E A L - W O R L D

    N E T W O R K P R O B L E M S

    C H R I S S A N D E R S

    3RD

    EDI T ION

    PR ACT ICA L PACKE T ANA LYSIS PR ACT ICA L PACKE T ANA LYSIS S A NDERS 3RD

    EDI T IONPraise for Practical Packet analysis

    “A wealth of information. Smart, yet very readable, and honestly made me

    excited to read about packet analysis.”

    —TechRepublic

    “I’d recommend this book to junior network analysts, software developers,and the newly minted CSECISSPetc.—folks that just need to roll up their

    sleeves and get started troubleshooting network (and security) problems.”

    —GunTeR Ollmann, fORmeR chief Technical OfficeR Of iOacTive

    “The next time I investigate a slow network, I’ll turn to Practical Packet Analysis.

    And that’s perhaps the best praise I can offer on any technical book.”

    —michael W. lucas, auThOR Of Absolute FreebsD and Network Flow ANAlysis

    “An essential book if you are responsible for network administration on

    any level.”

    —linux pRO maGazine

    “A wonderful, simple-to-use, and well-laid-out guide.”

    —aRsGeek.cOm

    “If you need to get the basics of packet analysis down pat, this is a very good

    place to start.”

    —sTaTeOfsecuRiTy.cOm

    “Very informative and held up to the key word in its title, practical. It does

    a great job of giving readers what they need to know to do packet analysis

    and then jumps right in with vivid real-life examples of what to do with

    Wireshark.”

    —linuxsecuRiTy.cOm

    “Are there unknown hosts chatting away with each other? Is my machine talk-

    ing to strangers? You need a packet sniffer to really find the answers to these

    questions. Wireshark is one of the best tools to do this job, and this book is

    one of the best ways to learn about that tool.”

    —fRee sOfTWaRe maGazine

    “Perfect for the beginner to intermediate.”

    —daemOn neWsPractical

    Packet analysis

    3rd edition

    Using Wireshark to solve

    real-World network Problems

    by Chris sanders

    San FranciscoPractical Packet analysis, 3rd edition. Copyright ? 2017 by Chris Sanders.

    All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means,electronic or mechanical, including photocopying, recording, or by any information storage or retrieval

    system, without the prior written permission of the copyright owner and the publisher.

    21 20 19 18 17 1 2 3 4 5 6 7 8 9

    ISBN-10: 1-59327-802-0

    ISBN-13: 978-1-59327-802-1

    Publisher: William Pollock

    Production Editor: Serena Yang

    Cover Illustration: Octopod Studios

    Interior Design: Octopod Studios

    Developmental Editor: William Pollock and Jan Cash

    Technical Reviewer: Tyler Reguly

    Copyeditor: Paula L. Fleming

    Compositor: Janelle Ludowise

    Proofreader: James Fraleigh

    Indexer: BIM Creatives, LLC.

    For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly:

    No Starch Press, Inc.

    245 8th Street, San Francisco, CA 94103

    phone: 1.415.863.9900; info@nostarch.com

    www.nostarch.com

    The Library of Congress has catalogued the first edition as follows:

    Sanders, Chris, 1986-

    Practical packet analysis : using Wireshark to solve real-world network problems Chris Sanders.

    p. cm.

    ISBN-13: 978-1-59327-149-7

    ISBN-10: 1-59327-149-2

    1. Computer network protocols. 2. Packet switching (Data transmission) I. Title.

    TK5105.55.S265 2007

    004.6'6--dc22

    2007013453

    No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other

    product and company names mentioned herein may be the trademarks of their respective owners. Rather

    than use a trademark symbol with every occurrence of a trademarked name, we are using the names only

    in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the

    trademark.

    The information in this book is distributed on an “As Is” basis, without warranty. While every precaution

    has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any

    liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or

    indirectly by the information contained in it. “Amazing grace, how sweet the sound

    That saved a wretch like me.

    I once was lost but now I’m found.

    Was blind but now I see.”Brief contents

    Acknowledgments xv

    Introduction xvii

    Chapter 1: Packet Analysis and Network Basics 1

    Chapter 2: Tapping into the Wire 17

    Chapter 3: Introduction to Wireshark 37

    Chapter 4: Working with Captured Packets 53

    Chapter 5: Advanced Wireshark Features 77

    Chapter 6: Packet Analysis on the Command Line 103

    Chapter 7: Network Layer Protocols 119

    Chapter 8: Transport Layer Protocols 151

    Chapter 9: Common Upper-Layer Protocols 163

    Chapter 10: Basic Real-World Scenarios 199

    Chapter 11: Fighting a Slow Network 231

    Chapter 12: Packet Analysis for Security 257

    Chapter 13: Wireless Packet Analysis 295

    Appendix A: Further Reading 317

    Appendix B: Navigating Packets 325

    Index 333contents in de tail

    acknowledGments xv

    introduction xvii

    Why This Book? xviii

    Concepts and Approach xviii

    How to Use This Book xx

    About the Sample Capture Files xx

    The Rural Technology Fund xxi

    Contacting Me xxi

    1

    Packet analysis and network Basics 1

    Packet Analysis and Packet Sniffers 2

    Evaluating a Packet Sniffer 2

    How Packet Sniffers Work 3

    How Computers Communicate 4

    Protocols 4

    The Seven-Layer OSI Model 5

    Network Hardware 10

    Traffic Classifications 15

    Broadcast Traffic 15

    Multicast Traffic 16

    Unicast Traffic 16

    Final Thoughts 16

    2

    taPPinG into the wire 17

    Living Promiscuously 18

    Sniffing Around Hubs 19

    Sniffing in a Switched Environment 20

    Port Mirroring 21

    Hubbing Out 23

    Using a Tap 24

    ARP Cache Poisoning 27

    Sniffing in a Routed Environment 31

    Sniffer Placement in Practice 33

    3

    introduction to wireshark 37

    A Brief History of Wireshark 37

    The Benefits of Wireshark 38x Contents in Detail

    Installing Wireshark 39

    Installing on Windows Systems 39

    Installing on Linux Systems 41

    Installing on OS X Systems 43

    Wireshark Fundamentals 44

    Your First Packet Capture 44

    Wireshark’s Main Window 45

    Wireshark Preferences 46

    Packet Color Coding 48

    Configuration Files 50

    Configuration Profiles 50

    4

    workinG with caPtured Packets 53

    Working with Capture Files 53

    Saving and Exporting Capture Files 54

    Merging Capture Files 55

    Working with Packets 56

    Finding Packets 56

    Marking Packets 57

    Printing Packets 58

    Setting Time Display Formats and References 58

    Time Display Formats 59

    Packet Time Referencing 60

    Time Shifting 60

    Setting Capture Options 61

    Input Tab 61

    Output Tab 62

    Options Tab 63

    Using Filters 65

    Capture Filters 65

    Display Filters 71

    Saving Filters 74

    Adding Display Filters to a Toolbar 75

    5

    advanced wireshark Features 77

    Endpoints and Network Conversations 78

    Viewing Endpoint Statistics 78

    Viewing Network Conversations 79

    Identifying Top Talkers with Endpoints and Conversations 80

    Protocol Hierarchy Statistics 83

    Name Resolution 84

    Enabling Name Resolution 84

    Potential Drawbacks to Name Resolution 86

    Using a Custom hosts File 86

    Manually Initiated Name Resolution 88

    Protocol Dissection 88

    Changing the Dissector 88

    Viewing Dissector Source Code 90Contents in Detail xi

    Following Streams 91

    Following SSL Streams 92

    Packet Lengths 93

    Graphing 95

    Viewing IO Graphs 95

    Round-Trip Time Graphing 98

    Flow Graphing 99

    Expert Information 99

    6

    Packet analysis on the command line 103

    Installing TShark 104

    Installing tcpdump 105

    Capturing and Saving Packets 106

    Manipulating Output 109

    Name Resolution 111

    Applying Filters 113

    Time Display Formats in TShark 114

    Summary Statistics in TShark 115

    Comparing TShark and tcpdump 118

    7

    network layer Protocols 119

    Address Resolution Protocol (ARP) 120

    ARP Packet Structure 121

    Packet 1: ARP Request 122

    Packet 2: ARP Response 123

    Gratuitous ARP 124

    Internet Protocol (IP) 125

    Internet Protocol Version 4 (IPv4) 125

    Internet Protocol Version 6 (IPv6) 133

    Internet Control Message Protocol (ICMP) 144

    ICMP Packet Structure 144

    ICMP Types and Messages 144

    Echo Requests and Responses 145

    traceroute 147

    ICMP Version 6 (ICMPv6) 150

    8

    transPort layer Protocols 151

    Transmission Control Protocol (TCP) 151

    TCP Packet Structure 152

    TCP Ports 152

    The TCP Three-Way Handshake 155

    TCP Teardown 158

    TCP Resets 159

    User Datagram Protocol (UDP) 160

    UDP Packet Structure 161xii Contents in Detail

    9

    common uPPer-layer Protocols 163

    Dynamic Host Configuration Protocol (DHCP) 163

    DHCP Packet Structure 164

    The DHCP Initialization Process 165

    DHCP In-Lease Renewal 170

    DHCP Options and Message Types 170

    DHCP Version 6 (DHCPv6) 171

    Domain Name System (DNS) 173

    DNS Packet Structure 173

    A Simple DNS Query 174

    DNS Question Types 176

    DNS Recursion 177

    DNS Zone Transfers 181

    Hypertext Transfer Protocol (HTTP) 183

    Browsing with HTTP 183

    Posting Data with HTTP 186

    Simple Mail Transfer Protocol (SMTP) 187

    Sending and Receiving Email 188

    Tracking an Email Message 189

    Sending Attachments via SMTP 196

    Final Thoughts 198

    10

    Basic real-world scenarios 199

    Missing Web Content 200

    Tapping into the Wire 200

    Analysis 201

    Lessons Learned 204

    Unresponsive Weather Service 205

    Tapping into the Wire 206

    Analysis 206

    Lessons Learned 209

    No Internet Access 210

    Gateway Configuration Problems 210

    Unwanted Redirection 213

    Upstream Problems 216

    Inconsistent Printer 219

    Tapping into the Wire 219

    Analysis 219

    Lessons Learned 222

    No Branch Office Connectivity 222

    Tapping into the Wire 223

    Analysis 223

    Lessons Learned 226

    Software Data Corruption 226

    Tapping into the Wire 226

    Analysis 227

    Lessons Learned 230

    Final Thoughts 230Contents in Detail xiii

    11

    FiGhtinG a slow network 231

    TCP Error-Recovery Features 232

    TCP Retransmissions 232

    TCP Duplicate Acknowledgments and Fast Retransmissions 235

    TCP Flow Control 240

    Adjusting the Window Size 241

    Halting Data Flow with a Zero Window Notification 242

    The TCP Sliding Window in Practice 243

    Learning from TCP Error-Control and Flow-Control Packets 247

    Locating the Source of High Latency 248

    Normal Communications 248

    Slow Communications: Wire Latency 248

    Slow Communications: Client Latency 249

    Slow Communications: Server Latency 250

    Latency Locating Framework 251

    Network Baselining 251

    Site Baseline 252

    Host Baseline 253

    Application Baseline 254

    Additional Notes on Baselines 255

    Final Thoughts 255

    12

    Packet analysis For security 257

    Reconnaissance 258

    SYN Scan 258

    Operating System Fingerprinting 263

    Traffic Manipulation 266

    ARP Cache Poisoning 267

    Session Hijacking 271

    Malware 275

    Operation Aurora 275

    Remote-Access Trojan 281

    Exploit Kit and Ransomware 288

    Final Thoughts 294

    13

    wireless Packet analysis 295

    Physical Considerations 296

    Sniffing One Channel at a Time 296

    Wireless Signal Interference 297

    Detecting and Analyzing Signal Interference 297

    Wireless Card Modes 298

    Sniffing Wirelessly in Windows 300

    Configuring AirPcap 300

    Capturing Traffic with AirPcap 302

    Sniffing Wirelessly in Linux 303

    802 11 Packet Structure 304xiv Contents in Detail

    Adding Wireless-Specific Columns to the Packet List Pane 305

    Wireless-Specific Filters 307

    Filtering Traffic for a Specific BSS ID 307

    Filtering Specific Wireless Packet Types 307

    Filtering a Specific Frequency 308

    Saving a Wireless Profile 309

    Wireless Security 309

    Successful WEP Authentication 309

    Failed WEP Authentication 311

    Successful WPA Authentication 312

    Failed WPA Authentication 314

    Final Thoughts 315

    a

    Further readinG 317

    Packet Analysis Tools 317

    CloudShark 317

    WireEdit 318

    Cain Abel 319

    Scapy 319

    TraceWrangler 319

    Tcpreplay 319

    NetworkMiner 319

    CapTipper 320

    ngrep 321

    libpcap 321

    Npcap 321

    hping 321

    Python 321

    Packet Analysis Resources 321

    Wireshark’s Home Page 322

    Practical Packet Analysis Online Course 322

    SANS’s Security Intrusion Detection In-Depth Course 322

    Chris Sanders’s Blog 322

    Brad Duncan’s Malware Traffic Analysis 322

    IANA’s Website 323

    W Richard Stevens’s TCPIP Illustrated Series 323

    The TCPIP Guide 323

    B

    naviGatinG Packets 325

    Packet Representation 326

    Using Packet Diagrams 328

    Navigating a Mystery Packet 330

    Final Thoughts 332

    index 333acknoWl edgments

    I’d like to express sincere gratitude for the people who’ve supported me

    and the development of this book.

    Ellen, thank you for your unconditional love and for putting up with

    me pecking away at the keyboard in bed for countless nights while you were

    trying to sleep.

    Mom, even in death the example of kindness you set continues to moti-

    vate me. Dad, I learned what hard work was from you and none of this hap-

    pens without that.

    Jason Smith, you’re like a brother to me, and I can’t thank you enough

    for being a constant sounding board.

    Regarding my coworkers past and present, I’m very fortunate to have

    surrounded myself with people who’ve made me a smarter, better person.

    There’s no way I can name everyone, but I want to sincerely thank Dustin,Alek, Martin, Patrick, Chris, Mike, and Grady for supporting me every day

    and embracing what it means to be servant leaders.

    Thanks to Tyler Reguly who served as the primary technical editor. I

    make stupid mistakes sometimes, and you make me look less stupid. Also,thanks to David Vaughan for providing an extra set of eyes, Jeff Carrell

    for helping edit the IPv6 content, Brad Duncan for providing a capture

    fle used in the security chapter, and the team at QA Café for providing a

    Cloudshark license that I used to organize the packet captures for the book.xvi Acknowledgments

    Of course, I also have to extend thanks to Gerald Combs and the

    Wireshark development team. It’s the dedication of Gerald and hundreds

    of other developers that makes Wireshark such a great analysis platform.

    If it weren’t for their efforts, information technology and network security

    would be signifcantly worse off.

    Finally, thanks to Bill, Serena, Anna, Jan, Amanda, Alison, and the rest

    of the No Starch Press staff for their diligence in editing and producing all

    three editions of Practical Packet Analysis. introdUc tion

    This third edition of Practical Packet

    Analysis was written and edited over the

    course of a year and a half, from late 2015

    to early 2017, approximately 6 years after the

    second edition’s release and 10 years since publica-

    tion of the original. This book contains a signifcant

    amount of new content, with completely new capture fles and scenarios

    and an entirely new chapter covering packet analysis from the command

    line with TShark and tcpdump. If you liked the frst two editions, then

    you’ll like this one. It’s written in the same tone and breaks down explana-

    tions in a simple, understandable manner. If you were hesitant to try out

    the last two editions because they didn’t include the latest information on

    networking or Wireshark updates, you’ll want to read this one because of

    the expanded content on new network protocols and updated information

    on Wireshark 2.x.xviii Introduction

    why this Book?

    You may fnd yourself wondering why you should buy this book as opposed

    to any other book about packet analysis. The answer lies in the title: Practical

    Packet Analysis. Let’s face it—nothing beats real-world experience, and the

    closest you can come to that experience in a book is through practical

    examples with real-world scenarios.

    The frst half of this book gives you the knowledge you’ll need to

    understand packet analysis and Wireshark. The second half of the book is

    devoted entirely to practical cases that you could easily encounter in day-

    to-day network management.

    Whether you’re a network technician, a network administrator, a chief

    information offcer, a desktop technician, or even a network security ana-

    lyst, you will beneft greatly from understanding and using the packet analy-

    sis techniques described in this book.

    concepts and approach

    I’m generally a really laid-back guy, so when I teach a concept, I try to do so

    in a really laid-back way. This holds true for the language used in this book.

    It’s easy to get lost in technical jargon, but I’ve tried my best to keep things

    as casual as possible. I’ve defned all the terms and concepts clearly and

    without any added fuff. After all, I’m from the great state of Kentucky, so I

    try to keep the big words to a minimum. (But you’ll have to forgive me for

    some of the backwoods country verbiage you’ll fnd throughout the text.)

    The frst several chapters are integral to understanding the rest of the

    book, so make it a point to master the concepts in these pages frst. The

    second half of the book is purely practical. You may not see these exact

    scenarios in your workplace, but you will be able to apply the concepts they

    teach in the situations you do encounter.

    Here is a quick breakdown of this book’s contents:

    Chapter 1: Packet Analysis and Network Basics

    What is packet analysis? How does it work? How do you do it? This chap-

    ter covers the basics of network communication and packet analysis.

    Chapter 2: Tapping into the Wire

    This chapter covers the different techniques for placing a packet sniffer

    on your network.

    Chapter 3: Introduction to Wireshark

    Here, we’ll look at the basics of Wireshark—where to get it, how to

    use it, what it does, why it’s great, and all that good stuff. This edition

    includes a new discussion about customizing Wireshark with confgura-

    tion profles.Introduction xix

    Chapter 4: Working with Captured Packets

    After you have Wireshark up and running, you’ll want to know how to

    interact with captured packets. This is where you’ll learn the basics,including new, more detailed sections on following packet streams and

    name resolution.

    Chapter 5: Advanced Wireshark Features

    Once you’ve learned to crawl, it’s time to take off running. This chap-

    ter delves into the advanced Wireshark features, taking you under the

    hood to show you some of the less apparent operations. This includes

    new, more detailed sections on following packet streams and name

    resolution.

    Chapter 6: Packet Analysis on the Command Line

    Wireshark is great, but sometimes you need to leave the comfort of a

    graphical interface and interact with a packet on the command line.

    This new chapter shows you how to use TShark and tcpdump, the two

    best command line packet analysis tools for the job.

    Chapter 7: Network Layer Protocols

    This chapter shows you what common network layer communication

    looks like at the packet level by examining ARP, IPv4, IPv6, and ICMP.

    To troubleshoot these protocols in real-life scenarios, you frst need to

    understand how they work.

    Chapter 8: Transport Layer Protocols

    Moving up the stack, this chapter discusses the two most common

    transport protocols, TCP and UDP. The majority of packets you look

    at will use one of these two protocols, so understanding what they look

    like at the packet level and how they differ is important.

    Chapter 9: Common Upper-Layer Protocols

    Continuing with protocol coverage, this chapter shows you what four

    of the most common upper-layer network communication protocols—

    HTTP, DNS, DHCP, and SMTP—look like at the packet level.

    Chapter 10: Basic Real-World Scenarios

    This chapter contains breakdowns of some common traffc and the

    frst set of real-world scenarios. Each scenario is presented in an easy-

    to- follow format, giving the problem, an analysis, and a solution. These

    basic scenarios deal with only a few computers and involve a limited

    amount of analysis—just enough to get your feet wet.

    Chapter 11: Fighting a Slow Network

    The most common problems network technicians hear about generally

    involve slow network performance. This chapter is devoted to solving

    these types of problems.xx Introduction

    Chapter 12: Packet Analysis for Security

    Network security is the biggest hot-button topic in the information

    technology area. Chapter 12 shows you some scenarios related to solv-

    ing security-related issues with packet analysis techniques.

    Chapter 13: Wireless Packet Analysis

    This chapter is a primer on wireless packet analysis. It discusses the dif-

    ferences between wireless analysis and wired analysis, and it includes

    some examples of wireless network traffc.

    Appendix A: Further Reading

    The frst appendix of this book suggests some other reference tools and

    websites that you might fnd useful as you continue to use the packet

    analysis techniques you’ve learned.

    Appendix B: Navigating Packets

    If you want to dig a little deeper into interpreting individual packets,the second appendix provides an overview of how packet information is

    stored in binary and how to convert binary into hexadecimal notation.

    Then it shows you how to dissect packets that are presented in hexa-

    decimal notation with packet diagrams. This is handy if you’re going to

    spend a lot of time analyzing custom protocols or using command line

    analysis tools.

    how to use this Book

    I have intended this book to be used in two ways:

    · As an educational text. You’ll read chapter by chapter, paying particular

    attention to the real-world scenarios in the later chapters, to gain an

    understanding of packet analysis.

    · As a reference. There are some features of Wireshark that you won’t use

    very often, so you may forget how they work. Practical Packet Analysis is a

    great book to have on your bookshelf when you need a quick refresher

    on how to use a specifc feature. When doing packet analysis for your

    job, you may want to reference the unique charts, diagrams, and meth-

    odologies I’ve provided.

    about the sample capture Files

    All of the capture fles used in this book are available from the book’s

    No Starch Press page, https:www.nostarch.compacketanalysis3. To maximize

    the potential of this book, download these fles and use them as you follow

    along with the examples.Introduction xxi

    the rural technology Fund

    I couldn’t write an introduction without mentioning the best thing to come

    from Practical Packet Analysis. Shortly after the release of the frst edition

    of this book, I founded a 501(c)(3) nonproft organization—the Rural

    Technology Fund (RTF).

    Rural students, even those with excellent grades, often have fewer

    opportunities for exposure to technology than their city or suburban

    counterparts. Established in 2008, the RTF is the culmination of one of

    my biggest dreams. It seeks to reduce the digital divide between rural com-

    munities and their urban and suburban counterparts. The RTF does this

    through targeted scholarship programs, community involvement, donations

    of educational technology resources to classrooms, and general promotion

    and advocacy of technology in rural and high-poverty areas.

    In 2016, the RTF was able to put technology education resources into the

    hands of more than 10,000 students in rural and high-poverty areas in the

    United States. I’m pleased to announce that all of the author’s proceeds

    from this book go directly to the RTF to support these goals. If you want to

    learn more about the Rural Technology Fund or how you can contribute,visit our website at http:www.ruraltechfund.org or follow us on Twitter

    @RuralTechFund.

    contacting me

    I’m always thrilled to get feedback from people who read my writing. If you

    would like to contact me for any reason, you can send all questions, com-

    ments, threats, and marriage proposals directly to me at chris@chrissanders

    .org. I also blog regularly at http:www.chrissanders.org and can be followed

    on Twitter at @chrissanders88.1 Packet analysis and

    ne twork Basics

    A million different things can go wrong

    with a computer network on any given

    day—from a simple spyware infection to

    a complex router confguration error—and

    it’s impossible to solve every problem immediately.

    The best we can hope for is to be fully prepared with

    the knowledge and tools we need to respond to these

    types of issues.

    To truly understand network problems, we go to the packet level. All

    network problems stem from this level, where even the prettiest-looking

    applications can reveal their horrible implementations and seemingly trust-

    worthy protocols can prove malicious. Here, nothing is hidden from us.

    Nothing is obscured by misleading menu structures, eye-catching graphics, 2 Chapter 1

    or untrustworthy employees—there are no true secrets (only encrypted

    ones). The more we can do at the packet level, the more we can control

    our network and solve problems. This is the world of packet analysis.

    This book dives into this world headfrst. Through real-world scenarios,you’ll learn how to tackle slow network communication, identify application

    bottlenecks, and even track hackers. By the time you’ve fnished reading

    this book, you should be able to implement packet analysis techniques that

    will help you solve even the most diffcult problems in your own network.

    In this chapter, we’ll begin with the basics, focusing on network com-

    munication. The material here will help you gain the tools you’ll need to

    examine different scenarios.

    Packet Analysis and Packet Sniffers

    Packet analysis, often referred to as packet sniffng or protocol analysis,describes the process of capturing and interpreting live data as it fows

    across a network in order to better understand what is happening on that

    network. Packet analysis is typically performed by a packet sniffer, a tool used

    to capture raw network data going across the wire.

    Packet analysis can help with the following:

    · Understanding network characteristics

    · Learning who is on a network

    · Determining who or what is utilizing available bandwidth

    · Identifying peak network usage times

    · Identifying malicious activity

    · Finding unsecured and bloated applications

    There are various types of packet-sniffng programs, including

    both free and commercial ones. Each program is designed with differ-

    ent goals in mind. A few popular packet analysis programs are tcpdump,OmniPeek, and Wireshark (we’ll primarily be using Wireshark in this

    book). OmniPeek and Wireshark have graphical user interfaces (GUIs),while tcpdump is a command line program.

    Evaluating a Packet Sniffer

    You need to consider a number of factors when selecting a packet sniffer,including the following:

    Supported protocols All packet sniffers can interpret various proto-

    cols. Most can interpret common network protocols (such as IPv4 and

    ICMP), transport protocols (such as TCP and UDP), and even applica-

    tion protocols (such as DNS and HTTP). However, they may not sup-

    port nontraditional, newer, or more complex protocols (such as IPv6,SMBv2, and SIP). When choosing a sniffer, make sure that it supports

    the protocols you’re going to use.Packet Analysis and Network Basics 3

    User friendliness Consider the packet sniffer’s layout, ease of instal-

    lation, and general workfow. The program you choose should ft your

    level of expertise. If you have very little packet analysis experience, you

    may want to avoid the more advanced command line packet sniffers

    like tcpdump. On the other hand, if you are a packet analysis veteran,you may fnd an advanced program more useful. As you gain experi-

    ence, you may even fnd it useful to combine multiple packet-sniffng

    programs to ft particular scenarios.

    Cost The great thing about packet sniffers is that there are many free

    ones that rival any commercial products. The most notable difference

    between commercial products and their free alternatives is their report-

    ing engines. Commercial products typically include some form of fancy

    report generation module, while free applications either lack this capa-

    bility or offer only very limited reporting.

    Program support Even after you have mastered the basics of a sniff-

    ing program, you may occasionally need support to solve new problems

    as they arise. When evaluating available support, look for developer

    documentation, public forums, and mailing lists. Although there may

    be a lack of formalized commercial support for free packet-sniffng

    programs like Wireshark, communities of users and contributors often

    provide active discussion boards, wikis, and blogs to help you get more

    out of your packet sniffer.

    Source code access Some packet sniffers are open source software.

    This means that you can view the source code of the program and, in

    some cases, even suggest and make changes to that source code. If you

    have a very specifc or advanced use case for a sniffng application, this

    might be an appealing feature. Most commercial applications don’t pro-

    vide source code access.

    Operating system support Unfortunately, not all packet sniffers sup-

    port every operating system. Choose one that will work on all the oper-

    ating systems that you need to support. If you are a consultant, you may

    be required to capture and analyze packets on a variety of operating

    systems, so you’ll need a tool that runs on most of them. Also, keep in

    mind that you’ll sometimes capture packets on one machine and review

    them on another. Variations between operating systems may force you

    to use a different application for each device.

    How Packet Sniffers Work

    The packet-sniffng process involves a cooperative effort between software

    and hardware. This process can be broken down into three steps:

    1. Collection: First, the packet sniffer collects raw binary data from the

    wire. Typically this is done by switching the selected network interface

    into promiscuous mode. In this mode, the network card can listen to all

    traffc on a network segment, not only the traffc that is addressed to it. 4 Chapter 1

    2. Conversion: Next, the captured binary data is converted into a read-

    able form. This is as far as most advanced command line packet sniffers

    can go. At this point, the network data can be interpreted only on a

    very basic level, leaving the majority of the analysis to the end user.

    3. Analysis: Finally, the packet sniffer conducts an analysis of the captured

    and converted data. The sniffer verifes the protocol of the captured net-

    work data based on the information extracted and begins its analysis of

    that protocol’s specifc features.

    How Computers Communicate

    To fully understand packet analysis, you must know exactly how computers

    communicate with each other. In this section, we’ll examine the basics of

    network protocols, the Open Systems Interconnections (OSI) model, net-

    work data frames, and the hardware that supports it all.

    Protocols

    Modern networks are made up of a variety of systems running on many dif-

    ferent platforms. To communicate between systems, we use a set of common

    languages called protocols. Common protocols include Transmission Control

    Protocol (TCP), Internet Protocol (IP), Address Resolution Protocol (ARP),and Dynamic Host Confguration Protocol (DHCP). A logical grouping of

    protocols that work together is called a protocol stack.

    It might help to think of protocols as similar to the rules that govern

    human language. Every language has rules such as how to conjugate verbs,how to greet people, and even how to properly thank someone. Protocols

    work in much the same fashion, allowing us to defne how packets should

    be routed, how to initiate a connection, and how to acknowledge the receipt

    of data.

    A protocol can be extremely simple or highly complex, depending on

    its function. Although the various protocols can differ signifcantly, many

    protocols address the following issues:

    Connection initiation Is it the client or server initiating the connec-

    tion? What information must be exchanged prior to communication?

    Negotiation of connection characteristics Is the communication of

    the protocol encrypted? How are encryption keys transmitted between

    communicating hosts?

    Data formatting How is the data contained within the packet orga-

    nized? In what order is the data processed by the devices receiving it?

    Error detection and correction What happens in the event that a

    packet takes too long to reach its destination? How does a client recover

    if it cannot establish communication with a server for a short duration?

    Connection termination How does one host signify to the other that

    communication has ended? What fnal information must be transmit-

    ted in order to gracefully terminate communication? Packet Analysis and Network Basics 5

    The Seven-Layer OSI Model

    Protocols are separated according to their

    functions based on the industry-standard OSI

    reference model. This hierarchical model,with seven distinct layers, is very helpful for

    understanding network communications. In

    Figure 1-1, the layers of the OSI model are on

    the right, and the proper terminology for data

    at each of these layers is on the left. The appli-

    cation layer at the top represents the pro-

    grams used to access network resources. The

    bottom layer is the physical layer, through

    which the network data travels. The protocols

    at each layer work together to ensure data is

    properly handled by the protocols at layers

    directly above and below.

    note The OSI model was originally published in 1983 by

    the International Organization for Standardization

    (ISO) as a document called ISO 7498. The OSI

    model is no more than an industry-recommended

    standard. Protocol developers are not required to fol-

    low it exactly. In fact, the OSI model is not the only

    networking model; for example, some people prefer

    the Department of Defense (DoD) model, also known

    as the TCPIP model.

    Each OSI model layer has a specifc function, as follows:

    Application layer (layer 7) The topmost layer of the OSI model pro-

    vides a means for users to access network resources. This is the only

    layer typically seen by end users, as it provides the interface that is the

    base for all of their network activities.

    Presentation layer (layer 6) This layer transforms the data it receives

    into a format that can be read by the application layer. The data encod-

    ing and decoding done here depends on the application layer protocol

    that is sending or receiving the data. The presentation layer also handles

    several forms of encryption and decryption used to secure data.

    Session layer (layer 5) This layer manages the dialogue, or session,between two computers. It establishes, manages, and terminates this

    connection among all communicating devices. The session layer is

    also responsible for establishing whether a connection is duplex (two-

    way) or half-duplex (one-way) and for gracefully closing a connection

    between hosts rather than dropping it abruptly.

    Transport layer (layer 4) The primary purpose of the transport layer

    is to provide reliable data transport services to lower layers. Through

    fow control, segmentationdesegmentation, and error control, the

    transport layer makes sure data gets from point to point error-free.

    Figure 1-1: A hierarchical

    view of the seven layers of

    the OSI model

    Application

    Presentation

    Session

    Transport

    Network

    Data Link

    Physical

    Data

    Data

    Data

    Segments

    Packets

    Frames

    Bits6 Chapter 1

    Because ensuring reliable data transportation can be extremely cum-

    bersome, the OSI model devotes an entire layer to it. The transport

    layer utilizes both connection-oriented and connectionless protocols.

    Certain frewalls and proxy servers operate at this layer.

    Network layer (layer 3) This layer, one of the most complex of the

    OSI layers, is responsible for routing data between physical networks. It

    sees to the logical addressing of network hosts (for example, through

    an IP address). It also handles splitting data streams into smaller frag-

    ments and, in some cases, error detection. Routers operate at this layer.

    Data link layer (layer 2) This layer provides a means of transporting

    data across a physical network. Its primary purpose is to provide an

    addressing scheme that can be used to identify physical devices (for

    example, MAC addresses). Bridges and switches are physical devices

    that operate at the data link layer.

    Physical layer (layer 1) The layer at the bottom of the OSI model is

    the physical medium through which network data is transferred. This

    layer defnes the physical and electrical nature of all hardware used,including voltages, hubs, network adapters, repeaters, and cabling spec-

    ifcations. The physical layer establishes and terminates connections,provides a means of sharing communication resources, and converts

    signals from digital to analog and vice versa.

    note A common mnemonic device for remembering the layers of the OSI model is Please

    Do Not Throw Sausage Pizza Away. The frst letter of each word refers to each

    layer of the OSI model, starting with the frst layer.

    Table 1-1 lists some of the more common protocols used at each layer of

    the OSI model.

    Table 1-1: Typical Protocols Used at Each Layer of the OSI Model

    Layer Protocols

    Application HT T P, SMT P, F T P, Te ln e t

    Presentation ASCII, MPEG, JPEG, MIDI

    Session NetBIOS, SAP, SDP, NWLink

    Transport TCP, UDP, SPX

    Network IP, IPX

    Data link Ethernet, Token Ring, FDDI, AppleTalk

    Physical wired, wireless

    Although the OSI model is no more than a recommended standard,you should know it by heart as it provides a useful vocabulary for thinking

    about and describing network problems. As we progress through this book,you will fnd that router issues soon become “layer 3 problems” and soft-

    ware issues are readily recognized as “layer 7 problems.”Packet Analysis and Network Basics 7

    note A colleague once told me about a user who complained that he could not access a net-

    work resource. The issue was the result of the user’s entering an incorrect password.

    My colleague referred to this as a layer 8 issue. Layer 8 is the unoffcial user layer.

    This term is commonly used among those who live at the packet level.

    Data Flow Through the OSI Model

    The initial data transfer on a network begins at the application layer of the

    transmitting system. Data works its way down the seven layers of the OSI

    model until it reaches the physical layer, at which point the physical layer of

    the transmitting system sends the data to the receiving system. The receiv-

    ing system picks up the data at its physical layer, and the data proceeds up

    the layers of the receiving system to the application layer at the top.

    Each layer in the OSI model is capable of communicating only with the

    layers directly above and below it. For example, layer 2 can send and receive

    data only from layers 1 and 3.

    None of the services provided by various protocols at any given level

    of the OSI is redundant. For example, if a protocol at one layer provides

    a particular service, then no other protocol at any other layer will provide

    this same service. Protocols at different levels may have features with similar

    goals, but they will function a bit differently.

    Protocols at corresponding

    layers on the sending and receiv-

    ing devices are complementary.

    So, for example, if a protocol at

    layer 7 of the sending device is

    responsible for formatting the

    data being transmitted, the cor-

    responding protocol at layer 7 of

    the receiving device is expected

    to be responsible for reading that

    formatted data.

    Figure 1-2 is a graphical rep-

    resentation of the OSI model as

    it relates to two communicating

    devices. You can see communica-

    tion going from top to bottom

    on one device and then reversing

    when it reaches the second device.

    Data Encapsulation

    The protocols at different layers of

    the OSI model pass data between

    each other with the aid of data

    encapsulation. Each layer in the

    stack is responsible for adding a

    Application

    Presentation

    Session

    Transport

    Network

    Data Link

    Physical

    Application

    Presentation

    Session

    Transport

    Network

    Data Link

    Physical

    Figure 1-2: Protocols working at the same

    layer on both the sending and receiving

    systems8 Chapter 1

    header or footer—extra bits of information that allow the layers to communi-

    cate—to the data being transferred. For example, when the transport layer

    receives data from the session layer, the transport layer adds its own header

    information to that data before passing it to the network layer.

    The encapsulation process creates a protocol data unit (PDU), which

    includes the data being sent and all header or footer information added

    to it. As data moves down the OSI model and the various protocols add

    header and footer information, the PDU changes and grows. The PDU is in

    its fnal form when it reaches the physical layer, at which point it is sent to

    the destination device. The receiving device strips the protocol headers and

    footers from the PDU as the data climbs up the OSI layers in the reverse of

    the order they were added. Once the PDU reaches the top layer of the OSI

    model, only the original application layer data remains.

    note The OSI model uses specifc terms to describe packaged data at each layer. The physi-

    cal layer contains bits, the data link layer contains frames, the network layer contains

    packets, and the transport layer contains segments. The top three layers simply use the

    term data. This nomenclature isn’t used much in practice, so we’ll generally just use

    the term packet to refer to a complete or partial PDU that includes header and footer

    information from a few or many layers of the OSI model.

    To illustrate how encapsulation of data works, we’ll look at a simpli-

    fed practical example of a packet being built, transmitted, and received

    in relation to the OSI model. Keep in mind that as analysts, we don’t often

    talk about the session or presentation layers, so those will be absent in this

    example (and the rest of this book).

    In this scenario, we are attempting to browse to http:www.google.com.

    First, we must generate a request packet that is transmitted from our source

    client computer to the destination server computer. This scenario assumes

    that a TCPIP communication session has already been initiated. Figure 1-3

    illustrates the data encapsulation process in this example.

    We begin on our client computer at the application layer. We are brows-

    ing to a website, so the application layer protocol being used is HTTP; the

    HTTP protocol will issue a command to download the fle index.html from

    google.com.

    note In practice, the browser will request the website document root frst, signifed by a for-

    ward slash (). When the web server receives this request, it will redirect the browser to

    whatever fle it is confgured to serve upon receiving a document root request. This is

    usually something like index.html or index.php. We’ll cover this more in Chapter 9

    when we discuss HTTP.

    Once our application layer protocol has sent the command, our concern

    is with getting the packet to its destination. The data in our packet is passed

    down the OSI stack to the transport layer. HTTP is an application layer pro-

    tocol that uses (or sits on) TCP, so TCP serves as the transport layer protocol

    used to ensure reliable delivery of the packet. A TCP header is generated Packet Analysis and Network Basics 9

    and added to the PDU, as shown in the transport layer of Figure 1-3. This

    TCP header includes sequence numbers and other data that are appended

    to the packet, ensuring that the packet is properly delivered.

    Application

    Presentation

    Session

    Transport

    Network

    Data Link

    Physical

    HTTP HTTP

    HTTP HTTP TCP TCP

    HTTP TCP IP

    HTTP TCP IP Ethernet

    HTTP TCP IP

    HTTP TCP IP Ethernet

    Figure 1-3: A graphical representation of encapsulation of data between client and server

    note We often say that one protocol “sits on” or “rides on” another protocol because of the

    top-down design of the OSI model. An application protocol such as HTTP provides a

    particular service and relies on TCP to ensure reliable delivery of its service. Both of

    those services rely on the IP protocol at the network level to address and deliver their

    data. Therefore, HTTP sits on TCP, which sits on IP.

    Having done its job, TCP hands the packet off to IP, which is the

    layer 3 protocol responsible for the logical addressing of the packet. IP

    creates a header containing logical addressing information, adds it to the

    PDU, and passes the packet along to the Ethernet on the data link layer.

    Physical Ethernet addresses are stored in the Ethernet header. The packet

    is now fully assembled and passed to the physical layer, where it is transmit-

    ted as zeros and ones across the network.

    The completed packet traverses the network cabling system, eventu-

    ally reaching the Google web server. The web server begins by reading the

    packet from the bottom up, meaning that it frst reads the data link layer,which contains the physical Ethernet addressing information that the 10 Chapter 1

    network card uses to determine that the packet is intended for a particu-

    lar server. Once this information is processed, the layer 2 information is

    stripped away, and the layer 3 information is processed.

    The layer 3 IP addressing information is read to ensure that the packet

    is properly addressed and is not fragmented. This data is also stripped away

    so that the next layer can be processed.

    Layer 4 TCP information is now read to ensure that the packet has

    arrived in sequence. Then the layer 4 header information is stripped away

    to leave only the application layer data, which can be passed to the web

    server application hosting the website. In response to this packet from the

    client, the server should transmit a TCP acknowledgment packet so the

    client knows its request was received, followed by the index.html fle.

    All packets are built and processed as described in this example,regardless of which protocols are used. But at the same time, keep in mind

    that not every packet on a network is generated from an application layer

    protocol, so you will see packets that contain only information from layer 2,3, or 4 protocols.

    Network Hardware

    Now it’s time to look at network hardware, where the dirty work is done.

    We’ll focus on just a few of the more common pieces of network hardware:

    hubs, switches, and routers.

    Hubs

    A hub is generally a box with multiple RJ-45 ports, like the NETGEAR hub

    shown in Figure 1-4. Hubs range from very small 4-port devices to larger

    48-port devices designed for rack mounting in a corporate environment.

    Figure 1-4: A typical 4-port Ethernet hub

    Because hubs can generate a lot of unnecessary network traffc and are

    capable of operating only in half-duplex mode (they cannot send and receive

    data at the same time), you won’t typically see them used in most modern

    or high-density networks; switches are used instead (discussed in the next

    section). However, you should know how hubs work, since they will be very

    important to packet analysis when using the “hubbing out” technique dis-

    cussed in Chapter 2.

    A hub is no more than a repeating device that operates on the physical

    layer of the OSI model. It takes packets sent from one port and trans-

    mits (repeats) them to every other port on the device, and it’s up to the Packet Analysis and Network Basics 11

    receiving device to accept or reject each packet. For example, if a computer

    on port 1 of a 4-port hub needs to send data to a computer on port 2, the

    hub sends those packets to ports 2, 3, and 4. The clients connected to ports

    3 and 4 examine the destination Media Access Control (MAC) address feld

    in the Ethernet header of the packet and see that the packet is not for them,so they drop (discard) the packet. Figure 1-5 illustrates an example in which

    computer A is transmitting data to computer B. When computer A sends

    this data, all computers connected to the hub receive it. However, only com-

    puter B actually accepts the data; the other computers discard it.

    Computer A Computer C

    Computer B

    Computer D

    Figure 1-5: The fow of traffc when computer A transmits

    data to computer B through a hub

    As an analogy, suppose that you sent an email with the subject line

    “Attention all marketing staff” to every employee in your company, rather

    than to only those people who work in the marketing departm ......