Protocol Reverse Engineering

作者 wshobson46891e7e60da無授權條款收錄於 2026年10月8日更新於 2026年10月8日

Master network protocol reverse engineering including packet analysis, protocol dissection, and custom protocol documentation. Use when analyzing network traffic, understanding proprietary protocols, or debugging network communication.

AI 產生的概覽

用於擷取、剖析並記錄網路協定的技術,涵蓋二進位協定解析、加密分析與模糊測試。

功能
提供網路協定逆向工程的參考流程:使用 Wireshark、tshark、tcpdump 或中間人代理擷取流量,再透過顯示篩選器、串流追蹤、欄位擷取與統計進行分析。內容涵蓋以簽章與封包標頭模式辨識協定、長度前置與 TLV 等二進位結構解析、以熵值判斷加密、TLS 中介資料與解密方式,以及模糊測試與封包重放的主動測試。此外也提供協定規格範本、Wireshark Lua 解析器範例與一套分析工作流程。
適用情境
適用於分析已擷取的網路流量、辨識或記錄專有或未公開的協定,或偵錯網路通訊。也適合用於協定行為的資安研究,以及為自訂協定撰寫解析器、Wireshark 解析器或模糊測試工具。
執行需求
僅為說明文件,未附帶指令碼。文件中的流程引用外部工具與函式庫:Wireshark/tshark、tcpdump、mitmproxy、Burp Suite、Scapy、boofuzz,以及搭配 struct/dataclasses 的 Python。擷取即時流量需要網路存取與對應的網路介面權限;TLS 解密需要金鑰素材,例如 SSLKEYLOGFILE 或 RSA 私密金鑰。

Protocol Reverse Engineering

Comprehensive techniques for capturing, analyzing, and documenting network protocols for security research, interoperability, and debugging.

Traffic Capture

Wireshark Capture

bash
# Capture on specific interfacewireshark -i eth0 -k
# Capture with filterwireshark -i eth0 -k -f "port 443"
# Capture to filetshark -i eth0 -w capture.pcap
# Ring buffer capture (rotate files)tshark -i eth0 -b filesize:100000 -b files:10 -w capture.pcap

tcpdump Capture

bash
# Basic capturetcpdump -i eth0 -w capture.pcap
# With filtertcpdump -i eth0 port 8080 -w capture.pcap
# Capture specific bytestcpdump -i eth0 -s 0 -w capture.pcap  # Full packet
# Real-time displaytcpdump -i eth0 -X port 80

Man-in-the-Middle Capture

bash
# mitmproxy for HTTP/HTTPSmitmproxy --mode transparent -p 8080
# SSL/TLS interceptionmitmproxy --mode transparent --ssl-insecure
# Dump to filemitmdump -w traffic.mitm
# Burp Suite# Configure browser proxy to 127.0.0.1:8080

Protocol Analysis

Wireshark Analysis

# Display filterstcp.port == 8080http.request.method == "POST"ip.addr == 192.168.1.1tcp.flags.syn == 1 && tcp.flags.ack == 0frame contains "password"
# Following streamsRight-click > Follow > TCP StreamRight-click > Follow > HTTP Stream
# Export objectsFile > Export Objects > HTTP
# DecryptionEdit > Preferences > Protocols > TLS  - (Pre)-Master-Secret log filename  - RSA keys list

tshark Analysis

bash
# Extract specific fieldstshark -r capture.pcap -T fields -e ip.src -e ip.dst -e tcp.port
# Statisticstshark -r capture.pcap -q -z conv,tcptshark -r capture.pcap -q -z endpoints,ip
# Filter and extracttshark -r capture.pcap -Y "http" -T json > http_traffic.json
# Protocol hierarchytshark -r capture.pcap -q -z io,phs

Scapy for Custom Analysis

python
from scapy.all import *
# Read pcappackets = rdpcap("capture.pcap")
# Analyze packetsfor pkt in packets:    if pkt.haslayer(TCP):        print(f"Src: {pkt[IP].src}:{pkt[TCP].sport}")        print(f"Dst: {pkt[IP].dst}:{pkt[TCP].dport}")        if pkt.haslayer(Raw):            print(f"Data: {pkt[Raw].load[:50]}")
# Filter packetshttp_packets = [p for p in packets if p.haslayer(TCP)                and (p[TCP].sport == 80 or p[TCP].dport == 80)]
# Create custom packetspkt = IP(dst="target")/TCP(dport=80)/Raw(load="GET / HTTP/1.1\r\n")send(pkt)

Protocol Identification

Common Protocol Signatures

HTTP        - "HTTP/1." or "GET " or "POST " at startTLS/SSL     - 0x16 0x03 (record layer)DNS         - UDP port 53, specific header formatSMB         - 0xFF 0x53 0x4D 0x42 ("SMB" signature)SSH         - "SSH-2.0" bannerFTP         - "220 " response, "USER " commandSMTP        - "220 " banner, "EHLO" commandMySQL       - 0x00 length prefix, protocol versionPostgreSQL  - 0x00 0x00 0x00 startup lengthRedis       - "*" RESP array prefixMongoDB     - BSON documents with specific header

Protocol Header Patterns

+--------+--------+--------+--------+|  Magic number / Signature         |+--------+--------+--------+--------+|  Version       |  Flags          |+--------+--------+--------+--------+|  Length        |  Message Type   |+--------+--------+--------+--------+|  Sequence Number / Session ID     |+--------+--------+--------+--------+|  Payload...                       |+--------+--------+--------+--------+

Binary Protocol Analysis

Structure Identification

python
# Common patterns in binary protocols
# Length-prefixed messagestruct Message {    uint32_t length;      # Total message length    uint16_t msg_type;    # Message type identifier    uint8_t  flags;       # Flags/options    uint8_t  reserved;    # Padding/alignment    uint8_t  payload[];   # Variable-length payload};
# Type-Length-Value (TLV)struct TLV {    uint8_t  type;        # Field type    uint16_t length;      # Field length    uint8_t  value[];     # Field data};
# Fixed header + variable payloadstruct Packet {    uint8_t  magic[4];    # "ABCD" signature    uint32_t version;    uint32_t payload_len;    uint32_t checksum;    # CRC32 or similar    uint8_t  payload[];};

Python Protocol Parser

python
import structfrom dataclasses import dataclass
@dataclassclass MessageHeader:    magic: bytes    version: int    msg_type: int    length: int
    @classmethod    def from_bytes(cls, data: bytes):        magic, version, msg_type, length = struct.unpack(            ">4sHHI", data[:12]        )        return cls(magic, version, msg_type, length)
def parse_messages(data: bytes):    offset = 0    messages = []
    while offset < len(data):        header = MessageHeader.from_bytes(data[offset:])        payload = data[offset+12:offset+12+header.length]        messages.append((header, payload))        offset += 12 + header.length
    return messages
# Parse TLV structuredef parse_tlv(data: bytes):    fields = []    offset = 0
    while offset < len(data):        field_type = data[offset]        length = struct.unpack(">H", data[offset+1:offset+3])[0]        value = data[offset+3:offset+3+length]        fields.append((field_type, value))        offset += 3 + length
    return fields

Hex Dump Analysis

python
def hexdump(data: bytes, width: int = 16):    """Format binary data as hex dump."""    lines = []    for i in range(0, len(data), width):        chunk = data[i:i+width]        hex_part = ' '.join(f'{b:02x}' for b in chunk)        ascii_part = ''.join(            chr(b) if 32 <= b < 127 else '.'            for b in chunk        )        lines.append(f'{i:08x}  {hex_part:<{width*3}}  {ascii_part}')    return '\n'.join(lines)
# Example output:# 00000000  48 54 54 50 2f 31 2e 31  20 32 30 30 20 4f 4b 0d  HTTP/1.1 200 OK.# 00000010  0a 43 6f 6e 74 65 6e 74  2d 54 79 70 65 3a 20 74  .Content-Type: t

Encryption Analysis

Identifying Encryption

python
# Entropy analysis - high entropy suggests encryption/compressionimport mathfrom collections import Counter
def entropy(data: bytes) -> float:    if not data:        return 0.0    counter = Counter(data)    probs = [count / len(data) for count in counter.values()]    return -sum(p * math.log2(p) for p in probs)
# Entropy thresholds:# < 6.0: Likely plaintext or structured data# 6.0-7.5: Possibly compressed# > 7.5: Likely encrypted or random
# Common encryption indicators# - High, uniform entropy# - No obvious structure or patterns# - Length often multiple of block size (16 for AES)# - Possible IV at start (16 bytes for AES-CBC)

TLS Analysis

bash
# Extract TLS metadatatshark -r capture.pcap -Y "ssl.handshake" \    -T fields -e ip.src -e ssl.handshake.ciphersuite
# JA3 fingerprinting (client)tshark -r capture.pcap -Y "ssl.handshake.type == 1" \    -T fields -e ssl.handshake.ja3
# JA3S fingerprinting (server)tshark -r capture.pcap -Y "ssl.handshake.type == 2" \    -T fields -e ssl.handshake.ja3s
# Certificate extractiontshark -r capture.pcap -Y "ssl.handshake.certificate" \    -T fields -e x509sat.printableString

Decryption Approaches

bash
# Pre-master secret log (browser)export SSLKEYLOGFILE=/tmp/keys.log
# Configure Wireshark# Edit > Preferences > Protocols > TLS# (Pre)-Master-Secret log filename: /tmp/keys.log
# Decrypt with private key (if available)# Only works for RSA key exchange# Edit > Preferences > Protocols > TLS > RSA keys list

Custom Protocol Documentation

Protocol Specification Template

markdown
# Protocol Name Specification
## Overview
Brief description of protocol purpose and design.
## Transport
- Layer: TCP/UDP- Port: XXXX- Encryption: TLS 1.2+
## Message Format
### Header (12 bytes)
| Offset | Size | Field   | Description             || ------ | ---- | ------- | ----------------------- || 0      | 4    | Magic   | 0x50524F54 ("PROT")     || 4      | 2    | Version | Protocol version (1)    || 6      | 2    | Type    | Message type identifier || 8      | 4    | Length  | Payload length in bytes |
### Message Types
| Type | Name      | Description            || ---- | --------- | ---------------------- || 0x01 | HELLO     | Connection initiation  || 0x02 | HELLO_ACK | Connection accepted    || 0x03 | DATA      | Application data       || 0x04 | CLOSE     | Connection termination |
### Type 0x01: HELLO
| Offset | Size | Field      | Description              || ------ | ---- | ---------- | ------------------------ || 0      | 4    | ClientID   | Unique client identifier || 4      | 2    | Flags      | Connection flags         || 6      | var  | Extensions | TLV-encoded extensions   |
## State Machine

[INIT] --HELLO--> [WAIT_ACK] --HELLO_ACK--> [CONNECTED] | DATA/DATA | [CLOSED] <--CLOSE--+


## Examples### Connection Establishment

Client -> Server: HELLO (ClientID=0x12345678) Server -> Client: HELLO_ACK (Status=OK) Client -> Server: DATA (payload)


Wireshark Dissector (Lua)

lua
-- custom_protocol.lualocal proto = Proto("custom", "Custom Protocol")
-- Define fieldslocal f_magic = ProtoField.string("custom.magic", "Magic")local f_version = ProtoField.uint16("custom.version", "Version")local f_type = ProtoField.uint16("custom.type", "Type")local f_length = ProtoField.uint32("custom.length", "Length")local f_payload = ProtoField.bytes("custom.payload", "Payload")
proto.fields = { f_magic, f_version, f_type, f_length, f_payload }
-- Message type nameslocal msg_types = {    [0x01] = "HELLO",    [0x02] = "HELLO_ACK",    [0x03] = "DATA",    [0x04] = "CLOSE"}
function proto.dissector(buffer, pinfo, tree)    pinfo.cols.protocol = "CUSTOM"
    local subtree = tree:add(proto, buffer())
    -- Parse header    subtree:add(f_magic, buffer(0, 4))    subtree:add(f_version, buffer(4, 2))
    local msg_type = buffer(6, 2):uint()    subtree:add(f_type, buffer(6, 2)):append_text(        " (" .. (msg_types[msg_type] or "Unknown") .. ")"    )
    local length = buffer(8, 4):uint()    subtree:add(f_length, buffer(8, 4))
    if length > 0 then        subtree:add(f_payload, buffer(12, length))    endend
-- Register for TCP portlocal tcp_table = DissectorTable.get("tcp.port")tcp_table:add(8888, proto)

Active Testing

Fuzzing with Boofuzz

python
from boofuzz import *
def main():    session = Session(        target=Target(            connection=TCPSocketConnection("target", 8888)        )    )
    # Define protocol structure    s_initialize("HELLO")    s_static(b"\x50\x52\x4f\x54")  # Magic    s_word(1, name="version")       # Version    s_word(0x01, name="type")       # Type (HELLO)    s_size("payload", length=4)     # Length field    s_block_start("payload")    s_dword(0x12345678, name="client_id")    s_word(0, name="flags")    s_block_end()
    session.connect(s_get("HELLO"))    session.fuzz()
if __name__ == "__main__":    main()

Replay and Modification

python
from scapy.all import *
# Replay captured trafficpackets = rdpcap("capture.pcap")for pkt in packets:    if pkt.haslayer(TCP) and pkt[TCP].dport == 8888:        send(pkt)
# Modify and replayfor pkt in packets:    if pkt.haslayer(Raw):        # Modify payload        original = pkt[Raw].load        modified = original.replace(b"client", b"CLIENT")        pkt[Raw].load = modified        # Recalculate checksums        del pkt[IP].chksum        del pkt[TCP].chksum        send(pkt)

Best Practices

Analysis Workflow

  1. Capture traffic: Multiple sessions, different scenarios
  2. Identify boundaries: Message start/end markers
  3. Map structure: Fixed header, variable payload
  4. Identify fields: Compare multiple samples
  5. Document format: Create specification
  6. Validate understanding: Implement parser/generator
  7. Test edge cases: Fuzzing, boundary conditions

Common Patterns to Look For

  • Magic numbers/signatures at message start
  • Version fields for compatibility
  • Length fields (often before variable data)
  • Type/opcode fields for message identification
  • Sequence numbers for ordering
  • Checksums/CRCs for integrity
  • Timestamps for timing
  • Session/connection identifiers

來源與署名

來源:wshobson/agents位於plugins/reverse-engineering/skills/protocol-reverse-engineering提交46891e7

授權條款: 無授權條款

內容歸原作者所有。SourceWeft 從公開儲存庫中收錄這些內容。

檢舉或申請下架