HttpTunnel工作原理及源程序分析

2012年01月13日    点击数: 16908    字体:           一键关注汇讯

         这份报告对开源工程HttpTunnel的工作原理及源程序运行流程进行分析,重点分析HttpTunnel对HTTP协议的封装与实现。

1 工作原理概述

    HttpTunnel通过HTTP请求方式,提供了一个双向的、虚拟的数据通路,可以通过http proxy来使用(不是必需)。

    HttpTunnel编译连接之后得到两个可执行的文件,hts和htc,hts是服务器端程序,用于要连接的外部主机之上,htc是客户端程序,用于本地主机之上。这两个部分连接用于产生一个虚拟的数据通道(tunnel)。

    Http Tunnel利用HTPP的POST与GET两个命令建立了两个连接,分别用于客户端向服务器发送和接收数据,而且考虑到了HTTP数据的合法性,会随时检查所接收或发出的数据是否超过content-length规定的长度,如果是则填充完后重新开连接进行处理新的数据,保证了数据始终是合法的HTTP数据,完全等价于客户端在同时用HTTP协议在上传和下载一个大文件,文件数据中不需要任何的HTTP命令或HTML语言标记。

2 HttpTunnel程序流程分析

2.1 服务器端程序流程分析

hts是服务器端,安装在外部网络一侧,也就是没有防火墙的一端。

hts源文件包括hts.c, common.c, tunnel.c, http.c这几个文件和port目录下的库。

流程如下:

main() (hts.c)

         parse_arguments (argc, argv, &arg),解析命令行参数;

         调用tunnel = tunnel_new_server (arg.port, arg.content_length)创建新的tunnel的服务端;

       初始化tunnel, in_fd, out_fd都为-1

       在tunnel->server_socket监听server_socket (tunnel->dest.host_port, 1), backlog=1, 注意用了SO_REUSEADDR选项;                                              àcommon.c

         通道的一些选项:strict_content_length, keep_alive, max_connection_age, 这些在HTTP头说明中会用到;  

         写PID文件

         进入无限循环

如果定义输入输出的终端设备(arg.device),打开设备fd = open_device (arg.device),common.c;

       tunnel_accept接受外部连接                                          àtunnel.c

       accept()连接

      解析HTTP协议请求数据: http_parse_request (s, &request),                   http.c

       POST或PUT命令, 将此socket设置为tunnel的in_fd, 用于接收客户端的数据;

       GET命令, 将此socket设置为tunnel的out_fd, 用于发送去往客户端的数据;

       如果设置了转发端口(arg.forward_port != -1), 调用do_connect连接到这个端口,描述符为fd, 注意这个连接是本机的

      自己连自己的内部连接, hts起到代理转发作用, 目标端口也就是真正的被HTTP包裹的协议端口;

        进入下一个循环

      poll: fd 和tunnel->server_socket, 也就是网络通道数据和内部连接数据对倒;

        handle_input()

       handle_device_input()处理fd输入信息;                                common.c

       handle_input()

       handle_tunnel_input()处理来自通道的信息;                            common.c

服务器端接收了客户端的两个连接, POST命令对应的连接服务器接收数据, GET命令对应的连接服务器发送数据。

2.2 客户端程序流程分析

htc是客户端,安装在防火墙内部网络一侧,也即客户浏览器一端。

htc源文件包括htc.c, common.c, tunnel.c, http.c, base64.c这几个文件和port目录下的库程序

流程如下:

main.c (htc.c)
         parse_arguments (argc, argv, &arg); 解析命令行参数,                        àhtc.c

如果转发端口(arg.forward_port != -1),在此端口打开socket监听s = server_socket

(arg.forward_port, 0),                            

      backlog=0, 其实不收连接, 注意用了SO_REUSEADDR选项,               àcommon.c
         进入无限循环for (;;);

       定义输入输出的终端设备(arg.device),打开设备fd = open_device (arg.device),common.c;   

否则如果定义了转发端口(arg.forward_port != -1),输入输出通过连接的socket来进行fd = wait_for_connection_on_socket (s),就是accept() ,                               à htc.c

打开一个新的通道  tunnel = tunnel_new_client (arg.host_name, arg.host_port,

      arg.proxy_name, arg.proxy_port,

      arg.content_length); tunnel.c

       注意tunnel结构中的in_fd, out_fd都初始化为-1,表示没有连接, 缺省的content_length是100K字节.          

设置通道的一些选项:strict_content_length, keep_alive, max_connection_age, user_agent这些在HTTP头说明中会用到, 支持代理;              

如果要进行代理认证(arg.proxy_authorization != NULL),将认证参数进行base64编码,作为通道的proxy_authorization参数;

通道连接对方tunnel_connect (tunnel), tunnel.c, 建立http tunnel,主要是调用函数tunnel_write_request (tunnel, TUNNEL_OPEN, auth_data, sizeof auth_data)

 如果要写入和已经写入的数据长度超过content_length, 对tuenel进行填充;

       对于客户端已经连接好的tunnel,超时时进行断开;

       对于断开(或第一次连接)的客户端, 调用tunnel_out_connect (tunnel)发起连接

       调用do_connect()连接服务器(第一个连接), socket描述符为tunnel->out_fd.

       设置该socket的一些选项;

       调用shutdown(out_fd, 0), 不接收数据;

       调用http_post()函数向服务器发送HTTP的POST命令; http.c      

调用tunnel_write_data (tunnel, &request, sizeof request)函数向服务器写要执行的请求(此时为TUNNEL_OPEN)

       继续写data部分,先写长度,然后是数据(此时为dummy的auth_data=42, length=1),(即HTTP的POST命令向服务器写TUNNEL_OPEN命令和一个dummy数据),然后进行数据长度判断是否在此连接中数据写多了.

      进入函数tunnel_in_connect (tunnel) , 数据进入的连接

调用do_connect()连接服务器(第2个连接),socket描述符为tunnel->in_fd.

设置该socket的一些选项;

      调用http_get()函数向服务器发送HTTP的GET命令;                      à http.c

      调用shutdown(in_fd, 1), 不再写数据;
      调用http_parse_response (tunnel->in_fd, &response)解析HTTP服务器返回数据;

      处理统计信息

       此时tunnel_connect完成

       进入下一个循环

      poll选择用户的输入设备fd和网络tunnel的in_fd(第2条连接)数据,也就是用户的输入信息和服务器返回的信息进行对倒:

       fd输入->tunnel->out_fd输出; tunnel->in_fd输入->fd输出.

      handle_input()

       handle_device_input()处理用户输入信息,                             àcommon.c

       handle_input()

       handle_tunnel_input()处理来自通道的信息,                           àcommon.c

客户端一共向服务器发起了两个连接, 第一个连接用于发送数据,第2个连接用于接收数据

3 HttpTunnel源码相关分析

3.1 HTTP头的封装与实现

3.1.1 HTTP头结构

在HttpTunnel中,将HTTP头实现为名值对。是一个递归的结构。

typedef struct http_header Http_header;

struct http_header

{

const char *name;

const char *value;

Http_header *next; /* FIXME: this is ugly; need cons cell. */

};

3.1.2 创建HTTP

static inline Http_header *

http_alloc_header (const char *name, const char *value)

{

Http_header *header;

header = malloc (sizeof (Http_header));

if (header == NULL)

     return NULL;

header->name = header->value = NULL;

header->name = strdup (name);

header->value = strdup (value);

if (name == NULL || value == NULL)

{

     if (name == NULL)

              free ((char *)name);

     if (value == NULL)

              free ((char *)value);

     free (header);

     return NULL;

}

return header;

}

 

3.1.3 添加HTTP

Http_header *

http_add_header (Http_header **header, const char *name, const char *value)

{

     Http_header *new_header;

     new_header = http_alloc_header (name, value);

     if (new_header == NULL)

              return NULL;

     new_header->next = NULL;

     while (*header)

              header = &(*header)->next;

     *header = new_header;

     return new_header;

}

3.1.4 解析HTTP

static ssize_t

parse_header (int fd, Http_header **header)

{

     unsigned char buf[2];

     unsigned char *data;

     Http_header *h;

     size_t len;

     ssize_t n;

 

     *header = NULL;

     n = read_all (fd, buf, 2);

     if (n <= 0)

              return n;

     if (buf[0] == ' ' && buf[1] == ' ')

              return n;

 

     h = malloc (sizeof (Http_header));

     if (h == NULL)

     {

              log_error ("parse_header: malloc failed");

              return -1;

     }

     *header = h;

     h->name = NULL;

     h->value = NULL;

     n = read_until (fd, ':', &data);

     if (n <= 0)

              return n;

     data = realloc (data, n + 2);

     if (data == NULL)

     {

              log_error ("parse_header: realloc failed");

              return -1;

     }

     memmove (data + 2, data, n);

     memcpy (data, buf, 2);

     n += 2;

     data[n - 1] = 0;

     h->name = data;

     len = n;

     n = read_until (fd, ' ', &data);

     if (n <= 0)

              return n;

     data[n - 1] = 0;

     h->value = data;

     len += n;

     n = read_until (fd, ' ', &data);

     if (n <= 0)

              return n;

     free (data);

     if (n != 1)

     {

              log_error ("parse_header: invalid line ending");

              return -1;

     }

     len += n;

     log_verbose ("parse_header: %s:%s", h->name, h->value);

     n = parse_header (fd, &h->next);

     if (n <= 0)

              return n;

     len += n;

     return len;

}

3.1.5 HTTP

static ssize_t

http_write_header (int fd, Http_header *header)

{

     ssize_t n = 0, m;

     if (header == NULL)

              return write_all (fd, " ", 2);

     m = write_all (fd, (void *)header->name, strlen (header->name));

     if (m == -1)

     {

              return -1;

     }

     n += m;

     m = write_all (fd, ": ", 2);

     if (m == -1)

     {

              return -1;

     }

     n += m;

     m = write_all (fd, (void *)header->value, strlen (header->value));

     if (m == -1)

     {

              return -1;

     }

     n += m;

     m = write_all (fd, " ", 2);

     if (m == -1)

     {

              return -1;

     }

     n += m;

     m = http_write_header (fd, header->next);

     if (m == -1)

     {

              return -1;

     }

     n += m;

     return n;

}

3.1.6 查找HTTP

static Http_header *

http_header_find (Http_header *header, const char *name)

{

     if (header == NULL)

              return NULL;

     if (strcmp (header->name, name) == 0)

              return header;

     return http_header_find (header->next, name);

}

3.1.7 获取HTTP

const char *

http_header_get (Http_header *header, const char *name)

{

     Http_header *h;

     h = http_header_find (header, name);

     if (h == NULL)

              return NULL;

     return h->value;

}

3.1.8 销毁HTTP

static void

http_destroy_header (Http_header *header)

{

     if (header == NULL)

              return;

     http_destroy_header (header->next);

     if (header->name)

              free ((char *)header->name);

     if (header->value)

              free ((char *)header->value);

     free (header);

}

3.2 HTTP方法的封装与实现

3.2.1 HTTP方法简介

OPTIONS :

GET :

HEAD :

POST :

PUT :

DELETE:

TRACE :

CONNECT:

3.2.2 HTTP方法枚举

在程序中,定义以下的枚举类型。

typedef enum

{

 HTTP_GET,

 HTTP_PUT,

 HTTP_POST,

 HTTP_OPTIONS,

 HTTP_HEAD,

 HTTP_DELETE,

 HTTP_TRACE

} Http_method;

 

3.2.3 HTTP方法的通用实现

在HttpTunnel里面,首先定义了一个通用的实现HTTP方法的函数,即。

static inline ssize_t

http_method (int fd, Http_destination *dest,

                        Http_method method, ssize_t length)

{

     char str[1024]; /* FIXME: possible buffer overflow */

     Http_request *request;

     ssize_t n;

     if (fd == -1)

     {

              log_error ("http_method: fd == -1");

              return -1;

     }

     n = 0;

     if (dest->proxy_name != NULL)

              n = sprintf (str, "http://%s:%d", dest->host_name, dest->host_port);

     sprintf (str + n, "/index.html?crap=%ld", time (NULL));

     request = http_create_request (method, str, 1, 1);

     if (request == NULL)

              return -1;

     sprintf (str, "%s:%d", dest->host_name, dest->host_port);

     http_add_header (&request->header, "Host", str);

     if (length >= 0)

     {

              sprintf (str, "%d", length);

              http_add_header (&request->header, "Content-Length", str);

     }

     http_add_header (&request->header, "Connection", "close");

     if (dest->proxy_authorization)

     {

              http_add_header (&request->header,

                       "Proxy-Authorization",

                       dest->proxy_authorization);

     }

     if (dest->user_agent)

     {

              http_add_header (&request->header,

                       "User-Agent",

                       dest->user_agent);

     }

     n = http_write_request (fd, request);

     http_destroy_request (request);

     return n;

}

该函数工作流程分析如下:

3.2.4 GET方法的实现

ssize_t

http_get (int fd, Http_destination *dest)

{

     return http_method (fd, dest, HTTP_GET, -1);

}

3.2.5 POST方法的实现

http_post (int fd, Http_destination *dest, size_t length)

{

     return http_method (fd, dest, HTTP_POST, (ssize_t)length);

}

3.3 HTTP请求的封装与实现

3.3.1 HTTP请求结构

typedef struct

{

 Http_method method;

 const char *uri;

 int major_version;

 int minor_version;

 Http_header *header;

} Http_request;       

3.3.2 HTTP请求分配内存空间

static inline Http_request *

http_allocate_request (const char *uri)

{

     Http_request *request;

 

     request = malloc (sizeof (Http_request));

     if (request == NULL)

              return NULL;

 

     request->uri = strdup (uri);

     if (request->uri == NULL)

     {

              free (request);

              return NULL;

     }

 

     return request;

}

3.3.3 创建HTTP请求

Http_request *

http_create_request (Http_method method,

                                           const char *uri,

                                           int major_version,

                                           int minor_version)

{

     Http_request *request;

 

     request = http_allocate_request (uri);

     if (request == NULL)

              return NULL;

 

     request->method = method;

     request->major_version = major_version;

     request->minor_version = minor_version;

     request->header = NULL;

 

     return request;

}

3.3.4 解析HTTP请求

ssize_t

http_parse_request (int fd, Http_request **request_)

{

     Http_request *request;

     unsigned char *data;

     size_t len;

     ssize_t n;

 

     *request_ = NULL;

 

     request = malloc (sizeof (Http_request));

     if (request == NULL)

     {

              log_error ("http_parse_request: out of memory");

              return -1;

     }

 

     request->method = -1;

     request->uri = NULL;

     request->major_version = -1;

     request->minor_version = -1;

     request->header = NULL;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              free (request);

              return n;

     }

     request->method = http_string_to_method (data, n - 1);

     if (request->method == -1)

     {

              log_error ("http_parse_request: expected an HTTP method");

              free (data);

              free (request);

              return -1;

     }

     data[n - 1] = 0;

     log_verbose ("http_parse_request: method = "%s"", data);

     free (data);

     len = n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              free (request);

              return n;

     }

     data[n - 1] = 0;

     request->uri = data;

     len += n;

     log_verbose ("http_parse_request: uri = "%s"", request->uri);

 

     n = read_until (fd, '/', &data);

     if (n <= 0)

     {

              http_destroy_request (request);

              return n;

     }

     else if (n != 5 || memcmp (data, "HTTP", 4) != 0)

     {

              log_error ("http_parse_request: expected "HTTP"");

              free (data);

              http_destroy_request (request);

              return -1;

     }

     free (data);

     len = n;

 

     n = read_until (fd, '.', &data);

     if (n <= 0)

     {

              http_destroy_request (request);

              return n;

     }

     data[n - 1] = 0;

     request->major_version = atoi (data);

     log_verbose ("http_parse_request: major version = %d",

              request->major_version);

     free (data);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              http_destroy_request (request);

              return n;

     }

     data[n - 1] = 0;

     request->minor_version = atoi (data);

     log_verbose ("http_parse_request: minor version = %d",

              request->minor_version);

     free (data);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              http_destroy_request (request);

              return n;

     }

     free (data);

     if (n != 1)

     {

              log_error ("http_parse_request: invalid line ending");

              http_destroy_request (request);

              return -1;

     }

     len += n;

     n = parse_header (fd, &request->header);

     if (n <= 0)

     {

              http_destroy_request (request);

              return n;

     }

     len += n;

     *request_ = request;

     return len;

}

3.3.5 销毁HTTP请求

void

http_destroy_request (Http_request *request)

{

     if (request->uri)

              free ((char *)request->uri);

     http_destroy_header (request->header);

     free (request);

}

3.4 HTTP响应的封装与实现

3.4.1 HTTP响应结构

typedef struct

{

   int major_version;

   int minor_version;

   int status_code;

   const char *status_message;

   Http_header *header;

} Http_response;

3.4.2 HTTP响应分配内存空间

static inline Http_response *

http_allocate_response (const char *status_message)

{

     Http_response *response;

 

     response = malloc (sizeof (Http_response));

     if (response == NULL)

              return NULL;

 

     response->status_message = strdup (status_message);

     if (response->status_message == NULL)

     {

              free (response);

              return NULL;

     }

 

     return response;

}

3.4.3 创建HTTP响应

Http_response *

http_create_response (int major_version,

                                           int minor_version,

                                           int status_code,

                                           const char *status_message)

{

     Http_response *response;

 

     response = http_allocate_response (status_message);

     if (response == NULL)

              return NULL;

 

     response->major_version = major_version;

     response->minor_version = minor_version;

     response->status_code = status_code;

     response->header = NULL;

 

     return response;

}

3.4.4 解析HTTP响应

ssize_t

http_parse_response (int fd, Http_response **response_)

{

     Http_response *response;

     unsigned char *data;

     size_t len;

     ssize_t n;

 

     *response_ = NULL;

 

     response = malloc (sizeof (Http_response));

     if (response == NULL)

     {

              log_error ("http_parse_response: out of memory");

              return -1;

     }

 

     response->major_version = -1;

     response->minor_version = -1;

     response->status_code = -1;

     response->status_message = NULL;

     response->header = NULL;

 

     n = read_until (fd, '/', &data);

     if (n <= 0)

     {

              free (response);

              return n;

     }

     else if (n != 5 || memcmp (data, "HTTP", 4) != 0)

     {

              log_error ("http_parse_response: expected "HTTP"");

              free (data);

              free (response);

              return -1;

     }

     free (data);

     len = n;

 

     n = read_until (fd, '.', &data);

     if (n <= 0)

     {

              free (response);

              return n;

     }

     data[n - 1] = 0;

     response->major_version = atoi (data);

     log_verbose ("http_parse_response: major version = %d",

              response->major_version);

     free (data);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              free (response);

              return n;

     }

     data[n - 1] = 0;

     response->minor_version = atoi (data);

     log_verbose ("http_parse_response: minor version = %d",

              response->minor_version);

     free (data);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              free (response);

              return n;

     }

     data[n - 1] = 0;

     response->status_code = atoi (data);

     log_verbose ("http_parse_response: status code = %d",

              response->status_code);

     free (data);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              free (response);

              return n;

     }

     data[n - 1] = 0;

     response->status_message = data;

     log_verbose ("http_parse_response: status message = "%s"",

              response->status_message);

     len += n;

 

     n = read_until (fd, ' ', &data);

     if (n <= 0)

     {

              http_destroy_response (response);

              return n;

     }

     free (data);

     if (n != 1)

     {

              log_error ("http_parse_request: invalid line ending");

              http_destroy_response (response);

              return -1;

     }

     len += n;

 

     n = parse_header (fd, &response->header);

     if (n <= 0)

     {

              http_destroy_response (response);

              return n;

     }

     len += n;

 

     *response_ = response;

     return len;

}

3.5 HTTP目标的封装

typedef struct

{

 const char *host_name;

 int host_port;

 const char *proxy_name;

 int proxy_port;

 const char *proxy_authorization;

 const char *user_agent;

} Http_destination;

还有一些其他相关函数可以参考源程序。

上一篇:http tunnel 原理 及 穿透防火墙方法

下一篇:Kad的出现 结束了edonkey时代

Copyright © 2007-2021 汇讯Wiseuc. 粤ICP备10013541号    
展开