CMDB系统一直使用json 格式作为API接口进行数据的收集和渲染。当数据量比较大时，使用http协议对json数据传输时，传输比较耗时。无意间发现google有一款开源软件protocol buffer，通过测试发现的确性能比json提高不少。
protobuf
Google Protocol Buffer( 简称 Protobuf) 是 Google 公司内部的混合语言数据标准，目前已经正在使用的有超过 48,162 种报文格式定义和超过 12,183 个 .proto 文件。他们用于 RPC 系统和持续数据存储系统。
Protocol Buffers 是一种轻便高效的结构化数据存储格式，可以用于结构化数据串行化，或者说序列化。它很适合做数据存储或 RPC 数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。目前提供了 C++、Java、Python 三种语言的 API。
protobuf 优点
> 跨平台、跨语言
> 向下兼容性好，易于扩展
> 自动生成解析代码
> 序列化后占用空间小
> 特别方便用于RPC和消息存储的场合
RPC通讯模型和ZeroRpc
RPC通讯模型：
一种通过网络从远程计算机程序服务，而不需要了解底层网络技术的协议。
RPC协议假定某些传输协议的存在，如TCP或UDP，为通信程序之间携带信息数据。在OSI网络通信模型中，RPC跨越了传输层和应用层。
ZeroRPC:
ZeroRPC 是基于zeromq、gevent和msgpack 开发的分布式RPC框架zerorpc-python。这个框架简单、易用。
protobuf 安装和配置
I. 下载源码包 wget http://protobuf.googlecode.com/files/protobuf-2.3.0.zip II.安装软件包 unzip protobuf-2.3.0.zip cd protobuf-2.3.0 ./configure make make install III. 编写hostinfo.proto 描述文件 package hostinfo; message HostInfo { required string factername = 1; optional string hostname = 2; optional string serverhardtype = 3; optional string servertag = 4; optional string servertype = 5; optional string gatewayIP = 6; optional string serverproduct = 7; optional Ram rams = 8; repeated cpuinfo cpus = 9; message cpuinfo { optional string cpuCoreNum = 1; optional string cpuBit = 2; optional string cpuClockSpeed = 3; optional string cpupynum = 4; optional string cpuName = 5; optional string band = 6; optional string model = 7; } message Ram { repeated memory mem = 1; } message memory { optional string Serial_Number = 1; optional string Manufacturer = 2; optional string Speed = 3; optional string Size = 4; } } VI. 用protoc生成访问代码 protoc hostinfo.proto --python_out=./ 生成访问代码如下(hostinfo_pb2.py):# Generated by the protocol buffer compiler. DO NOT EDIT!from google.protobuf import descriptorfrom google.protobuf import messagefrom google.protobuf import reflectionfrom google.protobuf import descriptor_pb2# @@protoc_insertion_point(imports)DESCRIPTOR = descriptor.FileDescriptor( name='hostinfo.proto', package='info', serialized_pb='\n\x0ehostinfo.proto\x12\x04info\"\xd3\x03\n\x08HostInfo\x12\x10\n\x08hostname\x18\x02 \x01(\t\x12\x16\n\x0eserverhardtype\x18\x03 \x01(\t\x12\x11\n\tservertag\x18\x04 \x01(\t\x12\x12\n\nservertype\x18\x05 \x01(\t\x12\x11\n\tgatewayIP\x18\x06 \x01(\t\x12\x15\n\rserverproduct\x18\x07 \x01(\t\x12 \n\x04rams\x18\x08 \x01(\x0b\x32\x12.info.HostInfo.Ram\x12$\n\x04\x63pus\x18\x01 \x03(\x0b\x32\x16.info.HostInfo.cpuinfo\x1a\x84\x01\n\x07\x63puinfo\x12\x12\n\ncpuCoreNum\x18\x01 \x01(\t\x12\x0e\n\x06\x63puBit\x18\x02 \x01(\t\x12\x15\n\rcpuClockSpeed\x18\x03 \x01(\t\x12\x10\n\x08\x63pupynum\x18\x04 \x01(\t\x12\x0f\n\x07\x63puName\x18\x05 \x01(\t\x12\x0c\n\x04\x62\x61nd\x18\x06 \x01(\t\x12\r\n\x05model\x18\x07 \x01(\t\x1a)\n\x03Ram\x12\"\n\x03mem\x18\x01 \x03(\x0b\x32\x15.info.HostInfo.memory\x1aR\n\x06memory\x12\x15\n\rSerial_Number\x18\x01 \x01(\t\x12\x14\n\x0cManufacturer\x18\x02 \x01(\t\x12\r\n\x05Speed\x18\x03 \x01(\t\x12\x0c\n\x04Size\x18\x04 \x01(\t')_HOSTINFO_CPUINFO = descriptor.Descriptor( name='cpuinfo', full_name='info.HostInfo.cpuinfo', filename=None, file=DESCRIPTOR, containing_type=None, fields=[ descriptor.FieldDescriptor( name='cpuCoreNum', full_name='info.HostInfo.cpuinfo.cpuCoreNum', index=0, number=1, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='cpuBit', full_name='info.HostInfo.cpuinfo.cpuBit', index=1, number=2, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='cpuClockSpeed', full_name='info.HostInfo.cpuinfo.cpuClockSpeed', index=2, number=3, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='cpupynum', full_name='info.HostInfo.cpuinfo.cpupynum', index=3, number=4, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='cpuName', full_name='info.HostInfo.cpuinfo.cpuName', index=4, number=5, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='band', full_name='info.HostInfo.cpuinfo.band', index=5, number=6, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='model', full_name='info.HostInfo.cpuinfo.model', index=6, number=7, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), ], extensions=[ ], nested_types=[], enum_types=[ ], options=None, is_extendable=False, extension_ranges=[], serialized_start=233, serialized_end=365,)_HOSTINFO_RAM = descriptor.Descriptor( name='Ram', full_name='info.HostInfo.Ram', filename=None, file=DESCRIPTOR, containing_type=None, fields=[ descriptor.FieldDescriptor( name='mem', full_name='info.HostInfo.Ram.mem', index=0, number=1, type=11, cpp_type=10, label=3, has_default_value=False, default_value=[], message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), ], extensions=[ ], nested_types=[], enum_types=[ ], options=None, is_extendable=False, extension_ranges=[], serialized_start=367, serialized_end=408,)_HOSTINFO_MEMORY = descriptor.Descriptor( name='memory', full_name='info.HostInfo.memory', filename=None, file=DESCRIPTOR, containing_type=None, fields=[ descriptor.FieldDescriptor( name='Serial_Number', full_name='info.HostInfo.memory.Serial_Number', index=0, number=1, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='Manufacturer', full_name='info.HostInfo.memory.Manufacturer', index=1, number=2, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='Speed', full_name='info.HostInfo.memory.Speed', index=2, number=3, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='Size', full_name='info.HostInfo.memory.Size', index=3, number=4, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), ], extensions=[ ], nested_types=[], enum_types=[ ], options=None, is_extendable=False, extension_ranges=[], serialized_start=410, serialized_end=492,)_HOSTINFO = descriptor.Descriptor( name='HostInfo', full_name='info.HostInfo', filename=None, file=DESCRIPTOR, containing_type=None, fields=[ descriptor.FieldDescriptor( name='hostname', full_name='info.HostInfo.hostname', index=0, number=2, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='serverhardtype', full_name='info.HostInfo.serverhardtype', index=1, number=3, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='servertag', full_name='info.HostInfo.servertag', index=2, number=4, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='servertype', full_name='info.HostInfo.servertype', index=3, number=5, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='gatewayIP', full_name='info.HostInfo.gatewayIP', index=4, number=6, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='serverproduct', full_name='info.HostInfo.serverproduct', index=5, number=7, type=9, cpp_type=9, label=1, has_default_value=False, default_value=unicode("", "utf-8"), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='rams', full_name='info.HostInfo.rams', index=6, number=8, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), descriptor.FieldDescriptor( name='cpus', full_name='info.HostInfo.cpus', index=7, number=1, type=11, cpp_type=10, label=3, has_default_value=False, default_value=[], message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None), ], extensions=[ ], nested_types=[_HOSTINFO_CPUINFO, _HOSTINFO_RAM, _HOSTINFO_MEMORY, ], enum_types=[ ], options=None, is_extendable=False, extension_ranges=[], serialized_start=25, serialized_end=492,)_HOSTINFO_CPUINFO.containing_type = _HOSTINFO;_HOSTINFO_RAM.fields_by_name['mem'].message_type = _HOSTINFO_MEMORY_HOSTINFO_RAM.containing_type = _HOSTINFO;_HOSTINFO_MEMORY.containing_type = _HOSTINFO;_HOSTINFO.fields_by_name['rams'].message_type = _HOSTINFO_RAM_HOSTINFO.fields_by_name['cpus'].message_type = _HOSTINFO_CPUINFODESCRIPTOR.message_types_by_name['HostInfo'] = _HOSTINFOclass HostInfo(message.Message): __metaclass__ = reflection.GeneratedProtocolMessageType class cpuinfo(message.Message): __metaclass__ = reflection.GeneratedProtocolMessageType DESCRIPTOR = _HOSTINFO_CPUINFO # @@protoc_insertion_point(class_scope:info.HostInfo.cpuinfo) class Ram(message.Message): __metaclass__ = reflection.GeneratedProtocolMessageType DESCRIPTOR = _HOSTINFO_RAM # @@protoc_insertion_point(class_scope:info.HostInfo.Ram) class memory(message.Message): __metaclass__ = reflection.GeneratedProtocolMessageType DESCRIPTOR = _HOSTINFO_MEMORY # @@protoc_insertion_point(class_scope:info.HostInfo.memory) DESCRIPTOR = _HOSTINFO # @@protoc_insertion_point(class_scope:info.HostInfo) # @@protoc_insertion_point(module_scope)V.编写应用代码(RPC client端)#!/usr/bin/env pythonimport hostinfo_pb2import zerorpcimport syshostinfo = hostinfo_pb2.HostInfo()def ServerInfo(hostinfo): '''这些信息可以通过脚本进行收集''' hostinfo.hostname = 'cdn.oss.david.com' hostinfo.serverhardtype = 'IBM System x3550 M2 -[7946I1M]-' hostinfo.servertag = '99G0665' hostinfo.servertype = 'physical' hostinfo.gatewayIP = '113.207.61.1' hostinfo.serverproduct = 'IBM' cpu = hostinfo.cpus.add() cpu.cpuCoreNum = '1' cpu.cpuBit = '64' cpu.cpuClockSpeed = '2.00GHz' cpu.cpupynum = '1' cpu.cpuName = 'Intel Xeon' cpu.band = 'Intel' cpu.model = 'E5504' memory = hostinfo.rams.mem.add() memory.Serial_Number = '5F787C83' memory.Manufacturer = 'Samsung' memory.Speed = '800 MHz' memory.Size = '4096 MB'if __name__ == "__main__": ServerInfo(hostinfo) c = zerorpc.Client() c.connect("tcp://127.0.0.1:4242") #连接RPC 服务端 print c.info(hostinfo.SerializeToString()) #向服务端发送数据 VI. protobuf 基础 字段约束： required：必须赋值字段，禁止为空 optional：可选字段，可以为空 repeated：集合，可以填充零到多个对象 5.Zero RPC安装和配置
I.Zero RPC 安装 yum -y install zeromq yum install gcc gcc-c++ libuuid-devel python-uuid uuid wget http://download.zeromq.org/zeromq-2.1.9.tar.gz ./configure make make installII. 安装gevent pip install gevent III. 安装zerorpc pip install zerorpcVI. 编写ZeroRPC ServerPRC 服务端：#!/usr/bin/env pythonimport zerorpcimport hostinfo_pb2 class HostRpc(object): def info(self, name): data = hostinfo_pb2.HostInfo() data.ParseFromString(name) print data.hostname print data.serverhardtype print data.servertag print data.gatewayIP print data.serverproduct return 'success's = zerorpc.Server(HostRpc())s.bind("tcp://0.0.0.0:4242")s.run() 启动RPC Server：
查看RPC 状态:
客户端向服务端传输数据：
客户端：
服务端：
通过以上服务端的输出信息可以看出，客户端可以成功将信息发送至服务端。这只是为测试效果，可以将传输过来的数据通过Mysqldb写入到数据库。