| 1 |
XML via UDP - expected data from hosts |
| 2 |
====================================== |
| 3 |
|
| 4 |
|
| 5 |
<<< Provisional, pjm2@kent.ac.uk >>> |
| 6 |
|
| 7 |
|
| 8 |
Purpose of this document |
| 9 |
------------------------ |
| 10 |
|
| 11 |
This document shall help to provide a separate party with |
| 12 |
the knowledge required to use their own implementation of |
| 13 |
a piece of host monitoring software. In particular, this |
| 14 |
document details the data that is to be expected from a |
| 15 |
host. This document is additional to |
| 16 |
http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt |
| 17 |
|
| 18 |
|
| 19 |
Background |
| 20 |
---------- |
| 21 |
|
| 22 |
Hosts are expected to periodically send UDP packets to the |
| 23 |
central monitoring system. Such packets may contain various |
| 24 |
pieces of information about the host, such as how much free |
| 25 |
memory is remaining on the host, etc. Some of this data is |
| 26 |
deemed to be 'essential', some is 'expected' and the rest |
| 27 |
is not necessary, but shall not be ignored by the central |
| 28 |
monitoring system server. |
| 29 |
|
| 30 |
|
| 31 |
Specification |
| 32 |
------------- |
| 33 |
|
| 34 |
The UDP packet must contain a complete and well-formed |
| 35 |
piece of XML markup, describing the data that the host is |
| 36 |
submitting. In the event of a piece of data containing |
| 37 |
an invalid character, such as a "<", it is the responsibilty |
| 38 |
of the host to format this in a valid manner, for example |
| 39 |
by replacing all occurrences of "<" with "<" before it is |
| 40 |
sent in a UDP packet to the central monitoring system server. |
| 41 |
|
| 42 |
Alternatively, the data may be wrapped up in special CDATA |
| 43 |
tags. For example, if we wish to send the following text: |
| 44 |
|
| 45 |
<>&!" |
| 46 |
|
| 47 |
surrounded by <test_chars> tags then we can safely send this |
| 48 |
by wrapping it up to produce the following XML snippet: |
| 49 |
|
| 50 |
<test_chars><![CDATA[<>&!"]]></test_chars> |
| 51 |
|
| 52 |
The bare minimum that a host should send is the following: - |
| 53 |
|
| 54 |
<packet> |
| 55 |
</packet> |
| 56 |
|
| 57 |
Note that every XML tag must also have a matching closing |
| 58 |
tag. |
| 59 |
|
| 60 |
However, such a packet is rather useless when it is |
| 61 |
received by the server - it does not provide any useful |
| 62 |
information. As such, a packet will be rejected by the |
| 63 |
server if it does not specify the machine_name, its |
| 64 |
I.P. address (ip), a packet sequence number (seq_no) |
| 65 |
and a date. Note that the date is a long integer |
| 66 |
representing the number of seconds since the epoch. |
| 67 |
All of these are expected to be specified as an attribute |
| 68 |
of the packet tag. If they are not specified here, then |
| 69 |
the packet will be rejected. |
| 70 |
|
| 71 |
Thus, to ensure that a packet has a chance of being |
| 72 |
acted upon, a host must specify at least four attribute |
| 73 |
values - machine_name, ip, date and seq_no. These must |
| 74 |
be formed in the style shown below. |
| 75 |
|
| 76 |
<packet machine_name="raptor" ip="aaa.bbb.ccc.ddd" |
| 77 |
date="93456245245" seq_no="1234567"> |
| 78 |
</packet> |
| 79 |
|
| 80 |
Note that the server may run additional 'plugin' tools |
| 81 |
that are designed to filter/reject packets under certain |
| 82 |
conditions. Please check with the server administrator |
| 83 |
if you wish to know about the specific configuration of |
| 84 |
your central monitoring system. |
| 85 |
|
| 86 |
The I.P. address and machine_name could be found from |
| 87 |
the UDP packet itself, but these remain essential so |
| 88 |
as to provide resiliance from stray packets that do not |
| 89 |
contain any valid or useful data. |
| 90 |
|
| 91 |
Remember that malformed XML data would be rejected by the |
| 92 |
central monitoring system immediately, without acting upon |
| 93 |
it. |
| 94 |
|
| 95 |
|
| 96 |
Classification of XML data in our system |
| 97 |
---------------------------------------- |
| 98 |
|
| 99 |
|
| 100 |
ESSENTIAL: - |
| 101 |
|
| 102 |
The following values are ESSENTIAL in each packet. Any |
| 103 |
packet without these will be rejected. They MUST be |
| 104 |
specified as attributes of the root node ("packet"): - |
| 105 |
|
| 106 |
ip |
| 107 |
seq_no |
| 108 |
date |
| 109 |
machine_name |
| 110 |
|
| 111 |
|
| 112 |
EXPECTED: - |
| 113 |
|
| 114 |
The following values are EXPECTED in each packet. All |
| 115 |
packets are RECOMMENDED to specify these values, as the |
| 116 |
server is designed to permanently store such data in an |
| 117 |
easy and quick to access manner. It is not essential to |
| 118 |
specify such values, but it does limit the usefulness of |
| 119 |
the central monitoring system if these values are not |
| 120 |
specified: - |
| 121 |
|
| 122 |
mac_address (attribute) |
| 123 |
os_name (attribute) |
| 124 |
os_version (attribute) |
| 125 |
cpus * |
| 126 |
load * |
| 127 |
memory * |
| 128 |
swap * |
| 129 |
disk * |
| 130 |
users (attribute) |
| 131 |
|
| 132 |
* - These parameters are specified with tags of the same |
| 133 |
name. They contain nested XML tags. See later in |
| 134 |
this document for more details about that! |
| 135 |
|
| 136 |
All of the values marked as being attributes above must be |
| 137 |
specified as an attribute of the root tag ("packet") |
| 138 |
|
| 139 |
|
| 140 |
OTHER: - |
| 141 |
|
| 142 |
Any other arbitrary values may be specified by the host. |
| 143 |
The only provision here is that they may not share the |
| 144 |
same name as another tag or attribute within the same |
| 145 |
heirarchical level. The server will be able to handle |
| 146 |
such clashes, however, it cannot be guaranteed which |
| 147 |
of the duplicate parameters would gain priority. |
| 148 |
|
| 149 |
|
| 150 |
Detail of a complete 'expected' XML packet via UDP |
| 151 |
-------------------------------------------------- |
| 152 |
|
| 153 |
I shall start by showing an example of a complete |
| 154 |
'expected' packet. I shall then explain the contents |
| 155 |
of it. |
| 156 |
|
| 157 |
Here is an example of an XML packet, containing all of |
| 158 |
the 'essential' and 'expected' data: - |
| 159 |
|
| 160 |
<packet ip="" seq_no="" date="" machine_name="" |
| 161 |
mac_address="" os_name="" os_version="" |
| 162 |
user=""> |
| 163 |
<cpus> |
| 164 |
<cpu0></cpu0> |
| 165 |
<cpu1></cpu1> |
| 166 |
<cpu2></cpu2> |
| 167 |
</cpus> |
| 168 |
<load> |
| 169 |
<load1><load1> |
| 170 |
<load5><load5> |
| 171 |
<load15><load15> |
| 172 |
</load> |
| 173 |
<memory> |
| 174 |
<free></free> |
| 175 |
<total></total> |
| 176 |
</memory> |
| 177 |
<swap> |
| 178 |
<free></free> |
| 179 |
<total></total> |
| 180 |
</swap> |
| 181 |
<disk> |
| 182 |
<p0> |
| 183 |
<free></free> |
| 184 |
<total></total> |
| 185 |
<mounted></mounted> |
| 186 |
<label></label> |
| 187 |
</p0> |
| 188 |
<p1> |
| 189 |
<free></free> |
| 190 |
<total></total> |
| 191 |
<mount></mount> |
| 192 |
<label></label> |
| 193 |
</p1> |
| 194 |
</disk> |
| 195 |
</packet> |
| 196 |
|
| 197 |
Note that linefeeds and spaces would normally not be present. |
| 198 |
These are only shown above for structure clarity. |
| 199 |
|
| 200 |
Note that all of the essential attributes have been specified |
| 201 |
as attributes of the root node. The expected attribute |
| 202 |
values are also attributes of the root node. |
| 203 |
|
| 204 |
The <cpus> tag begins a list of cpu load information. |
| 205 |
Each subtag herein must be of the form cpux, where x is |
| 206 |
the number of the processor in multiprocessor systems. |
| 207 |
Note that this numbering starts from zero. |
| 208 |
|
| 209 |
The <load> tag begins a list of system load information. |
| 210 |
The load tag is only expected to contain 3 subtags, named |
| 211 |
"load1", "load5" and "load15". These represent the average |
| 212 |
load 1 minute ago, 5 minutes ago and 15 minutes ago |
| 213 |
respectively. |
| 214 |
|
| 215 |
The <memory> tag begins a list of system memory information. |
| 216 |
This tag is only expected to contain two subtags, as shown |
| 217 |
in the example above. These two subtags specify how much |
| 218 |
memory the system has free and how much memory the system |
| 219 |
has in total. |
| 220 |
|
| 221 |
The <swap> tag is identical in nature to the <memory> tag. |
| 222 |
|
| 223 |
The <disk> tag begins a list of partition information. |
| 224 |
Each subtag must be of the form px, where x is the number |
| 225 |
of the disk partition that is unique for that particular |
| 226 |
system. Note that the numbering of px is expected to |
| 227 |
start from zero. Each <px> tag contains the following |
| 228 |
subtags: - |
| 229 |
<free> - the free partition space in bytes. |
| 230 |
<total> - the total partition size in bytes. |
| 231 |
<mount> - the mount point, eg, "/var". |
| 232 |
<label> - the label of the physical disk. |
| 233 |
|
| 234 |
The <disk_total> tag begins a list of total disk space, |
| 235 |
similar in implementation to the <disk_free> tag. |
| 236 |
|
| 237 |
Any other values and attributes may be specified in the |
| 238 |
packet, providing they are not specified in the same |
| 239 |
heirarchical level as another attribute or value with |
| 240 |
the same name. |
| 241 |
|
| 242 |
|
| 243 |
Flexibility and Minimising bandwidth |
| 244 |
------------------------------------ |
| 245 |
|
| 246 |
The following issues still stand: - |
| 247 |
|
| 248 |
The means of submitting host data via UDP containing XML |
| 249 |
markup is provided so that future customisation is easily |
| 250 |
possible. It would be possible to easily tailor a custom |
| 251 |
piece of host monitoring software to provide exactly what |
| 252 |
data is desired for adequate monitoring. |
| 253 |
|
| 254 |
Some of the XML markup demonstrated above contains a lot |
| 255 |
of rendundant features. For example, it is not necessary |
| 256 |
to lay the contents out neatly (although this certainly |
| 257 |
helps visualise the contents). |
| 258 |
|
| 259 |
The amount of data sent within each UDP packet may be (in |
| 260 |
some cases, vastly) reduced by using some of the ideas |
| 261 |
described below: - |
| 262 |
|
| 263 |
- Remove unnecessary linefeeds and 'white space' |
| 264 |
|
| 265 |
- If a single piece of data is to be represented, it |
| 266 |
will usually occupy less space if it is stored as |
| 267 |
an attribute to a tag, rather than within a pair |
| 268 |
of tags. |
| 269 |
|
| 270 |
- Comments within the XML may be useful for testing |
| 271 |
purposes, however, the server ignores all comments |
| 272 |
so these can be removed to reduce packet sizes. |
| 273 |
|
| 274 |
Taking the above into account, here is an example of some |
| 275 |
ideally formed XML to be contained within a UDP packet: - |
| 276 |
|
| 277 |
<packet machine_name="raptor" ip="aaa.bbb.ccc.ddd" |
| 278 |
><freespace><drive1>23677</drive1><drive2>23534</d |
| 279 |
rive2><drive3>10390</drive3></freespace></packet> |
| 280 |
|
| 281 |
Notice how all unnecessary 'white space' and linefeeds have |
| 282 |
been removed. The comment has also been removed. Values |
| 283 |
such as "machine_name" and "ip" have both been stored as an |
| 284 |
attribute of the root node ("packet") as this results in |
| 285 |
a smaller packet size. |
| 286 |
|
| 287 |
|
| 288 |
About |
| 289 |
----- |
| 290 |
|
| 291 |
This document was written by Paul Mutton [pjm2@kent.ac.uk] |
| 292 |
for use by the team working on a 3rd year Computer Science |
| 293 |
project called "i-scream". More details can be found on the |
| 294 |
project website - http://www.i-scream.org |
| 295 |
|
| 296 |
|
