ViewVC Help
View File | Revision Log | Show Annotations | Revision Graph | Root Listing
root/i-scream/projects/cms/documentation/specification/expected_data.txt
Revision: 1.7
Committed: Sun Aug 1 10:39:55 2004 UTC (19 years, 8 months ago) by tdb
Content type: text/plain
Branch: MAIN
CVS Tags: HEAD
Changes since 1.6: +4 -4 lines
Log Message:
Catch a lot of old URL's and update them. Also remove a couple of old files
that aren't used.

File Contents

# Content
1 XML via UDP - expected data from hosts
2 ======================================
3
4
5 <<< Provisional, pjm2@kent.ac.uk >>>
6
7
8 Purpose of this document
9 ------------------------
10
11 This document shall help to provide a separate party with
12 the knowledge required to use their own implementation of
13 a piece of host monitoring software. In particular, this
14 document details the data that is to be expected from a
15 host. This document is additional to
16 http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt
17
18
19 Background
20 ----------
21
22 Hosts are expected to periodically send UDP packets to the
23 central monitoring system. Such packets may contain various
24 pieces of information about the host, such as how much free
25 memory is remaining on the host, etc. Some of this data is
26 deemed to be 'essential', some is 'expected' and the rest
27 is not necessary, but shall not be ignored by the central
28 monitoring system server.
29
30
31 Specification
32 -------------
33
34 The UDP packet must contain a complete and well-formed
35 piece of XML markup, describing the data that the host is
36 submitting. In the event of a piece of data containing
37 an invalid character, such as a "<", it is the responsibilty
38 of the host to format this in a valid manner, for example
39 by replacing all occurrences of "<" with "&lt" before it is
40 sent in a UDP packet to the central monitoring system server.
41
42 Alternatively, the data may be wrapped up in special CDATA
43 tags. For example, if we wish to send the following text:
44
45 <>&!"
46
47 surrounded by <test_chars> tags then we can safely send this
48 by wrapping it up to produce the following XML snippet:
49
50 <test_chars><![CDATA[<>&!"]]></test_chars>
51
52 The bare minimum that a host should send is the following: -
53
54 <packet>
55 </packet>
56
57 Note that every XML tag must also have a matching closing
58 tag.
59
60 However, such a packet is rather useless when it is
61 received by the server - it does not provide any useful
62 information. As such, a packet will be rejected by the
63 server if it does not specify the machine_name, its
64 I.P. address (ip), a packet sequence number (seq_no)
65 and a date. Note that the date is a long integer
66 representing the number of seconds since the epoch.
67 All of these are expected to be specified as an attribute
68 of the packet tag. If they are not specified here, then
69 the packet will be rejected.
70
71 Thus, to ensure that a packet has a chance of being
72 acted upon, a host must specify at least four attribute
73 values - machine_name, ip, date and seq_no. These must
74 be formed in the style shown below.
75
76 <packet machine_name="raptor" ip="aaa.bbb.ccc.ddd"
77 date="93456245245" seq_no="1234567">
78 </packet>
79
80 Note that the server may run additional 'plugin' tools
81 that are designed to filter/reject packets under certain
82 conditions. Please check with the server administrator
83 if you wish to know about the specific configuration of
84 your central monitoring system.
85
86 The I.P. address and machine_name could be found from
87 the UDP packet itself, but these remain essential so
88 as to provide resiliance from stray packets that do not
89 contain any valid or useful data.
90
91 Remember that malformed XML data would be rejected by the
92 central monitoring system immediately, without acting upon
93 it.
94
95
96 Classification of XML data in our system
97 ----------------------------------------
98
99
100 ESSENTIAL: -
101
102 The following values are ESSENTIAL in each packet. Any
103 packet without these will be rejected. They MUST be
104 specified as attributes of the root node ("packet"): -
105
106 ip
107 seq_no
108 date
109 machine_name
110
111
112 EXPECTED: -
113
114 The following values are EXPECTED in each packet. All
115 packets are RECOMMENDED to specify these values, as the
116 server is designed to permanently store such data in an
117 easy and quick to access manner. It is not essential to
118 specify such values, but it does limit the usefulness of
119 the central monitoring system if these values are not
120 specified: -
121
122 mac_address (attribute)
123 os_name (attribute)
124 os_version (attribute)
125 cpus *
126 load *
127 memory *
128 swap *
129 disk *
130 users (attribute)
131
132 * - These parameters are specified with tags of the same
133 name. They contain nested XML tags. See later in
134 this document for more details about that!
135
136 All of the values marked as being attributes above must be
137 specified as an attribute of the root tag ("packet")
138
139
140 OTHER: -
141
142 Any other arbitrary values may be specified by the host.
143 The only provision here is that they may not share the
144 same name as another tag or attribute within the same
145 heirarchical level. The server will be able to handle
146 such clashes, however, it cannot be guaranteed which
147 of the duplicate parameters would gain priority.
148
149
150 Detail of a complete 'expected' XML packet via UDP
151 --------------------------------------------------
152
153 I shall start by showing an example of a complete
154 'expected' packet. I shall then explain the contents
155 of it.
156
157 Here is an example of an XML packet, containing all of
158 the 'essential' and 'expected' data: -
159
160 <packet ip="" seq_no="" date="" machine_name=""
161 mac_address="" os_name="" os_version=""
162 user="">
163 <cpus>
164 <cpu0></cpu0>
165 <cpu1></cpu1>
166 <cpu2></cpu2>
167 </cpus>
168 <load>
169 <load1><load1>
170 <load5><load5>
171 <load15><load15>
172 </load>
173 <memory>
174 <free></free>
175 <total></total>
176 </memory>
177 <swap>
178 <free></free>
179 <total></total>
180 </swap>
181 <disk>
182 <p0>
183 <free></free>
184 <total></total>
185 <mounted></mounted>
186 <label></label>
187 </p0>
188 <p1>
189 <free></free>
190 <total></total>
191 <mount></mount>
192 <label></label>
193 </p1>
194 </disk>
195 </packet>
196
197 Note that linefeeds and spaces would normally not be present.
198 These are only shown above for structure clarity.
199
200 Note that all of the essential attributes have been specified
201 as attributes of the root node. The expected attribute
202 values are also attributes of the root node.
203
204 The <cpus> tag begins a list of cpu load information.
205 Each subtag herein must be of the form cpux, where x is
206 the number of the processor in multiprocessor systems.
207 Note that this numbering starts from zero.
208
209 The <load> tag begins a list of system load information.
210 The load tag is only expected to contain 3 subtags, named
211 "load1", "load5" and "load15". These represent the average
212 load 1 minute ago, 5 minutes ago and 15 minutes ago
213 respectively.
214
215 The <memory> tag begins a list of system memory information.
216 This tag is only expected to contain two subtags, as shown
217 in the example above. These two subtags specify how much
218 memory the system has free and how much memory the system
219 has in total.
220
221 The <swap> tag is identical in nature to the <memory> tag.
222
223 The <disk> tag begins a list of partition information.
224 Each subtag must be of the form px, where x is the number
225 of the disk partition that is unique for that particular
226 system. Note that the numbering of px is expected to
227 start from zero. Each <px> tag contains the following
228 subtags: -
229 <free> - the free partition space in bytes.
230 <total> - the total partition size in bytes.
231 <mount> - the mount point, eg, "/var".
232 <label> - the label of the physical disk.
233
234 The <disk_total> tag begins a list of total disk space,
235 similar in implementation to the <disk_free> tag.
236
237 Any other values and attributes may be specified in the
238 packet, providing they are not specified in the same
239 heirarchical level as another attribute or value with
240 the same name.
241
242
243 Flexibility and Minimising bandwidth
244 ------------------------------------
245
246 The following issues still stand: -
247
248 The means of submitting host data via UDP containing XML
249 markup is provided so that future customisation is easily
250 possible. It would be possible to easily tailor a custom
251 piece of host monitoring software to provide exactly what
252 data is desired for adequate monitoring.
253
254 Some of the XML markup demonstrated above contains a lot
255 of rendundant features. For example, it is not necessary
256 to lay the contents out neatly (although this certainly
257 helps visualise the contents).
258
259 The amount of data sent within each UDP packet may be (in
260 some cases, vastly) reduced by using some of the ideas
261 described below: -
262
263 - Remove unnecessary linefeeds and 'white space'
264
265 - If a single piece of data is to be represented, it
266 will usually occupy less space if it is stored as
267 an attribute to a tag, rather than within a pair
268 of tags.
269
270 - Comments within the XML may be useful for testing
271 purposes, however, the server ignores all comments
272 so these can be removed to reduce packet sizes.
273
274 Taking the above into account, here is an example of some
275 ideally formed XML to be contained within a UDP packet: -
276
277 <packet machine_name="raptor" ip="aaa.bbb.ccc.ddd"
278 ><freespace><drive1>23677</drive1><drive2>23534</d
279 rive2><drive3>10390</drive3></freespace></packet>
280
281 Notice how all unnecessary 'white space' and linefeeds have
282 been removed. The comment has also been removed. Values
283 such as "machine_name" and "ip" have both been stored as an
284 attribute of the root node ("packet") as this results in
285 a smaller packet size.
286
287
288 About
289 -----
290
291 This document was written by Paul Mutton [pjm2@kent.ac.uk]
292 for use by the team working on a 3rd year Computer Science
293 project called "i-scream". More details can be found on the
294 project website - http://www.i-scream.org
295
296