documentation/specification/protocols.txt

i-scream Protocols
==================

Contents
--------
 - Conventions

 - Host Connection Protocol
 - Host Heartbeat Protocol
 - Host Data Protocol

 - Client Control Protocol
 - Client Data Protocol

Conventions
-----------
All protocols in this document assume a valid connection of
the appropriate type has been made, and that data streams
are already available.  All strings should, and will be,
terminated with a \n (newline character) to indicate the end
of the message.  All messages are sent as ASCII Strings.

Fixed protocol messages will appear in [] brackets, if there
is a variety of options they will be separated with the |
character.  For example:

    [OK|ERROR]
  
Indicates that the message "OK" OR the message "ERROR" will
be sent depending on the result of the last action.

Data messages will appear in {} brackets, where the name in
the brackets indicates the type (as in kind, not data type)
of data that will be returned.  For example:

    {lastmodified}

Indicates that data for use as 'last modified' will be
returned.  

If [ERROR] is sent back at any time, this indicates that all
communication is over. EXCEPT where otherwise specified!

Host Connection Protocol
------------------------
The initial connection of a host to the i-scream server is
through the FilterManager.  A Host gets its configuration
and then gets assigned a Filter to connect to and start
sending data.

The port number of the FilterManager is fully configurable,
however the default at this time is 4567.

Host         (direction) Server         Comment
------------------------------------------------------------
[STARTCONFIG]---------->                Requests to start
                                        receiving config
                                        information
           
             <---------- [OK|ERROR]     If the server ok's
                                        the request or not

[LASTMODIFIED]---------->               Asks when the config
                                        was last modified
                                        (used when checking
                                        if the config has
                                        changed)
           
             <---------- [{lastmodified}Returns a long int
                         |ERROR]        time since epoch
                                        eg, 123456789
                                        To indicate what
                                        format this is in,
                                        I quote from the
                                        Java 1.3 JDK API
                                        "measured in milli
                                        -seconds since the
                                        epoch"
                                       
[FILELIST]   ---------->                Asks the server for
                                        the list of files
                                        that were used to
                                        build the config
                                        (used when checking
                                        if the config has
                                        changed)
      
             <---------- [{filelist}    Returns a semi-colon
                         |ERROR]        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf

[FQDN]       ---------->                Asks the server to
                                        send the FQDN (fully
                                        qualified domain
                                        name) of the
                                        connecting host.
                                        
             <---------- [{fqdn}        Returns the FQDN of
                         |ERROR]        the connected host,
                                        or error if it can't
                                        be looked up.

             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{property}] ---------->                Sends the name of a
                                        property to be
                                        retrieved from the
                                        config file
                                        eg, UDPUpdateTime
           
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested
                                        property it returns
                                        ERROR to indicate
                                        that fact, but
                                        communication still
                                        proceeds
                                      
             ********** LOOP UNTIL **********
             The loop continues until the host
             sends the following message
           
[ENDCONFIG]  ---------->                Indicates that the
                                        host requires no
                                        more config
           
             ********** LOOP  END **********
             Communication continues
           
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed
                                      
[FILTER]     ---------->                Asks the server to
                                        send it the host
                                        information of a
                                        filter that it
                                        should connect to
                                      
             <---------- {filter}|ERROR Returns a semi-colon
                                        separated list of
                                        FQDN hostname, UDP
                                        port number and TCP
                                        port number that a
                                        configured Filter is
                                        running on and
                                        assigned to the
                                        calling host
                                        eg,
                                        raptor.ukc.ac.uk;
                                        1234;5678
                                        or ERROR if none of
                                        configured Filter's
                                        could be found.

[END]        ---------->                Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
           
             <---------- [OK|ERROR]     Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
------------------------------------------------------------
      
Host Heartbeat Protocol
-----------------------
When a host is configured after it has connected it should
obtain a property TCPUpdateTime.  This indicates how often
a host should send a "Heart Beat", which is a pro-active
connection to the server by the host to indicate that it is
still alive.  This "Heart Beat" also allows a host to see
its configuration has changed and act accordingly.  In a
well written host this should just be a case of dropping out
of a loop and heading back to the start (connecting to the
filter manager).

Host            (direction) Server      Comment
------------------------------------------------------------
[HEARTBEAT]     ---------->             Starts the
                                        heartbeat protocol
                                        off
                                      
                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[CONFIG]        ---------->             Indicates that the
                                        host wants to check
                                        its config

                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[{filelist}]    ---------->             Send a semi-colon
                                        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol

                <---------- [OK]        Indicates that the
                                        server is ok

[{lastmodified}]---------->             Returns a long int
                                        time since epoch
                                        eg, 123456789
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol
                                        To indicate what
                                        format this is in,
                                        I quote from the
                                        Java 1.3 JDK API
                                        "measured in milli
                                        -seconds since the
                                        epoch"

                <---------- [OK|ERROR]  The server then
                                        checks all the files
                                        in the file list to
                                        see if they have
                                        been modified more
                                        recently than the
                                        lastmodified value.
                                        If they HAVE that
                                        indicates that the
                                        configuration has
                                        changed and the host
                                        should re-configure,
                                        thus it sends ERROR.
                                        If the files remain
                                        unchanged the server
                                        will return OK to
                                        indicate the host
                                        should continue as
                                        before

[ENDHEARTBEAT]  ---------->             Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
              
                <---------- [OK|ERROR]  Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
------------------------------------------------------------

Host Data Protocol
------------------
The UDP data packets that are sent by the host contain
simply XML data.  For information on the format of this
data see the XML via UDP specification document at:

http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt

Client Control Protocol
-----------------------

The client control protocol channel is opened by the client
and allows a variety of actions to be carried out by the
client at anytime.  Unlike previous protocols, this is NOT
sequential, all of the requests can be carried out in
any order.

All client protocols are backwards compatible, and the
version is shown by the protocol identifier.

There are three sections to this protocol.

1) Initialise (sent only at start)
2) Send command(s) - unlimited number in any order
3) Disconnect (sent only at end)

If at anytime the client sends something the server does
not understand, an [ERROR] will be sent

All are indicated with <> brackets.

v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
       Supported commands:
            STARTCONFIG
            STARTDATA
            STOPDATA

Host         (direction) Server         Comment
------------------------------------------------------------
<initialise>
             <---------- [{protocol}]   The server sends the
                                        protocol identifier
                                       
[{name}]     ---------->                The client sends its
                                        "name".  This name
                                        is used to identify
                                        the type of client
                                        to the system and
                                        obtain its config
                                        eg, Conient
            
             <---------- [OK]           Indicates the server
                                        is ok to proceed
                       

<command #1> (allows client to obtain its configuration)
[STARTCONFIG]---------->                Tell the server we
                                        want to start this
                                        command
            
             <---------- [OK]           Indicates the server
                                        is ok to proceed
            
             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{config};{property}] ---------->       Sends the name of a
                                        configuration and
                                        property to be
                                        retrieved from the
                                        config file
                                        Clients can obtain
                                        host information
                                        eg,
                                        Host.UDPUpdateTime
                                        Otherwise it must
                                        prefix requests
                                        with "Client."
                                        All other requests
                                        will be ignored as
                                        if it was unable to
                                        retrieve the
                                        property
                                       
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested
                                        property it returns
                                        ERROR to indicate
                                        that fact
                                       
             ********** LOOP UNTIL **********
             The loop continues until the client
             sends the following message
            
[ENDCONFIG]  ---------->                Indicates that the
                                        client requires no
                                        more config
            
             ********** LOOP  END **********
             Communication continues
            
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed

<command #2> (tells the server to start the data link)
[STARTDATA]  ---------->                Tell the server we
                                        want to start this
                                        command
           
             <---------- [{portnumber}  The server then sets
                         |ERROR]        itself listening for
                                        a connection on its
                                        data link socket for
                                        this client.  It
                                        returns the port no.
                                        that it is listening
                                        on eg, 12367
                                        If the data link is
                                        already started the
                                        server will return
                                        ERROR
            
             <---------- [OK]           Indicates the client 
                                        has successfully 
                                        connected to the 
                                        data socket.

<command #3> (tells the server to stop the data link)
[STOPDATA]   ---------->                Tell the server we
                                        want to start this
                                        command
                                        The server then
                                        shuts down the
                                        data link to the
                                        client

             <---------- [OK|ERROR]     Returns OK is the
                                        server is ready to
                                        proceed, or ERROR
                                        if the data link
                                        was not open in the
                                        first place

<disconnect>
[DISCONNECT] ---------->                Tells the server the
                                        client wants to
                                        close the control
                                        link
                                       
             <---------- [OK]           The last word from
                                        the server, it will
                                        disappear after 
                                        this, and close the 
                                        data link if 
                                        required
------------------------------------------------------------

v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
       Supported commands:
            SETHOSTLIST

Host         (direction) Server         Comment
------------------------------------------------------------
<command #4> (indicates to the server which hosts the client
              wants data from)
[SETHOSTLIST]---------->                Tell the server we
                                        want to start this
                                        command
            
             <---------- [OK|ERROR]     The server will
                                        return an ERROR if
                                        the data link is
                                        open, as a host list
                                        must ONLY be set
                                        if the data link is
                                        closed
                                        If the server is
                                        ok to proceed with
                                        the command it says
                                        [OK]

[{hostlist}] ---------->                This is a semi-colon
                                        seperated list of
                                        FQDN hostnames that
                                        the client wishes to
                                        recieve data from
                                        eg,
        raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
                                       
                                        If the list is sent
                                        blank (or if no list
                                        is set at all) then
                                        data from ALL hosts
                                        will be sent to the
                                        client (this is the 
                                        default if no 
                                        SETHOSTLIST is not 
                                        run by the client
                                       
             <---------- [OK]           Indicates the server
                                        is ok to proceed and
                                        the host list has
                                        been accepted
------------------------------------------------------------

Client Data Protocol
--------------------
Once the data link has been opened by the control link, the
server will send XML formatted data packets separated by the
\n (newline) character.  For information on the format of
this data see the XML via UDP specification document at:

http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt

About
-----
This document was written by AJ Moore for use by the team working on a
3rd year Computer Science project called "i-scream". More details can
be found on the project website;

http://www.i-scream.org
Revision:	1.12
Committed:	Wed Jul 16 14:54:47 2008 UTC (15 years, 9 months ago) by tdb
Content type:	text/plain
Branch:	MAIN
CVS Tags:	HEAD
Changes since 1.11:	+3 -4 lines
Log Message:	Remove AJ's email address.
#	Content
1	i-scream Protocols
2	==================
3
4	Contents
5	--------
6	- Conventions
7
8	- Host Connection Protocol
9	- Host Heartbeat Protocol
10	- Host Data Protocol
11
12	- Client Control Protocol
13	- Client Data Protocol
14
15	Conventions
16	-----------
17	All protocols in this document assume a valid connection of
18	the appropriate type has been made, and that data streams
19	are already available. All strings should, and will be,
20	terminated with a \n (newline character) to indicate the end
21	of the message. All messages are sent as ASCII Strings.
22
23	Fixed protocol messages will appear in [] brackets, if there
24	is a variety of options they will be separated with the \|
25	character. For example:
26
27	[OK\|ERROR]
28
29	Indicates that the message "OK" OR the message "ERROR" will
30	be sent depending on the result of the last action.
31
32	Data messages will appear in {} brackets, where the name in
33	the brackets indicates the type (as in kind, not data type)
34	of data that will be returned. For example:
35
36	{lastmodified}
37
38	Indicates that data for use as 'last modified' will be
39	returned.
40
41	If [ERROR] is sent back at any time, this indicates that all
42	communication is over. EXCEPT where otherwise specified!
43
44	Host Connection Protocol
45	------------------------
46	The initial connection of a host to the i-scream server is
47	through the FilterManager. A Host gets its configuration
48	and then gets assigned a Filter to connect to and start
49	sending data.
50
51	The port number of the FilterManager is fully configurable,
52	however the default at this time is 4567.
53
54	Host (direction) Server Comment
55	------------------------------------------------------------
56	[STARTCONFIG]----------> Requests to start
57	receiving config
58	information
59
60	<---------- [OK\|ERROR] If the server ok's
61	the request or not
62
63	[LASTMODIFIED]----------> Asks when the config
64	was last modified
65	(used when checking
66	if the config has
67	changed)
68
69	<---------- [{lastmodified}Returns a long int
70	\|ERROR] time since epoch
71	eg, 123456789
72	To indicate what
73	format this is in,
74	I quote from the
75	Java 1.3 JDK API
76	"measured in milli
77	-seconds since the
78	epoch"
79
80	[FILELIST] ----------> Asks the server for
81	the list of files
82	that were used to
83	build the config
84	(used when checking
85	if the config has
86	changed)
87
88	<---------- [{filelist} Returns a semi-colon
89	\|ERROR] separated list of
90	filenames
91	eg,
92	a.conf;b.conf;c.conf
93
94	[FQDN] ----------> Asks the server to
95	send the FQDN (fully
96	qualified domain
97	name) of the
98	connecting host.
99
100	<---------- [{fqdn} Returns the FQDN of
101	\|ERROR] the connected host,
102	or error if it can't
103	be looked up.
104
105	******** LOOP START ********
106	This loop reads configuration
107	properties from the config
108
109	[{property}] ----------> Sends the name of a
110	property to be
111	retrieved from the
112	config file
113	eg, UDPUpdateTime
114
115	<---------- [{value}\|ERROR]Returns the value of
116	the requested config
117	property
118	eg, 120
119	If it is unable to
120	find the requested
121	property it returns
122	ERROR to indicate
123	that fact, but
124	communication still
125	proceeds
126
127	******** LOOP UNTIL ********
128	The loop continues until the host
129	sends the following message
130
131	[ENDCONFIG] ----------> Indicates that the
132	host requires no
133	more config
134
135	******** LOOP END ********
136	Communication continues
137
138	<---------- [OK] Indicates that the
139	server is ready to
140	proceed
141
142	[FILTER] ----------> Asks the server to
143	send it the host
144	information of a
145	filter that it
146	should connect to
147
148	<---------- {filter}\|ERROR Returns a semi-colon
149	separated list of
150	FQDN hostname, UDP
151	port number and TCP
152	port number that a
153	configured Filter is
154	running on and
155	assigned to the
156	calling host
157	eg,
158	raptor.ukc.ac.uk;
159	1234;5678
160	or ERROR if none of
161	configured Filter's
162	could be found.
163
164	[END] ----------> Indicates to the
165	server that the host
166	has finished an will
167	disconnect
168
169	<---------- [OK\|ERROR] Indicates that the
170	server is either ok
171	or that it thought
172	there was an error
173	------------------------------------------------------------
174
175	Host Heartbeat Protocol
176	-----------------------
177	When a host is configured after it has connected it should
178	obtain a property TCPUpdateTime. This indicates how often
179	a host should send a "Heart Beat", which is a pro-active
180	connection to the server by the host to indicate that it is
181	still alive. This "Heart Beat" also allows a host to see
182	its configuration has changed and act accordingly. In a
183	well written host this should just be a case of dropping out
184	of a loop and heading back to the start (connecting to the
185	filter manager).
186
187	Host (direction) Server Comment
188	------------------------------------------------------------
189	[HEARTBEAT] ----------> Starts the
190	heartbeat protocol
191	off
192
193	<---------- [OK\|ERROR] Indicates that the
194	server is ok or not
195
196	[CONFIG] ----------> Indicates that the
197	host wants to check
198	its config
199
200	<---------- [OK\|ERROR] Indicates that the
201	server is ok or not
202
203	[{filelist}] ----------> Send a semi-colon
204	separated list of
205	filenames
206	eg,
207	a.conf;b.conf;c.conf
208	This should be
209	identical to the one
210	received in the
211	connection protocol
212
213	<---------- [OK] Indicates that the
214	server is ok
215
216	[{lastmodified}]----------> Returns a long int
217	time since epoch
218	eg, 123456789
219	This should be
220	identical to the one
221	received in the
222	connection protocol
223	To indicate what
224	format this is in,
225	I quote from the
226	Java 1.3 JDK API
227	"measured in milli
228	-seconds since the
229	epoch"
230
231	<---------- [OK\|ERROR] The server then
232	checks all the files
233	in the file list to
234	see if they have
235	been modified more
236	recently than the
237	lastmodified value.
238	If they HAVE that
239	indicates that the
240	configuration has
241	changed and the host
242	should re-configure,
243	thus it sends ERROR.
244	If the files remain
245	unchanged the server
246	will return OK to
247	indicate the host
248	should continue as
249	before
250
251	[ENDHEARTBEAT] ----------> Indicates to the
252	server that the host
253	has finished an will
254	disconnect
255
256	<---------- [OK\|ERROR] Indicates that the
257	server is either ok
258	or that it thought
259	there was an error
260	------------------------------------------------------------
261
262	Host Data Protocol
263	------------------
264	The UDP data packets that are sent by the host contain
265	simply XML data. For information on the format of this
266	data see the XML via UDP specification document at:
267
268	http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt
269
270	Client Control Protocol
271	-----------------------
272
273	The client control protocol channel is opened by the client
274	and allows a variety of actions to be carried out by the
275	client at anytime. Unlike previous protocols, this is NOT
276	sequential, all of the requests can be carried out in
277	any order.
278
279	All client protocols are backwards compatible, and the
280	version is shown by the protocol identifier.
281
282	There are three sections to this protocol.
283
284	1) Initialise (sent only at start)
285	2) Send command(s) - unlimited number in any order
286	3) Disconnect (sent only at end)
287
288	If at anytime the client sends something the server does
289	not understand, an [ERROR] will be sent
290
291	All are indicated with <> brackets.
292
293	v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
294	Supported commands:
295	STARTCONFIG
296	STARTDATA
297	STOPDATA
298
299	Host (direction) Server Comment
300	------------------------------------------------------------
301	<initialise>
302	<---------- [{protocol}] The server sends the
303	protocol identifier
304
305	[{name}] ----------> The client sends its
306	"name". This name
307	is used to identify
308	the type of client
309	to the system and
310	obtain its config
311	eg, Conient
312
313	<---------- [OK] Indicates the server
314	is ok to proceed
315
316
317	<command #1> (allows client to obtain its configuration)
318	[STARTCONFIG]----------> Tell the server we
319	want to start this
320	command
321
322	<---------- [OK] Indicates the server
323	is ok to proceed
324
325	******** LOOP START ********
326	This loop reads configuration
327	properties from the config
328
329	[{config};{property}] ----------> Sends the name of a
330	configuration and
331	property to be
332	retrieved from the
333	config file
334	Clients can obtain
335	host information
336	eg,
337	Host.UDPUpdateTime
338	Otherwise it must
339	prefix requests
340	with "Client."
341	All other requests
342	will be ignored as
343	if it was unable to
344	retrieve the
345	property
346
347	<---------- [{value}\|ERROR]Returns the value of
348	the requested config
349	property
350	eg, 120
351	If it is unable to
352	find the requested
353	property it returns
354	ERROR to indicate
355	that fact
356
357	******** LOOP UNTIL ********
358	The loop continues until the client
359	sends the following message
360
361	[ENDCONFIG] ----------> Indicates that the
362	client requires no
363	more config
364
365	******** LOOP END ********
366	Communication continues
367
368	<---------- [OK] Indicates that the
369	server is ready to
370	proceed
371
372	<command #2> (tells the server to start the data link)
373	[STARTDATA] ----------> Tell the server we
374	want to start this
375	command
376
377	<---------- [{portnumber} The server then sets
378	\|ERROR] itself listening for
379	a connection on its
380	data link socket for
381	this client. It
382	returns the port no.
383	that it is listening
384	on eg, 12367
385	If the data link is
386	already started the
387	server will return
388	ERROR
389
390	<---------- [OK] Indicates the client
391	has successfully
392	connected to the
393	data socket.
394
395	<command #3> (tells the server to stop the data link)
396	[STOPDATA] ----------> Tell the server we
397	want to start this
398	command
399	The server then
400	shuts down the
401	data link to the
402	client
403
404	<---------- [OK\|ERROR] Returns OK is the
405	server is ready to
406	proceed, or ERROR
407	if the data link
408	was not open in the
409	first place
410
411	<disconnect>
412	[DISCONNECT] ----------> Tells the server the
413	client wants to
414	close the control
415	link
416
417	<---------- [OK] The last word from
418	the server, it will
419	disappear after
420	this, and close the
421	data link if
422	required
423	------------------------------------------------------------
424
425	v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
426	Supported commands:
427	SETHOSTLIST
428
429	Host (direction) Server Comment
430	------------------------------------------------------------
431	<command #4> (indicates to the server which hosts the client
432	wants data from)
433	[SETHOSTLIST]----------> Tell the server we
434	want to start this
435	command
436
437	<---------- [OK\|ERROR] The server will
438	return an ERROR if
439	the data link is
440	open, as a host list
441	must ONLY be set
442	if the data link is
443	closed
444	If the server is
445	ok to proceed with
446	the command it says
447	[OK]
448
449	[{hostlist}] ----------> This is a semi-colon
450	seperated list of
451	FQDN hostnames that
452	the client wishes to
453	recieve data from
454	eg,
455	raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
456
457	If the list is sent
458	blank (or if no list
459	is set at all) then
460	data from ALL hosts
461	will be sent to the
462	client (this is the
463	default if no
464	SETHOSTLIST is not
465	run by the client
466
467	<---------- [OK] Indicates the server
468	is ok to proceed and
469	the host list has
470	been accepted
471	------------------------------------------------------------
472
473	Client Data Protocol
474	--------------------
475	Once the data link has been opened by the control link, the
476	server will send XML formatted data packets separated by the
477	\n (newline) character. For information on the format of
478	this data see the XML via UDP specification document at:
479
480	http://www.i-scream.org/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt
481
482	About
483	-----
484	This document was written by AJ Moore for use by the team working on a
485	3rd year Computer Science project called "i-scream". More details can
486	be found on the project website;
487
488	http://www.i-scream.org