documentation/specification/protocols.txt

i-scream Protocols
==================

Contents
--------
 - Conventions

 - Host Connection Protocol
 - Host Heartbeat Protocol
 - Host Data Protocol

 - Client Control Protocol
 - Client Data Protocol


Conventions
-----------
All protocols in this document assume a valid connection of
the appropriate type has been made, and that data streams
are already available.  All strings should, and will be,
terminated with a \n (newline character) to indicate the end
of the message.  All messages are sent as ASCII Strings.

Fixed protocol messages will appear in [] brackets, if there
is a variety of options they will be separated with the |
character.  For example:

    [OK|ERROR]
   
Indicates that the message "OK" OR the message "ERROR" will
be sent depending on the result of the last action.

Data messages will appear in {} brackets, where the name in
the brackets indicates the type (as in kind, not data type)
of data that will be returned.  For example:

    {lastmodified}

Indicates that data for use as 'last modified' will be 
returned.   

If [ERROR] is sent back at any time, this indicates that all 
communication is over. EXCEPT where otherwise specified!


Host Connection Protocol
------------------------
The initial connection of a host to the i-scream server is
through the FilterManager.  A Host gets its configuration
and then gets assigned a Filter to connect to and start
sending data.

The port number of the FilterManager is fully configurable,
however the default at this time is 4567.

Host         (direction) Server         Comment
------------------------------------------------------------
[STARTCONFIG]---------->                Requests to start
                                        receiving config
                                        information
            
             <---------- [OK|ERROR]     If the server ok's
                                        the request or not

[LASTMODIFIED]---------->               Asks when the config
                                        was last modified
                                        (used when checking
                                        if the config has
                                        changed)
            
             <---------- [{lastmodified}Returns a long int
                         |ERROR]        time since epoch
                                        eg, 123456789

[FILELIST]   ---------->                Asks the server for
                                        the list of files
                                        that were used to
                                        build the config
                                        (used when checking
                                        if the config has
                                        changed)
       
             <---------- [{filelist}    Returns a semi-colon
                         |ERROR]        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf

             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{property}] ---------->                Sends the name of a
                                        property to be
                                        retrieved from the
                                        config file
                                        eg, UDPUpdateTime
            
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested
                                        property it returns
                                        ERROR to indicate
                                        that fact, but 
                                        communication still 
                                        proceeds
                                       
             ********** LOOP UNTIL **********
             The loop continues until the host
             sends the following message
            
[ENDCONFIG]  ---------->                Indicates that the
                                        host requires no
                                        more config
            
             ********** LOOP  END **********
             Communication continues
            
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed
                                       
[FILTER]     ---------->                Asks the server to
                                        send it the host
                                        information of a
                                        filter that it
                                        should connect to
                                       
             <---------- {filter}       Returns a semi-colon
                                        separated list of
                                        FQDN hostname, UDP
                                        port number and TCP
                                        port number that a
                                        configured Filter is
                                        running on and
                                        assigned to the
                                        calling host
                                        eg,
                                        raptor.ukc.ac.uk;
                                        1234;5678

[END]        ---------->                Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
            
             <---------- [OK|ERROR]     Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
<<<<<<< protocols.txt
------------------------------------------------------------

=======
       
>>>>>>> 1.2
Host Heartbeat Protocol
-----------------------
When a host is configured after it has connected it should
obtain a property TCPUpdateTime.  This indicates how often
a host should send a "Heart Beat", which is a pro-active
connection to the server by the host to indicate that it is
still alive.  This "Heart Beat" also allows a host to see
its configuration has changed and act accordingly.  In a
well written host this should just be a case of dropping out
of a loop and heading back to the start (connecting to the
filter manager).

Host            (direction) Server      Comment
------------------------------------------------------------
[HEARTBEAT]     ---------->             Starts the
                                        heartbeat protocol
                                        off
                                       
                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[CONFIG]        ---------->             Indicates that the
                                        host wants to check
                                        its config

                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[{filelist}]    ---------->             Send a semi-colon
                                        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol

                <---------- [OK]        Indicates that the
                                        server is ok

[{lastmodified}]---------->             Returns a long int
                                        time since epoch
                                        eg, 123456789
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol

                <---------- [OK|ERROR]  The server then
                                        checks all the files
                                        in the file list to
                                        see if they have
                                        been modified more
                                        recently than the
                                        lastmodified value.
                                        If they HAVE that
                                        indicates that the
                                        configuration has
                                        changed and the host
                                        should re-configure,
                                        thus it sends ERROR.
                                        If the files remain
                                        unchanged the server
                                        will return OK to
                                        indicate the host
                                        should continue as
                                        before

[ENDHEARTBEAT]  ---------->             Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
               
                <---------- [OK|ERROR]  Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
------------------------------------------------------------

Host Data Protocol
------------------
The UDP data packets that are sent by the host contain
simply XML data.  For information on the format of this
data see the XML via UDP specification document at:

http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt

Client Control Protocol
-----------------------

The client control protocol channel is opened by the client
and allows a variety of actions to be carried out by the
client at anytime.  Unlike previous protocols, this is NOT
sequential, all of the requests can be carried out in 
any order.

All client protocols are backwards compatible, and the
version is shown by the protocol identifier.

There are three sections to this protocol.

1) Initialise (sent only at start)
2) Send command(s) - unlimited number in any order
3) Disconnect (sent only at end)

If at anytime the client sends something the server does
not understand, an [ERROR] will be sent

All are indicated with <> brackets.

v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
       Supported commands:
            STARTCONFIG
            STARTDATA
            STOPDATA

Host         (direction) Server         Comment
------------------------------------------------------------
<initialise>
[CONNECT]    ---------->                Starts off the 
                                        client protocol
             
             <---------- [{protocol}]   The server sends the
                                        protocol identifier
                                        
[{name}]     ---------->                The client sends its
                                        "name".  This name
                                        is used to identify
                                        the type of client
                                        to the system and
                                        obtain its config
                                        eg, Conient
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed
                        

<command #1> (allows client to obtain its configuration)
[STARTCONFIG]---------->                Tell the server we
                                        want to start this
                                        command
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed
             
             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{property}] ---------->                Sends the name of a
                                        property to be
                                        retrieved from the
                                        config file
                                        Clients can obtain
                                        host information
                                        eg, 
                                        Host.UDPUpdateTime
                                        Otherwise it must
                                        prefix requests
                                        with "Client."
                                        All other requests
                                        will be ignored as
                                        if it was unable to
                                        retieve the property
                                        
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested 
                                        property it returns
                                        ERROR to indicate
                                        that fact
                                        
             ********** LOOP UNTIL **********
             The loop continues until the client
             sends the following message
             
[ENDCONFIG]  ---------->                Indicates that the 
                                        client requires no
                                        more config
             
             ********** LOOP  END **********
             Communication continues
             
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed

<command #2> (tells the server to start the data link)
[STARTDATA]  ---------->                Tell the server we
                                        want to start this
                                        command
            
             <---------- [{portnumber}  The server then sets
                         |ERROR]        itself listening for
                                        a connection on its
                                        data link socket for
                                        this client.  It
                                        returns the port no.
                                        that it is listening
                                        on eg, 12367
                                        If the data link is
                                        yet to be started
                                        the server will
                                        return ERROR
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed and
                                        the client can 
                                        then connect its
                                        data link

<command #3> (tells the server to stop the data link)
[STOPDATA]   ---------->                Tell the server we
                                        want to start this
                                        command
                                        The server then
                                        shuts down the 
                                        data link to the
                                        client

             <---------- [OK|ERROR]     Returns OK is the
                                        server is ready to
                                        proceed, or ERROR
                                        if the data link
                                        was not open in the
                                        first place

<disconnect>
[DISCONNECT] ---------->                Tells the server the
                                        client wants to
                                        close the control
                                        link
                                        
             <---------- [OK]           The last word from
                                        the server, it will
                                        disappear after this
------------------------------------------------------------

v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
       Supported commands:
            SETHOSTLIST

Host         (direction) Server         Comment
------------------------------------------------------------
<command #4> (indicates to the server which hosts the client
              wants data from)
[SETHOSTLIST]---------->                Tell the server we
                                        want to start this
                                        command
             
             <---------- [OK|ERROR]     The server will
                                        return an ERROR if
                                        the data link is
                                        open, as a host list
                                        must ONLY be set 
                                        if the data link is
                                        closed
                                        If the server is
                                        ok to proceed with
                                        the command it says
                                        [OK]

[{hostlist}] ---------->                This is a semi-colon
                                        seperated list of
                                        FQDN hostnames that
                                        the client wishes to
                                        recieve data from
                                        eg,
        raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
                                        
                                        If the list is sent
                                        blank (or if no list
                                        is set at all) then
                                        data from ALL hosts
                                        will be sent to the
                                        client
                                        
             <---------- [OK]           Indicates the server
                                        is ok to proceed and
                                        the host list has 
                                        been accepted
------------------------------------------------------------

Client Data Protocol
--------------------
Once the data link has been opened by the control link, the
server will send XML formated data packets seperated by the 
\n (newline) character.  For information on the format of 
this data see the XML via UDP specification document at:

http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt

About
-----
This document was written by AJ Moore [ajm4@ukc.ac.uk] for
use by the team working on a 3rd year Computer Science
project called "i-scream". More details can be found on the
project website;

http://www.i-scream.org.uk
Revision:	1.4
Committed:	Sun Jan 28 20:16:19 2001 UTC (23 years, 9 months ago) by ajm
Content type:	text/plain
Branch:	MAIN
Changes since 1.3:	+2 -4 lines
Log Message:	changed links as they didn't work
#	User	Rev	Content
1	ajm	1.1	i-scream Protocols
2			==================
3
4			Contents
5			--------
6			- Conventions
7
8			- Host Connection Protocol
9			- Host Heartbeat Protocol
10			- Host Data Protocol
11
12			- Client Control Protocol
13			- Client Data Protocol
14
15	tdb	1.2
16	ajm	1.1	Conventions
17			-----------
18			All protocols in this document assume a valid connection of
19			the appropriate type has been made, and that data streams
20	tdb	1.2	are already available. All strings should, and will be,
21	ajm	1.1	terminated with a \n (newline character) to indicate the end
22			of the message. All messages are sent as ASCII Strings.
23
24			Fixed protocol messages will appear in [] brackets, if there
25	tdb	1.2	is a variety of options they will be separated with the \|
26	ajm	1.1	character. For example:
27
28			[OK\|ERROR]
29	tdb	1.2
30	ajm	1.1	Indicates that the message "OK" OR the message "ERROR" will
31			be sent depending on the result of the last action.
32
33			Data messages will appear in {} brackets, where the name in
34			the brackets indicates the type (as in kind, not data type)
35			of data that will be returned. For example:
36
37			{lastmodified}
38
39	tdb	1.2	Indicates that data for use as 'last modified' will be
40			returned.
41
42			If [ERROR] is sent back at any time, this indicates that all
43			communication is over. EXCEPT where otherwise specified!
44	ajm	1.1
45
46			Host Connection Protocol
47			------------------------
48			The initial connection of a host to the i-scream server is
49			through the FilterManager. A Host gets its configuration
50	tdb	1.2	and then gets assigned a Filter to connect to and start
51	ajm	1.1	sending data.
52
53	tdb	1.2	The port number of the FilterManager is fully configurable,
54	ajm	1.1	however the default at this time is 4567.
55
56			Host (direction) Server Comment
57			------------------------------------------------------------
58			[STARTCONFIG]----------> Requests to start
59			receiving config
60			information
61	tdb	1.2
62	ajm	1.1	<---------- [OK\|ERROR] If the server ok's
63			the request or not
64
65			[LASTMODIFIED]----------> Asks when the config
66			was last modified
67			(used when checking
68	tdb	1.2	if the config has
69	ajm	1.1	changed)
70	tdb	1.2
71	ajm	1.1	<---------- [{lastmodified}Returns a long int
72	tdb	1.2	\|ERROR] time since epoch
73	ajm	1.1	eg, 123456789
74
75			[FILELIST] ----------> Asks the server for
76			the list of files
77			that were used to
78			build the config
79			(used when checking
80			if the config has
81			changed)
82	tdb	1.2
83	ajm	1.1	<---------- [{filelist} Returns a semi-colon
84	tdb	1.2	\|ERROR] separated list of
85	ajm	1.1	filenames
86			eg,
87			a.conf;b.conf;c.conf
88
89			******** LOOP START ********
90			This loop reads configuration
91			properties from the config
92
93			[{property}] ----------> Sends the name of a
94			property to be
95			retrieved from the
96			config file
97			eg, UDPUpdateTime
98	tdb	1.2
99	ajm	1.1	<---------- [{value}\|ERROR]Returns the value of
100			the requested config
101			property
102			eg, 120
103			If it is unable to
104	tdb	1.2	find the requested
105	ajm	1.1	property it returns
106			ERROR to indicate
107	tdb	1.2	that fact, but
108			communication still
109			proceeds
110
111	ajm	1.1	******** LOOP UNTIL ********
112			The loop continues until the host
113			sends the following message
114	tdb	1.2
115			[ENDCONFIG] ----------> Indicates that the
116	ajm	1.1	host requires no
117			more config
118	tdb	1.2
119	ajm	1.1	******** LOOP END ********
120			Communication continues
121	tdb	1.2
122	ajm	1.1	<---------- [OK] Indicates that the
123			server is ready to
124			proceed
125	tdb	1.2
126	ajm	1.1	[FILTER] ----------> Asks the server to
127			send it the host
128	tdb	1.2	information of a
129	ajm	1.1	filter that it
130			should connect to
131	tdb	1.2
132	ajm	1.1	<---------- {filter} Returns a semi-colon
133	tdb	1.2	separated list of
134	ajm	1.1	FQDN hostname, UDP
135			port number and TCP
136	tdb	1.2	port number that a
137	ajm	1.1	configured Filter is
138			running on and
139	tdb	1.2	assigned to the
140	ajm	1.1	calling host
141			eg,
142			raptor.ukc.ac.uk;
143			1234;5678
144
145			[END] ----------> Indicates to the
146			server that the host
147			has finished an will
148			disconnect
149	tdb	1.2
150	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
151			server is either ok
152			or that it thought
153			there was an error
154	ajm	1.3	<<<<<<< protocols.txt
155			------------------------------------------------------------
156
157			=======
158	tdb	1.2
159	ajm	1.3	>>>>>>> 1.2
160	ajm	1.1	Host Heartbeat Protocol
161			-----------------------
162			When a host is configured after it has connected it should
163			obtain a property TCPUpdateTime. This indicates how often
164	tdb	1.2	a host should send a "Heart Beat", which is a pro-active
165	ajm	1.1	connection to the server by the host to indicate that it is
166			still alive. This "Heart Beat" also allows a host to see
167	tdb	1.2	its configuration has changed and act accordingly. In a
168	ajm	1.1	well written host this should just be a case of dropping out
169			of a loop and heading back to the start (connecting to the
170			filter manager).
171
172			Host (direction) Server Comment
173			------------------------------------------------------------
174			[HEARTBEAT] ----------> Starts the
175			heartbeat protocol
176			off
177	tdb	1.2
178	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
179			server is ok or not
180
181			[CONFIG] ----------> Indicates that the
182			host wants to check
183			its config
184
185			<---------- [OK\|ERROR] Indicates that the
186			server is ok or not
187
188			[{filelist}] ----------> Send a semi-colon
189	tdb	1.2	separated list of
190	ajm	1.1	filenames
191			eg,
192			a.conf;b.conf;c.conf
193			This should be
194			identical to the one
195	tdb	1.2	received in the
196	ajm	1.1	connection protocol
197
198			<---------- [OK] Indicates that the
199			server is ok
200
201			[{lastmodified}]----------> Returns a long int
202	tdb	1.2	time since epoch
203	ajm	1.1	eg, 123456789
204			This should be
205			identical to the one
206	tdb	1.2	received in the
207	ajm	1.1	connection protocol
208
209			<---------- [OK\|ERROR] The server then
210			checks all the files
211			in the file list to
212			see if they have
213			been modified more
214			recently than the
215	tdb	1.2	lastmodified value.
216			If they HAVE that
217	ajm	1.1	indicates that the
218			configuration has
219			changed and the host
220			should re-configure,
221			thus it sends ERROR.
222			If the files remain
223			unchanged the server
224			will return OK to
225			indicate the host
226			should continue as
227			before
228
229			[ENDHEARTBEAT] ----------> Indicates to the
230			server that the host
231			has finished an will
232			disconnect
233	tdb	1.2
234	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
235			server is either ok
236			or that it thought
237			there was an error
238	ajm	1.3	------------------------------------------------------------
239	ajm	1.1
240			Host Data Protocol
241			------------------
242			The UDP data packets that are sent by the host contain
243			simply XML data. For information on the format of this
244			data see the XML via UDP specification document at:
245
246	ajm	1.4	http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt
247	ajm	1.1
248			Client Control Protocol
249			-----------------------
250
251	ajm	1.3	The client control protocol channel is opened by the client
252			and allows a variety of actions to be carried out by the
253			client at anytime. Unlike previous protocols, this is NOT
254			sequential, all of the requests can be carried out in
255			any order.
256
257			All client protocols are backwards compatible, and the
258			version is shown by the protocol identifier.
259
260			There are three sections to this protocol.
261
262			1) Initialise (sent only at start)
263			2) Send command(s) - unlimited number in any order
264			3) Disconnect (sent only at end)
265
266			If at anytime the client sends something the server does
267			not understand, an [ERROR] will be sent
268
269			All are indicated with <> brackets.
270
271			v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
272			Supported commands:
273			STARTCONFIG
274			STARTDATA
275			STOPDATA
276
277			Host (direction) Server Comment
278			------------------------------------------------------------
279			<initialise>
280			[CONNECT] ----------> Starts off the
281			client protocol
282
283			<---------- [{protocol}] The server sends the
284			protocol identifier
285
286			[{name}] ----------> The client sends its
287			"name". This name
288			is used to identify
289			the type of client
290			to the system and
291			obtain its config
292			eg, Conient
293
294			<---------- [OK] Indicates the server
295			is ok to proceed
296
297
298			<command #1> (allows client to obtain its configuration)
299			[STARTCONFIG]----------> Tell the server we
300			want to start this
301			command
302
303			<---------- [OK] Indicates the server
304			is ok to proceed
305
306			******** LOOP START ********
307			This loop reads configuration
308			properties from the config
309
310			[{property}] ----------> Sends the name of a
311			property to be
312			retrieved from the
313			config file
314			Clients can obtain
315			host information
316			eg,
317			Host.UDPUpdateTime
318			Otherwise it must
319			prefix requests
320			with "Client."
321			All other requests
322			will be ignored as
323			if it was unable to
324			retieve the property
325
326			<---------- [{value}\|ERROR]Returns the value of
327			the requested config
328			property
329			eg, 120
330			If it is unable to
331			find the requested
332			property it returns
333			ERROR to indicate
334			that fact
335
336			******** LOOP UNTIL ********
337			The loop continues until the client
338			sends the following message
339
340			[ENDCONFIG] ----------> Indicates that the
341			client requires no
342			more config
343
344			******** LOOP END ********
345			Communication continues
346
347			<---------- [OK] Indicates that the
348			server is ready to
349			proceed
350
351			<command #2> (tells the server to start the data link)
352			[STARTDATA] ----------> Tell the server we
353			want to start this
354			command
355
356			<---------- [{portnumber} The server then sets
357			\|ERROR] itself listening for
358			a connection on its
359			data link socket for
360			this client. It
361			returns the port no.
362			that it is listening
363			on eg, 12367
364			If the data link is
365			yet to be started
366			the server will
367			return ERROR
368
369			<---------- [OK] Indicates the server
370			is ok to proceed and
371			the client can
372			then connect its
373			data link
374
375			<command #3> (tells the server to stop the data link)
376			[STOPDATA] ----------> Tell the server we
377			want to start this
378			command
379			The server then
380			shuts down the
381			data link to the
382			client
383
384			<---------- [OK\|ERROR] Returns OK is the
385			server is ready to
386			proceed, or ERROR
387			if the data link
388			was not open in the
389			first place
390
391			<disconnect>
392			[DISCONNECT] ----------> Tells the server the
393			client wants to
394			close the control
395			link
396
397			<---------- [OK] The last word from
398			the server, it will
399			disappear after this
400			------------------------------------------------------------
401
402			v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
403			Supported commands:
404			SETHOSTLIST
405
406			Host (direction) Server Comment
407			------------------------------------------------------------
408			<command #4> (indicates to the server which hosts the client
409			wants data from)
410			[SETHOSTLIST]----------> Tell the server we
411			want to start this
412			command
413
414			<---------- [OK\|ERROR] The server will
415			return an ERROR if
416			the data link is
417			open, as a host list
418			must ONLY be set
419			if the data link is
420			closed
421			If the server is
422			ok to proceed with
423			the command it says
424			[OK]
425
426			[{hostlist}] ----------> This is a semi-colon
427			seperated list of
428			FQDN hostnames that
429			the client wishes to
430			recieve data from
431			eg,
432			raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
433
434			If the list is sent
435			blank (or if no list
436			is set at all) then
437			data from ALL hosts
438			will be sent to the
439			client
440
441			<---------- [OK] Indicates the server
442			is ok to proceed and
443			the host list has
444			been accepted
445			------------------------------------------------------------
446	ajm	1.1
447			Client Data Protocol
448			--------------------
449	ajm	1.3	Once the data link has been opened by the control link, the
450			server will send XML formated data packets seperated by the
451			\n (newline) character. For information on the format of
452			this data see the XML via UDP specification document at:
453	ajm	1.1
454	ajm	1.4	http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specification/xml_via_udp.txt
455	ajm	1.1
456			About
457			-----
458			This document was written by AJ Moore [ajm4@ukc.ac.uk] for
459			use by the team working on a 3rd year Computer Science
460			project called "i-scream". More details can be found on the
461			project website;
462
463			http://www.i-scream.org.uk