documentation/specification/protocols.txt

i-scream Protocols
==================

Contents
--------
 - Conventions

 - Host Connection Protocol
 - Host Heartbeat Protocol
 - Host Data Protocol

 - Client Control Protocol
 - Client Data Protocol


Conventions
-----------
All protocols in this document assume a valid connection of
the appropriate type has been made, and that data streams
are already available.  All strings should, and will be,
terminated with a \n (newline character) to indicate the end
of the message.  All messages are sent as ASCII Strings.

Fixed protocol messages will appear in [] brackets, if there
is a variety of options they will be separated with the |
character.  For example:

    [OK|ERROR]
   
Indicates that the message "OK" OR the message "ERROR" will
be sent depending on the result of the last action.

Data messages will appear in {} brackets, where the name in
the brackets indicates the type (as in kind, not data type)
of data that will be returned.  For example:

    {lastmodified}

Indicates that data for use as 'last modified' will be 
returned.   

If [ERROR] is sent back at any time, this indicates that all 
communication is over. EXCEPT where otherwise specified!


Host Connection Protocol
------------------------
The initial connection of a host to the i-scream server is
through the FilterManager.  A Host gets its configuration
and then gets assigned a Filter to connect to and start
sending data.

The port number of the FilterManager is fully configurable,
however the default at this time is 4567.

Host         (direction) Server         Comment
------------------------------------------------------------
[STARTCONFIG]---------->                Requests to start
                                        receiving config
                                        information
            
             <---------- [OK|ERROR]     If the server ok's
                                        the request or not

[LASTMODIFIED]---------->               Asks when the config
                                        was last modified
                                        (used when checking
                                        if the config has
                                        changed)
            
             <---------- [{lastmodified}Returns a long int
                         |ERROR]        time since epoch
                                        eg, 123456789

[FILELIST]   ---------->                Asks the server for
                                        the list of files
                                        that were used to
                                        build the config
                                        (used when checking
                                        if the config has
                                        changed)
       
             <---------- [{filelist}    Returns a semi-colon
                         |ERROR]        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf

             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{property}] ---------->                Sends the name of a
                                        property to be
                                        retrieved from the
                                        config file
                                        eg, UDPUpdateTime
            
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested
                                        property it returns
                                        ERROR to indicate
                                        that fact, but 
                                        communication still 
                                        proceeds
                                       
             ********** LOOP UNTIL **********
             The loop continues until the host
             sends the following message
            
[ENDCONFIG]  ---------->                Indicates that the
                                        host requires no
                                        more config
            
             ********** LOOP  END **********
             Communication continues
            
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed
                                       
[FILTER]     ---------->                Asks the server to
                                        send it the host
                                        information of a
                                        filter that it
                                        should connect to
                                       
             <---------- {filter}       Returns a semi-colon
                                        separated list of
                                        FQDN hostname, UDP
                                        port number and TCP
                                        port number that a
                                        configured Filter is
                                        running on and
                                        assigned to the
                                        calling host
                                        eg,
                                        raptor.ukc.ac.uk;
                                        1234;5678

[END]        ---------->                Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
            
             <---------- [OK|ERROR]     Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
<<<<<<< protocols.txt
------------------------------------------------------------

=======
       
>>>>>>> 1.2
Host Heartbeat Protocol
-----------------------
When a host is configured after it has connected it should
obtain a property TCPUpdateTime.  This indicates how often
a host should send a "Heart Beat", which is a pro-active
connection to the server by the host to indicate that it is
still alive.  This "Heart Beat" also allows a host to see
its configuration has changed and act accordingly.  In a
well written host this should just be a case of dropping out
of a loop and heading back to the start (connecting to the
filter manager).

Host            (direction) Server      Comment
------------------------------------------------------------
[HEARTBEAT]     ---------->             Starts the
                                        heartbeat protocol
                                        off
                                       
                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[CONFIG]        ---------->             Indicates that the
                                        host wants to check
                                        its config

                <---------- [OK|ERROR]  Indicates that the
                                        server is ok or not

[{filelist}]    ---------->             Send a semi-colon
                                        separated list of
                                        filenames
                                        eg,
                                        a.conf;b.conf;c.conf
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol

                <---------- [OK]        Indicates that the
                                        server is ok

[{lastmodified}]---------->             Returns a long int
                                        time since epoch
                                        eg, 123456789
                                        This should be
                                        identical to the one
                                        received in the
                                        connection protocol

                <---------- [OK|ERROR]  The server then
                                        checks all the files
                                        in the file list to
                                        see if they have
                                        been modified more
                                        recently than the
                                        lastmodified value.
                                        If they HAVE that
                                        indicates that the
                                        configuration has
                                        changed and the host
                                        should re-configure,
                                        thus it sends ERROR.
                                        If the files remain
                                        unchanged the server
                                        will return OK to
                                        indicate the host
                                        should continue as
                                        before

[ENDHEARTBEAT]  ---------->             Indicates to the
                                        server that the host
                                        has finished an will
                                        disconnect
               
                <---------- [OK|ERROR]  Indicates that the
                                        server is either ok
                                        or that it thought
                                        there was an error
------------------------------------------------------------

Host Data Protocol
------------------
The UDP data packets that are sent by the host contain
simply XML data.  For information on the format of this
data see the XML via UDP specification document at:

http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specificatio
n/xml_via_udp.txt

Client Control Protocol
-----------------------

The client control protocol channel is opened by the client
and allows a variety of actions to be carried out by the
client at anytime.  Unlike previous protocols, this is NOT
sequential, all of the requests can be carried out in 
any order.

All client protocols are backwards compatible, and the
version is shown by the protocol identifier.

There are three sections to this protocol.

1) Initialise (sent only at start)
2) Send command(s) - unlimited number in any order
3) Disconnect (sent only at end)

If at anytime the client sends something the server does
not understand, an [ERROR] will be sent

All are indicated with <> brackets.

v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
       Supported commands:
            STARTCONFIG
            STARTDATA
            STOPDATA

Host         (direction) Server         Comment
------------------------------------------------------------
<initialise>
[CONNECT]    ---------->                Starts off the 
                                        client protocol
             
             <---------- [{protocol}]   The server sends the
                                        protocol identifier
                                        
[{name}]     ---------->                The client sends its
                                        "name".  This name
                                        is used to identify
                                        the type of client
                                        to the system and
                                        obtain its config
                                        eg, Conient
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed
                        

<command #1> (allows client to obtain its configuration)
[STARTCONFIG]---------->                Tell the server we
                                        want to start this
                                        command
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed
             
             ********** LOOP START **********
             This loop reads configuration
             properties from the config

[{property}] ---------->                Sends the name of a
                                        property to be
                                        retrieved from the
                                        config file
                                        Clients can obtain
                                        host information
                                        eg, 
                                        Host.UDPUpdateTime
                                        Otherwise it must
                                        prefix requests
                                        with "Client."
                                        All other requests
                                        will be ignored as
                                        if it was unable to
                                        retieve the property
                                        
             <---------- [{value}|ERROR]Returns the value of
                                        the requested config
                                        property
                                        eg, 120
                                        If it is unable to
                                        find the requested 
                                        property it returns
                                        ERROR to indicate
                                        that fact
                                        
             ********** LOOP UNTIL **********
             The loop continues until the client
             sends the following message
             
[ENDCONFIG]  ---------->                Indicates that the 
                                        client requires no
                                        more config
             
             ********** LOOP  END **********
             Communication continues
             
             <---------- [OK]           Indicates that the
                                        server is ready to
                                        proceed

<command #2> (tells the server to start the data link)
[STARTDATA]  ---------->                Tell the server we
                                        want to start this
                                        command
            
             <---------- [{portnumber}  The server then sets
                         |ERROR]        itself listening for
                                        a connection on its
                                        data link socket for
                                        this client.  It
                                        returns the port no.
                                        that it is listening
                                        on eg, 12367
                                        If the data link is
                                        yet to be started
                                        the server will
                                        return ERROR
             
             <---------- [OK]           Indicates the server
                                        is ok to proceed and
                                        the client can 
                                        then connect its
                                        data link

<command #3> (tells the server to stop the data link)
[STOPDATA]   ---------->                Tell the server we
                                        want to start this
                                        command
                                        The server then
                                        shuts down the 
                                        data link to the
                                        client

             <---------- [OK|ERROR]     Returns OK is the
                                        server is ready to
                                        proceed, or ERROR
                                        if the data link
                                        was not open in the
                                        first place

<disconnect>
[DISCONNECT] ---------->                Tells the server the
                                        client wants to
                                        close the control
                                        link
                                        
             <---------- [OK]           The last word from
                                        the server, it will
                                        disappear after this
------------------------------------------------------------

v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
       Supported commands:
            SETHOSTLIST

Host         (direction) Server         Comment
------------------------------------------------------------
<command #4> (indicates to the server which hosts the client
              wants data from)
[SETHOSTLIST]---------->                Tell the server we
                                        want to start this
                                        command
             
             <---------- [OK|ERROR]     The server will
                                        return an ERROR if
                                        the data link is
                                        open, as a host list
                                        must ONLY be set 
                                        if the data link is
                                        closed
                                        If the server is
                                        ok to proceed with
                                        the command it says
                                        [OK]

[{hostlist}] ---------->                This is a semi-colon
                                        seperated list of
                                        FQDN hostnames that
                                        the client wishes to
                                        recieve data from
                                        eg,
        raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
                                        
                                        If the list is sent
                                        blank (or if no list
                                        is set at all) then
                                        data from ALL hosts
                                        will be sent to the
                                        client
                                        
             <---------- [OK]           Indicates the server
                                        is ok to proceed and
                                        the host list has 
                                        been accepted
------------------------------------------------------------

Client Data Protocol
--------------------
Once the data link has been opened by the control link, the
server will send XML formated data packets seperated by the 
\n (newline) character.  For information on the format of 
this data see the XML via UDP specification document at:

http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specificatio
n/xml_via_udp.txt

About
-----
This document was written by AJ Moore [ajm4@ukc.ac.uk] for
use by the team working on a 3rd year Computer Science
project called "i-scream". More details can be found on the
project website;

http://www.i-scream.org.uk
Revision:	1.3
Committed:	Sun Jan 28 20:05:35 2001 UTC (23 years, 3 months ago) by ajm
Content type:	text/plain
Branch:	MAIN
Changes since 1.2:	+207 -2 lines
Log Message:	added details of client protocol
#	User	Rev	Content
1	ajm	1.1	i-scream Protocols
2			==================
3
4			Contents
5			--------
6			- Conventions
7
8			- Host Connection Protocol
9			- Host Heartbeat Protocol
10			- Host Data Protocol
11
12			- Client Control Protocol
13			- Client Data Protocol
14
15	tdb	1.2
16	ajm	1.1	Conventions
17			-----------
18			All protocols in this document assume a valid connection of
19			the appropriate type has been made, and that data streams
20	tdb	1.2	are already available. All strings should, and will be,
21	ajm	1.1	terminated with a \n (newline character) to indicate the end
22			of the message. All messages are sent as ASCII Strings.
23
24			Fixed protocol messages will appear in [] brackets, if there
25	tdb	1.2	is a variety of options they will be separated with the \|
26	ajm	1.1	character. For example:
27
28			[OK\|ERROR]
29	tdb	1.2
30	ajm	1.1	Indicates that the message "OK" OR the message "ERROR" will
31			be sent depending on the result of the last action.
32
33			Data messages will appear in {} brackets, where the name in
34			the brackets indicates the type (as in kind, not data type)
35			of data that will be returned. For example:
36
37			{lastmodified}
38
39	tdb	1.2	Indicates that data for use as 'last modified' will be
40			returned.
41
42			If [ERROR] is sent back at any time, this indicates that all
43			communication is over. EXCEPT where otherwise specified!
44	ajm	1.1
45
46			Host Connection Protocol
47			------------------------
48			The initial connection of a host to the i-scream server is
49			through the FilterManager. A Host gets its configuration
50	tdb	1.2	and then gets assigned a Filter to connect to and start
51	ajm	1.1	sending data.
52
53	tdb	1.2	The port number of the FilterManager is fully configurable,
54	ajm	1.1	however the default at this time is 4567.
55
56			Host (direction) Server Comment
57			------------------------------------------------------------
58			[STARTCONFIG]----------> Requests to start
59			receiving config
60			information
61	tdb	1.2
62	ajm	1.1	<---------- [OK\|ERROR] If the server ok's
63			the request or not
64
65			[LASTMODIFIED]----------> Asks when the config
66			was last modified
67			(used when checking
68	tdb	1.2	if the config has
69	ajm	1.1	changed)
70	tdb	1.2
71	ajm	1.1	<---------- [{lastmodified}Returns a long int
72	tdb	1.2	\|ERROR] time since epoch
73	ajm	1.1	eg, 123456789
74
75			[FILELIST] ----------> Asks the server for
76			the list of files
77			that were used to
78			build the config
79			(used when checking
80			if the config has
81			changed)
82	tdb	1.2
83	ajm	1.1	<---------- [{filelist} Returns a semi-colon
84	tdb	1.2	\|ERROR] separated list of
85	ajm	1.1	filenames
86			eg,
87			a.conf;b.conf;c.conf
88
89			******** LOOP START ********
90			This loop reads configuration
91			properties from the config
92
93			[{property}] ----------> Sends the name of a
94			property to be
95			retrieved from the
96			config file
97			eg, UDPUpdateTime
98	tdb	1.2
99	ajm	1.1	<---------- [{value}\|ERROR]Returns the value of
100			the requested config
101			property
102			eg, 120
103			If it is unable to
104	tdb	1.2	find the requested
105	ajm	1.1	property it returns
106			ERROR to indicate
107	tdb	1.2	that fact, but
108			communication still
109			proceeds
110
111	ajm	1.1	******** LOOP UNTIL ********
112			The loop continues until the host
113			sends the following message
114	tdb	1.2
115			[ENDCONFIG] ----------> Indicates that the
116	ajm	1.1	host requires no
117			more config
118	tdb	1.2
119	ajm	1.1	******** LOOP END ********
120			Communication continues
121	tdb	1.2
122	ajm	1.1	<---------- [OK] Indicates that the
123			server is ready to
124			proceed
125	tdb	1.2
126	ajm	1.1	[FILTER] ----------> Asks the server to
127			send it the host
128	tdb	1.2	information of a
129	ajm	1.1	filter that it
130			should connect to
131	tdb	1.2
132	ajm	1.1	<---------- {filter} Returns a semi-colon
133	tdb	1.2	separated list of
134	ajm	1.1	FQDN hostname, UDP
135			port number and TCP
136	tdb	1.2	port number that a
137	ajm	1.1	configured Filter is
138			running on and
139	tdb	1.2	assigned to the
140	ajm	1.1	calling host
141			eg,
142			raptor.ukc.ac.uk;
143			1234;5678
144
145			[END] ----------> Indicates to the
146			server that the host
147			has finished an will
148			disconnect
149	tdb	1.2
150	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
151			server is either ok
152			or that it thought
153			there was an error
154	ajm	1.3	<<<<<<< protocols.txt
155			------------------------------------------------------------
156
157			=======
158	tdb	1.2
159	ajm	1.3	>>>>>>> 1.2
160	ajm	1.1	Host Heartbeat Protocol
161			-----------------------
162			When a host is configured after it has connected it should
163			obtain a property TCPUpdateTime. This indicates how often
164	tdb	1.2	a host should send a "Heart Beat", which is a pro-active
165	ajm	1.1	connection to the server by the host to indicate that it is
166			still alive. This "Heart Beat" also allows a host to see
167	tdb	1.2	its configuration has changed and act accordingly. In a
168	ajm	1.1	well written host this should just be a case of dropping out
169			of a loop and heading back to the start (connecting to the
170			filter manager).
171
172			Host (direction) Server Comment
173			------------------------------------------------------------
174			[HEARTBEAT] ----------> Starts the
175			heartbeat protocol
176			off
177	tdb	1.2
178	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
179			server is ok or not
180
181			[CONFIG] ----------> Indicates that the
182			host wants to check
183			its config
184
185			<---------- [OK\|ERROR] Indicates that the
186			server is ok or not
187
188			[{filelist}] ----------> Send a semi-colon
189	tdb	1.2	separated list of
190	ajm	1.1	filenames
191			eg,
192			a.conf;b.conf;c.conf
193			This should be
194			identical to the one
195	tdb	1.2	received in the
196	ajm	1.1	connection protocol
197
198			<---------- [OK] Indicates that the
199			server is ok
200
201			[{lastmodified}]----------> Returns a long int
202	tdb	1.2	time since epoch
203	ajm	1.1	eg, 123456789
204			This should be
205			identical to the one
206	tdb	1.2	received in the
207	ajm	1.1	connection protocol
208
209			<---------- [OK\|ERROR] The server then
210			checks all the files
211			in the file list to
212			see if they have
213			been modified more
214			recently than the
215	tdb	1.2	lastmodified value.
216			If they HAVE that
217	ajm	1.1	indicates that the
218			configuration has
219			changed and the host
220			should re-configure,
221			thus it sends ERROR.
222			If the files remain
223			unchanged the server
224			will return OK to
225			indicate the host
226			should continue as
227			before
228
229			[ENDHEARTBEAT] ----------> Indicates to the
230			server that the host
231			has finished an will
232			disconnect
233	tdb	1.2
234	ajm	1.1	<---------- [OK\|ERROR] Indicates that the
235			server is either ok
236			or that it thought
237			there was an error
238	ajm	1.3	------------------------------------------------------------
239	ajm	1.1
240			Host Data Protocol
241			------------------
242			The UDP data packets that are sent by the host contain
243			simply XML data. For information on the format of this
244			data see the XML via UDP specification document at:
245
246	tdb	1.2	http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specificatio
247			n/xml_via_udp.txt
248	ajm	1.1
249			Client Control Protocol
250			-----------------------
251
252	ajm	1.3	The client control protocol channel is opened by the client
253			and allows a variety of actions to be carried out by the
254			client at anytime. Unlike previous protocols, this is NOT
255			sequential, all of the requests can be carried out in
256			any order.
257
258			All client protocols are backwards compatible, and the
259			version is shown by the protocol identifier.
260
261			There are three sections to this protocol.
262
263			1) Initialise (sent only at start)
264			2) Send command(s) - unlimited number in any order
265			3) Disconnect (sent only at end)
266
267			If at anytime the client sends something the server does
268			not understand, an [ERROR] will be sent
269
270			All are indicated with <> brackets.
271
272			v1.0 - Protocol identifier: "PROTOCOL 1.0" (without quotes)
273			Supported commands:
274			STARTCONFIG
275			STARTDATA
276			STOPDATA
277
278			Host (direction) Server Comment
279			------------------------------------------------------------
280			<initialise>
281			[CONNECT] ----------> Starts off the
282			client protocol
283
284			<---------- [{protocol}] The server sends the
285			protocol identifier
286
287			[{name}] ----------> The client sends its
288			"name". This name
289			is used to identify
290			the type of client
291			to the system and
292			obtain its config
293			eg, Conient
294
295			<---------- [OK] Indicates the server
296			is ok to proceed
297
298
299			<command #1> (allows client to obtain its configuration)
300			[STARTCONFIG]----------> Tell the server we
301			want to start this
302			command
303
304			<---------- [OK] Indicates the server
305			is ok to proceed
306
307			******** LOOP START ********
308			This loop reads configuration
309			properties from the config
310
311			[{property}] ----------> Sends the name of a
312			property to be
313			retrieved from the
314			config file
315			Clients can obtain
316			host information
317			eg,
318			Host.UDPUpdateTime
319			Otherwise it must
320			prefix requests
321			with "Client."
322			All other requests
323			will be ignored as
324			if it was unable to
325			retieve the property
326
327			<---------- [{value}\|ERROR]Returns the value of
328			the requested config
329			property
330			eg, 120
331			If it is unable to
332			find the requested
333			property it returns
334			ERROR to indicate
335			that fact
336
337			******** LOOP UNTIL ********
338			The loop continues until the client
339			sends the following message
340
341			[ENDCONFIG] ----------> Indicates that the
342			client requires no
343			more config
344
345			******** LOOP END ********
346			Communication continues
347
348			<---------- [OK] Indicates that the
349			server is ready to
350			proceed
351
352			<command #2> (tells the server to start the data link)
353			[STARTDATA] ----------> Tell the server we
354			want to start this
355			command
356
357			<---------- [{portnumber} The server then sets
358			\|ERROR] itself listening for
359			a connection on its
360			data link socket for
361			this client. It
362			returns the port no.
363			that it is listening
364			on eg, 12367
365			If the data link is
366			yet to be started
367			the server will
368			return ERROR
369
370			<---------- [OK] Indicates the server
371			is ok to proceed and
372			the client can
373			then connect its
374			data link
375
376			<command #3> (tells the server to stop the data link)
377			[STOPDATA] ----------> Tell the server we
378			want to start this
379			command
380			The server then
381			shuts down the
382			data link to the
383			client
384
385			<---------- [OK\|ERROR] Returns OK is the
386			server is ready to
387			proceed, or ERROR
388			if the data link
389			was not open in the
390			first place
391
392			<disconnect>
393			[DISCONNECT] ----------> Tells the server the
394			client wants to
395			close the control
396			link
397
398			<---------- [OK] The last word from
399			the server, it will
400			disappear after this
401			------------------------------------------------------------
402
403			v1.1 - Protocol identifier: "PROTOCOL 1.1" (without quotes)
404			Supported commands:
405			SETHOSTLIST
406
407			Host (direction) Server Comment
408			------------------------------------------------------------
409			<command #4> (indicates to the server which hosts the client
410			wants data from)
411			[SETHOSTLIST]----------> Tell the server we
412			want to start this
413			command
414
415			<---------- [OK\|ERROR] The server will
416			return an ERROR if
417			the data link is
418			open, as a host list
419			must ONLY be set
420			if the data link is
421			closed
422			If the server is
423			ok to proceed with
424			the command it says
425			[OK]
426
427			[{hostlist}] ----------> This is a semi-colon
428			seperated list of
429			FQDN hostnames that
430			the client wishes to
431			recieve data from
432			eg,
433			raptor.ukc.ac.uk;killigrew.ukc.ac.uk;chalk.ukc.ac.uk
434
435			If the list is sent
436			blank (or if no list
437			is set at all) then
438			data from ALL hosts
439			will be sent to the
440			client
441
442			<---------- [OK] Indicates the server
443			is ok to proceed and
444			the host list has
445			been accepted
446			------------------------------------------------------------
447	ajm	1.1
448			Client Data Protocol
449			--------------------
450	ajm	1.3	Once the data link has been opened by the control link, the
451			server will send XML formated data packets seperated by the
452			\n (newline) character. For information on the format of
453			this data see the XML via UDP specification document at:
454	ajm	1.1
455	ajm	1.3	http://www.i-scream.org.uk/cgi-bin/docs.cgi?doc=specificatio
456			n/xml_via_udp.txt
457	ajm	1.1
458			About
459			-----
460			This document was written by AJ Moore [ajm4@ukc.ac.uk] for
461			use by the team working on a 3rd year Computer Science
462			project called "i-scream". More details can be found on the
463			project website;
464
465			http://www.i-scream.org.uk