documentation/specification/spec-realtime.txt

I-Scream Specification Outline (Realtime side only)
===================================================

ajm4, 30/10/2000
tdb1, 30/10/2000

System Component Startup
************************

CORE
----


Client Interface
----------------
The Client Interface is essentially just one component with 
a series of lists within it. When run it should, obviously, 
create an instance of the Client Interface, and then bind 
this to the ORB and register with the naming service. It 
then needs to construct the "local clients". These clients 
communicate with the system using the same interface as the 
external clients, but they are tailored to specific 
purposes, such as E-Mail alerts, and SMS alerts. The Client 
Interface then listens on a "well known" address for clients 
to request a connection.

Filter
------
The filter is broken down into three main subcomponents. 

  - Filter Manager
      The Filter Manager is responsible for managing which 
      filters are used by the hosts. The Filter Manager is 
      available at a "well known" location which is pre-
      programmed into the hosts. The Filter Manager is 
      responsible for creating and managing the other 
      components of the filter system.
  
  - Main Filter
      The Main Filter is the single point that links back 
      into the CORE of the system. It will connect to the 
      DBI and the CLI to deliver data.
  
  - Filters
      There can be multipler Filters, and these are the 
      "front line" to the hosts. They all link back to the 
      Main Filter to send data into the system. It is 
      possible to run these Filters on any machine, allowing 
      management of data flow.

At startup a Filter Manager object is activated at the "well 
known" location (probably a given machine name at a 
predefined port). The Filter Manager will create an instance 
of the Main Filter, and any Filters under it's control. It 
should also bind itself to the ORB and register with the 
naming service. Through some mechanism the other Filters, 
elsewhere on the network, will register with the Filter 
Manager. The Filter Manager will need to tell each Filter 
the location of the Main Filter upon registering. The Filter 
Manager will then be in a position to receive connections 
from hosts and pass them off to Filters.

System Running State
********************

CORE
----


Client Interface
----------------
In the running state the Client Interface is always 
listening for clients on the "well known" address. When a 
connection is received it is passed in to the main Client 
Interface and the client is queried about which hosts it 
wishes to receive information about. This is then stored in 
an internal "routing table" so the Client Interface knows 
which hosts to send the information on to. This routing 
table is constructed with this form;

  host1: client1 client2 client5
  host2: client2
  host3: client3 client4
  host4: client1 client3

This design is such that when a piece of information is 
recieved from a host the Client Interface can immediately 
see which clients wish to receive this data, without too 
much searching.

The "local clients" function just like any other client, 
although they are local, in that they will wish to receive 
information about hosts they are interested in. However, 
they will contain a lot more logic, and be required to work 
out who wants to be alerted about what, and when. They will 
also be responsible for sending the alert.

Filter
------
When a host first loads up it knows where to locate the 
Filter Manager because it's located at a "well known" 
location. The host will fire up a TCP connection to the 
Filter Manager to announce itself. The Filter Manager will 
use some method (logically) to allocate a Filter to the 
host. The Filter Manager should base this decision on 
various factors, such as the load on the selection of 
filters, and possibly the location in relation to the host. 
The host will then be directed to this Filter for all 
further communications.

As the system runs the host will send data with (maybe) UDP 
to the Filter (that it's been allocated to). This choice has 
been made because it puts less onus on the host to make the 
connection, rather the data is just sent out. However, to 
ensure that the data isn't just disappearing into the depths 
of the network a periodic heartbeat will occur (at a 
predefined interval) over TCP to the Filter. This heartbeat 
can be used as a form of two-way communication, ensuring 
that everything is ok, and if required, to send any 
information back to the host. This heartbeat must occur 
otherwise the server may infer the host has died.

This could link in to alerting. An amber alert could be 
initiated for a host if the server stops receiving UDP 
packets, but an red alert be raised if the heartbeat doesn't 
occur.

If, for some reason, the Filter were to disappear the host 
should fall back on it's initial discovering mechanism - ie. 
contacting the Filter Manager at it's "well known" location. 
The host should report that it's lost it's Filter (so the 
Filter Manager can investigate and remove from it's list of 
Filters), and then the Filter Manager will reassign a new 
Filter to the host. Communication can then continue.

The idea of plugins to the Filters has been introduced. 
These plugins will implement a predefined plugin interface, 
and can be chained together at the Filter. Using the 
interface we can easily add future plugins that can do 
anything from parsing new data formats, to implementing 
encryption algorithms. The Filter will pass incoming data to 
each plugin in turn that it has available, and then finally 
pass the data on to the Main Filter. The Filter need not 
have any real knowledge about the content of the data.
Revision:	1.3
Committed:	Mon Oct 30 21:45:33 2000 UTC (24 years, 1 month ago) by tdb
Content type:	text/plain
Branch:	MAIN
Changes since 1.2:	+43 -9 lines
Log Message:	Written the section on the Client Interface, although it probably needs some more work.
#	User	Rev	Content
1	ajm	1.1	I-Scream Specification Outline (Realtime side only)
2			===================================================
3
4			ajm4, 30/10/2000
5	tdb	1.2	tdb1, 30/10/2000
6	ajm	1.1
7			System Component Startup
8	tdb	1.2	************************
9	ajm	1.1
10			CORE
11	tdb	1.2	----
12	ajm	1.1
13
14			Client Interface
15	tdb	1.2	----------------
16	tdb	1.3	The Client Interface is essentially just one component with
17			a series of lists within it. When run it should, obviously,
18			create an instance of the Client Interface, and then bind
19			this to the ORB and register with the naming service. It
20			then needs to construct the "local clients". These clients
21			communicate with the system using the same interface as the
22			external clients, but they are tailored to specific
23			purposes, such as E-Mail alerts, and SMS alerts. The Client
24			Interface then listens on a "well known" address for clients
25			to request a connection.
26	ajm	1.1
27			Filter
28	tdb	1.2	------
29			The filter is broken down into three main subcomponents.
30
31			- Filter Manager
32			The Filter Manager is responsible for managing which
33			filters are used by the hosts. The Filter Manager is
34			available at a "well known" location which is pre-
35			programmed into the hosts. The Filter Manager is
36			responsible for creating and managing the other
37			components of the filter system.
38
39			- Main Filter
40			The Main Filter is the single point that links back
41			into the CORE of the system. It will connect to the
42			DBI and the CLI to deliver data.
43
44			- Filters
45			There can be multipler Filters, and these are the
46			"front line" to the hosts. They all link back to the
47			Main Filter to send data into the system. It is
48			possible to run these Filters on any machine, allowing
49			management of data flow.
50
51			At startup a Filter Manager object is activated at the "well
52			known" location (probably a given machine name at a
53			predefined port). The Filter Manager will create an instance
54	tdb	1.3	of the Main Filter, and any Filters under it's control. It
55			should also bind itself to the ORB and register with the
56			naming service. Through some mechanism the other Filters,
57			elsewhere on the network, will register with the Filter
58			Manager. The Filter Manager will need to tell each Filter
59			the location of the Main Filter upon registering. The Filter
60			Manager will then be in a position to receive connections
61			from hosts and pass them off to Filters.
62	ajm	1.1
63			System Running State
64	tdb	1.2	********************
65	ajm	1.1
66			CORE
67	tdb	1.2	----
68	ajm	1.1
69
70			Client Interface
71	tdb	1.2	----------------
72	tdb	1.3	In the running state the Client Interface is always
73			listening for clients on the "well known" address. When a
74			connection is received it is passed in to the main Client
75			Interface and the client is queried about which hosts it
76			wishes to receive information about. This is then stored in
77			an internal "routing table" so the Client Interface knows
78			which hosts to send the information on to. This routing
79			table is constructed with this form;
80
81			host1: client1 client2 client5
82			host2: client2
83			host3: client3 client4
84			host4: client1 client3
85
86			This design is such that when a piece of information is
87			recieved from a host the Client Interface can immediately
88			see which clients wish to receive this data, without too
89			much searching.
90
91			The "local clients" function just like any other client,
92			although they are local, in that they will wish to receive
93			information about hosts they are interested in. However,
94			they will contain a lot more logic, and be required to work
95			out who wants to be alerted about what, and when. They will
96			also be responsible for sending the alert.
97	ajm	1.1
98			Filter
99	tdb	1.2	------
100			When a host first loads up it knows where to locate the
101			Filter Manager because it's located at a "well known"
102			location. The host will fire up a TCP connection to the
103			Filter Manager to announce itself. The Filter Manager will
104			use some method (logically) to allocate a Filter to the
105			host. The Filter Manager should base this decision on
106			various factors, such as the load on the selection of
107			filters, and possibly the location in relation to the host.
108			The host will then be directed to this Filter for all
109			further communications.
110
111			As the system runs the host will send data with (maybe) UDP
112			to the Filter (that it's been allocated to). This choice has
113			been made because it puts less onus on the host to make the
114			connection, rather the data is just sent out. However, to
115			ensure that the data isn't just disappearing into the depths
116			of the network a periodic heartbeat will occur (at a
117			predefined interval) over TCP to the Filter. This heartbeat
118			can be used as a form of two-way communication, ensuring
119			that everything is ok, and if required, to send any
120			information back to the host. This heartbeat must occur
121			otherwise the server may infer the host has died.
122
123			This could link in to alerting. An amber alert could be
124			initiated for a host if the server stops receiving UDP
125			packets, but an red alert be raised if the heartbeat doesn't
126			occur.
127
128			If, for some reason, the Filter were to disappear the host
129			should fall back on it's initial discovering mechanism - ie.
130			contacting the Filter Manager at it's "well known" location.
131			The host should report that it's lost it's Filter (so the
132			Filter Manager can investigate and remove from it's list of
133			Filters), and then the Filter Manager will reassign a new
134			Filter to the host. Communication can then continue.
135
136			The idea of plugins to the Filters has been introduced.
137			These plugins will implement a predefined plugin interface,
138			and can be chained together at the Filter. Using the
139			interface we can easily add future plugins that can do
140			anything from parsing new data formats, to implementing
141			encryption algorithms. The Filter will pass incoming data to
142			each plugin in turn that it has available, and then finally
143			pass the data on to the Main Filter. The Filter need not
144			have any real knowledge about the content of the data.