77 lines
3.7 KiB
Markdown
77 lines
3.7 KiB
Markdown
|
|
Design
|
|
---------
|
|
|
|
aprsc's basic design was drawn out in a pizza session in early 2008. The
|
|
design goals were:
|
|
|
|
* High throughput and small enough latency
|
|
* Support for thousands of clients per server
|
|
* Support for heavy bursts of new clients (CWOP hits every 5 or 10 minutes)
|
|
* Scalability over multiple CPUs
|
|
* Low context switch overhead
|
|
* Low lock contention between threads
|
|
|
|
A modern hybrid threaded / event-driven approach was selected. All recently
|
|
developed high-performance Internet servers work in this mode (some with
|
|
multiple event-driven processes, some with event-driven threads). There is
|
|
a small, fixed number of threads, close to the number of CPU cores on the
|
|
server, so that multiple CPU cores can be utilized, but the relatively
|
|
expensive context switches between a high number of threads will not cause
|
|
serious overhead.
|
|
|
|
When the server is under heavy load, data transfers between threads happen
|
|
in blocks of multiple data units, so that contention on mutexes and
|
|
read-write locks will not block concurrent execution of the threads. Lock
|
|
contention makes many multi-threaded servers effectively single-threaded and
|
|
unable to utilize more than a single CPU core.
|
|
|
|
Main work is done by 1 to N worker threads. In real-world APRS-IS today, 1
|
|
worker thread is enough, but if a server was really heavily loaded with
|
|
thousands of clients, 1 less than the number of CPU cores would be optimal.
|
|
|
|
A worker thread's workflow goes like this:
|
|
|
|
1. Read data from connected clients
|
|
2. Do initial APRS-IS packet parsing (SRCCALL>DSTCALL,PATH:DATA)
|
|
3. Do Q-construct processing (,qAx in the PATH)
|
|
4. Parse APRS formatted information in the DATA to extract enough details
|
|
to support filtering in the outgoing / filtering phase
|
|
5. Pass on received packets to the dupecheck thread for duplicate removal
|
|
6. Get packets, sorted to unique and duplicate packets, from the dupecheck
|
|
thread
|
|
7. Send out packets to clients as instructed by the listening port's
|
|
configuration and the client's filter settings
|
|
|
|
The Dupecheck thread maintains a cache of packets heard during the past 30
|
|
seconds. There is a dedicated thread for this cache, so that the worker
|
|
threads do not need to compete for access to the shared resource. The thread
|
|
gets packets from the worker threads, does dupe checking, and puts the
|
|
unique and duplicate packets in two global ordered buffer queues. The
|
|
workers then walk through those buffers and do filtering to decide which
|
|
packets should be sent to which clients.
|
|
|
|
An Uplink threads initiates connections to upstream servers and reconnects
|
|
them as needed. After a successful connection the socket will be passed on
|
|
to a Worker thread which will proceed to exchange traffic with the remote
|
|
server for the duration of the connection.
|
|
|
|
An Accept thread listens on the TCP ports for new incoming connections, does
|
|
access list checks, and distributes allowed connections evenly across worker
|
|
threads.
|
|
|
|
An HTTP thread runs an event-driven HTTP server (libevent2 based) to support
|
|
the status page and HTTP position uploads. Since implementing nice web user
|
|
interfaces in plain C is not very convenient or effective, the status page
|
|
is produced using modern Web 2.0 methods. The HTTP server can only generate
|
|
a dynamic JSON-encoded status file and serve static files. An empty
|
|
index.html file loads a static JavaScript file, which then periodically
|
|
loads the JSON status data and formats the contents of the status page
|
|
within the client's browser. This approach allowed clean separation of
|
|
server code (C) and web presentation (HTML5/JavaScript/jQuery/flot).
|
|
|
|
Both developers are experienced professional Unix C programmers, so the
|
|
programming language was easy to select. We also had plenty of existing
|
|
code that could be re-used in this project.
|
|
|