From 3c2bf608b910dca1d5dadea9976ff31d40b780e8 Mon Sep 17 00:00:00 2001 From: Heikki Hannikainen Date: Wed, 15 Aug 2012 06:56:29 +0000 Subject: [PATCH] Documentation, more documentation. git-svn-id: http://repo.ham.fi/svn/aprsc/trunk@638 3ce903b1-3385-4e86-93cd-f9a4a239f7ac --- doc/BUILDING.md | 52 ++++++++++++-- doc/CONFIGURATION.md | 6 ++ doc/README.md | 161 +++++++++++++++++++++++++++++++++++++------ 3 files changed, 195 insertions(+), 24 deletions(-) create mode 100644 doc/CONFIGURATION.md diff --git a/doc/BUILDING.md b/doc/BUILDING.md index 7f97132..725f837 100644 --- a/doc/BUILDING.md +++ b/doc/BUILDING.md @@ -2,6 +2,10 @@ Compiling aprsc from source =========================== +WARNING: THIS DOCUMENT CURRENTLY DOCUMENTS FUTURE FUNCTIONALITY - THE +CONFIGURE SCRIPT IS NOT QUITE FINISHED YET AND YOU CAN'T MAKE INSTALL +EITHER. + If you're familiar with compiling software from the source code, and pre-built binary packages are not available for your platform, this is where you need to start. Binary packages will be provided shortly for Debian and @@ -11,9 +15,9 @@ aprsc has been built and tested on: * Ubuntu LTS (10.04, 12.04) * Debian stable (6.0, "squeeze") -* Debian 5.0 ("lenny"): i386 and x86_64 +* Debian 5.0 "lenny": i386 and x86_64 * Mac OS X 10.8 (Snow Leopard): x86 -* FreeBSD 9.0 amd64 +* FreeBSD 8.2 and 9.0 on amd64, 7.2 on i386 * Solaris 11 (SunOS 5.11 11.0 i86pc i386) If you wish to have decent support, please pick Debian or Ubuntu. The other @@ -21,8 +25,8 @@ platforms do work, but when it comes to building and installing, you're mostly on your own. -Prerequirements ------------------- +Prerequirements and dependencies +----------------------------------- aprsc requires a recent gcc version (4.1) to compile due to the need for built-in functions for atomic memory access. @@ -36,4 +40,44 @@ libevent or libevent2, but check if it's a 2.0). Older versions do not come with it, so you need to download it and compile it from source (http://libevent.org/). +libevent2 is also available in MacPorts for OS X and FreeBSD ports. + +Downloading aprsc +-------------------- + +Head to http://he.fi/aprsc/down/ + +Compiling aprsc +------------------ + + Delete old version if necessary: + $ rm aprsc-latest.tar.gz + + Download the latest source tree: + $ wget http://he.fi/aprsc/down/aprsc-latest.tar.gz + + Extract it: + $ tar xvfz aprsc-latest.tar.gz + + Go to the newly-created directory and configure the build: + $ cd aprsc-1.0.0 + $ ./configure + +At this point the configuration either succeeds or fails. If it fails, it is +probably due to a missing dependency, in which case it tries to tell clearly +what is missing. If your platform does not appear in the list of tested +platforms, the failures might be more "interesting". + + Compile it: + $ make + + Install it: + $ make install + + Install example configuration: + $ make installconf + + + + diff --git a/doc/CONFIGURATION.md b/doc/CONFIGURATION.md new file mode 100644 index 0000000..55cb41d --- /dev/null +++ b/doc/CONFIGURATION.md @@ -0,0 +1,6 @@ + +Configuring aprsc +================= + +TODO: write it. + diff --git a/doc/README.md b/doc/README.md index c797987..011b40c 100644 --- a/doc/README.md +++ b/doc/README.md @@ -2,15 +2,17 @@ aprsc - an APRS-IS server in C ============================== -aprsc (pronounced a-purrs-c) is a plain APRS-IS server intended to be used on the core and Tier2 -APRS-IS servers. It is written in the C language, and it runs on Linux and Unix servers. +aprsc (pronounced a-purrs-c) is a plain APRS-IS server intended to be used +on the core and Tier2 APRS-IS servers. It is written in the C language, and +it runs on Linux and Unix servers. If you need igate or other radio-interfacing features, aprsc is not for you. If you need to run a server on Windows, aprsc is not for you. Sorry! + A word of warning ---------------------- +-------------------- The aprsc software is brand new, under active development, and release cycles are currently very quick - be ready to upgrade on a short notice when @@ -22,14 +24,36 @@ should be more peaceful then. Documentation is not there yet, either, but if you're an experienced APRS-IS server operator, there should be no big surprises around. -Features (lack of) ---------------------- + +Features (and lack of) +------------------------- aprsc has been designed strictly for use within the APRS-IS core, hub and Tier2 servers. It includes only the basic functionality required by those -servers. It does not, and will not, have any additional functions such as -igating, digipeating, interfacing to radios, D-PRS or other gateway -functions, or object generation. It will not work on Windows. +servers: + +* Duplicate packet filtering +* Q construct processing +* Client-defined filters +* APRS packet parsing as necessary to support filtering +* i-gate client support +* Messaging support +* UDP client support +* UDP core peer links +* Uplink server support +* Passcode validation +* Web status page + Machine-readable JSON status on HTTP server +* HTTP position upload using POST +* Full IPv4 and IPv6 support +* Configurable access lists on client ports +* Logging to either syslog, file or stderr +* Built-in log rotation when logging to a file + +It does not, and will not, have any additional functions such as igating, +digipeating, interfacing to radios, D-PRS or other gateway functions, or +object generation. It will not work on Windows. The idea is to keep aprsc +relatively simple and lean, and leave the more specialized features for more +specialized software. If you need a nice, compact igate software for Linux, please take a look at either aprsg, aprx, or aprs4r. If you need to run an APRS-IS server on @@ -39,20 +63,109 @@ is the right choice for you – it's got a lot of good features that many of you need, and it works on virtually all operating systems. If you need an igate for Windows, APRSIS32 should be good. + Licensing, environments and requirements ------------------------------------------- aprsc is open source, licensed under the BSD license. It has about 11000 lines of relatively clean C code, built using the usual ./configure && make && make install method. The embedded HTTP status server is built using the -libevent2 library, no other extra libraries are needed. Linux and OS X are -the main development environments and will receive premium support, but -FreeBSD and Solaris 11 are known to work too. Packaged binaries for Debian -and Ubuntu will be available shortly for super-easy installations and -automatic upgrades. +libevent2 library, no other extra libraries are needed. + +Linux and OS X are the main development environments and will receive +premium support, but FreeBSD and Solaris 11 are known to work too. Packaged +binaries for Debian and Ubuntu will be available shortly for super-easy +installations and automatic upgrades. + + +Design +--------- + +aprsc's basic design was drawn out in a pizza session in early 2008. The +design goals were: + +* High throughput and small enough latency +* Support for thousands of clients per server +* Support for heavy bursts of new clients +* Scalability over multiple CPUs +* Low context switch overhead +* Low lock contention between threads + +A modern hybrid threaded / event-driven approach was selected. All recently +developed high-performance Internet servers work in this mode (some with +multiple event-driven processes, some with event-driven threads). There is +a small, fixed number of threads, close to the number of CPU cores on the +server, so that multiple CPU cores can be utilized, but the relatively +expensive context switches between a high number of threads will not cause +serious overhead. + +When the server is under heavy load, data transfers between threads happen +in blocks of multiple data units, so that contention on mutexes and +read-write locks will not block concurrent execution of the threads. Lock +contention makes many multi-threaded servers effectively single-threaded and +unable to use more than a single CPU core. + +Main work is done by 1 to N worker threads. In real-world APRS-IS today, 1 +worker thread is enough, but if a server was really heavily loaded with +thousands of clients, 1 less than the number of CPUs would be optimal. + +A worker thread's workflow goes like this: + +1. Read data from connected clients +2. Do initial APRS-IS packet parsing (SRCCALL>DSTCALL,PATH:DATA) +3. Do Q-construct processing (,qAx in the PATH) +4. Parse APRS formatted information in the DATA to extract enough details + to support filtering in the outgoing phase +5. Pass on received packets to the dupecheck thread for duplicate removal +6. Get packets, sorted to unique and duplicate packets, from the dupecheck + thread +7. Send out packets to clients as instructed by the listening port's + configuration and the client's filter settings + +The Dupecheck thread maintains a cache of packets heard during the past 30 +seconds. There is a dedicated thread for this cache, so that the worker +threads do not need to compete for access to the shared resource. The thread +gets packets from the worker threads, does dupe checking, and puts the +unique and duplicate packets in two global ordered buffers. The workers then +walk through those buffers and do filtering to decide which packets should +be sent to which clients. + +An Uplink threads initiates connections to upstream servers and reconnects +them as needed. After a successful connection the socket will be passed on +to a Worker thread which will proceed to exchange traffic with the remote +server. + +An Accept thread listens on the TCP ports for new incoming connections, does +access list checks, and distributes allowed connections evenly across worker +threads. + +An HTTP thread runs an event-driven HTTP server (libevent2 based) to support +the status page and HTTP position uploads. Since implementing nice web user +interfaces in plain C is not very convenient or effective, the status page +is produced using modern Web 2.0 methods. The HTTP server can only generate +a dynamic JSON-encoded status file and spit out static files. An empty +index.html file loads a static JavaScript file, which then periodically +loads the JSON status data and formats the contents of the status page +within the client's browser. This approach allowed clean separation of +server code (C) and web presentation (HTML5/JavaScript/jQuery/flot). + +Both developers are experienced professional Unix C programmers, so the +programming language was easy to select. We also had plenty of existing +code that could be re-used in this project. + +In hindsight, aprsc is slightly over-engineered and over-optimized for the +current real-word requirements. Performance, however, appears to match +the initial goals. + + +Performance +-------------- + +TODO: figures here + Quality control -------------------- +------------------ aprsc comes with an APRS-IS server test suite, implemented using the Perl Test framework. A "make test" executed in the tests/ subdirectory will @@ -70,17 +183,25 @@ the actual feature has been implemented in aprsc. This approach should ensure a good level of compatibility between the components and prevent old bugs from creeping back in. + Who's who, and how long did it take ---------------------------------------- +-------------------------------------- aprsc has been developed between 2008 and 2012 by Matti Aarnio, OH2MQK -(aprx), and Heikki Hannikainen, OH7LZB (aprs.fi). Design phase and most of -the core development happened during 2008, but the final sprint for feature -completeness happened during the summer of 2012. Substantial code reuse -happens between aprx, aprsc and other projects of the authors. +(aprx, zmailer, etc), and Heikki Hannikainen, OH7LZB (aprs.fi). Design +phase and most of the core development happened during 2008, but the final +sprint for feature completeness happened during the summer of 2012. +Substantial code reuse happens between aprx, aprsc and other projects of the +authors. + +aprsc was inspired by the performance problems experienced on the core +APRS-IS servers in early 2008, which were mitigated by moving CWOP clients +to their dedicated servers. It has also been driven by the desire for a +free, open-source server. + Discussion group --------------------- +------------------- aprsc has it's own discussion group and mailing list. If you run aprsc, please subscribe to the list to keep updated on new versions.