FireBrick - Firewalls, Bonding ADSL, Routers, Traffic Shaping...
FireBrick FB6000
Constant Quality Monitoring technical interface specification
The FireBrick provides constant quality monitoring. The main purpose of this is to provide a graphical representation of the performance of an interface or traffic shaper - typically used for broadband lines on L2TP
- 100 second interval statistics available graphically as png and in text as csv covering at least the last 25 hours (one day)
- Loss latency stats where available (e.g. LCP echos on L2TP broadband lines) including minimum, average, and maximum latency for the 100 second sample, and percentage packet loss
- Throughput stats where available (e.g. interfaces, shapers, L2TP broadband lines) including average tx and rx rate for 100 second sample
Graphs can be loss/latency or throughput of both. A ping only system would only have loss/latency. An L2TP broadband line has both. An interface or shaper normally has only throughput data.
Broadband back-haul providers
When using L2TP (FB6202) the CQM graphs are invaluable for diagnosing line faults. They are useful to the ISP but also useful to the back-haul provider which is often a separate company (e.g. BT or Be). We recommend that you consider providing access to graphs for live circuits and archived data to your back-haul provider when discussing faults with them. FireBrick are working with several ISPs to ensure back-haul providers are aware of the CQM graphs and how to use them to assist in diagnosis.
Access to graphs and csvs
Graphs can be accessed by http using the normal web management interface. This can be used as a direct link from a web browser, or using common tools such as curl and wget
The web management interface (services/http) define the port, and allowed user list and also a trusted IP access list. The CQM config defines a secret which is used to authorise untrusted access using an SHA1 hash in the URL.
All CQM URLs are in the /cqm/ path.
Trusted access
To access a graph you simply need to request the URL that is the graph name, followed by the file extension. E.g. http://host:port/cqm/circuit.png
| png | PNG image |
| csv | COMMA separated values list |
| tsv | TAB separated values list |
| txt | SPACE separated values list |
| xml | XML data |
Dated information
Without any date the data returned is the latest. For csv it is all data points available. For graph it is the last 24 to 25 hours.
You can display data for a specific date. This only makes sense for today, and during the first couple of hours of the day you can get yesterday in full.
The syntax is that of a date first in the form YYYY-MM-DD/, e.g. http://host:port/cqm/YYYY-MM-DD/circuit.png
Authenticated access
Authenticate access requires a prefix of a hex md5 string. e.g. http://host:port/cqm/longhexmd5/circuit.png or http://host:port/cqm/longhexmd5/YYYY-MM-DD/circuit.png
The SHA1 is 40 character hex of the SHA1 hash made from the graph name, the date, and the http-secret. The date is in the form YYYY-MM-DD, and is today's date for undated access (based on local time).
This means a graph URL can be composed that is valid for a specific graph name for a specific day.
Graph display options
The graphs can have a number of options which define the colours, text and layout. These are defined as http form get attributes on the URL, e.g. http://host:port/cqm/circuit.png?H=a+heading
Note that they can also be included in the path before the graph name, e.g. http://host:port/cqm/H=a+heading/circuit.png in which case they can be separated by / rather than &
The attributes are processed in order
Data points
The data point controls can be included as either fieldname or fieldname=colour. To make a valid URL either escape the # prefix or omit it. If any of these are included, then only those that are included are shown. If just fieldname is specified then the default colour is applied. The text on the right shows what fields are included and their colour key.
| M | Defines colour for minimum latency |
| A | Defines colour for average latency |
| X | Defines colour for max latency |
| U | Defines colour for upload rate |
| D | Defines colour for download rate |
| S | Defines colour for sent echos |
| F | Defines colour for failed (no response) echos |
| O | Defines colour for off-line |
Additional text
Additional text is shown on the graph based on the values in the configuration if not specified. There are 4 lines on the top left in small text and two heading lines top right in large text.
| z | Clean output, clears all additional text fields |
| Z | Clean and clear, as z but also sets inside background and off-line colours to transparent so graphs are easy to merge with those other LNSs |
| C | Line 1 top left text, default if not set in config is system name |
| c | Line 2 top left text |
| N | Line 3 top left text |
| n | Line 4 top left text/td> |
| H | Main heading text, default if not set in config is graph name |
| h | Sub heading text |
Other colours and spacing
Colours can be in the form of RGB, RRGGBB, RGBA, RRGGBBAA defining red/green/blue/alpha, or some simple colour names.
| L | Defines a number of pixels to be provided on the left of the graph. Bandwidth and scale axis shown based on space provided left and right. |
| R | Defines a number of pixels to be provided on the right of the graph. Bandwidth and scale axis is shown based on space provided left and right. |
| T | Defines a number of pixels to be provided on the top of the graph. Time axes is show based on space at top and bottom. |
| B | Defines a number of pixels to be provided on the bottom of the graph. Time axes is show based on space at top and bottom. |
| I | Defines colour for graticule |
| i | Defines colour for axis lines |
| g | Defines colour for background within axis |
| G | Defines colour for background outside axis |
| W | Defines colour for writing (text) |
Over night archiving
The system is designed to make it easy to archive all graphs or png, xml, etc files over night. The graphs hold 1000 data points, which is 27 hours 46 minutes. This means you can access a full day's data for the previous day in the first 3 hours 46 minutes of the new day (2 hours 46 or 4 hours 46 when clocks change in previous day). As such it is recommended that over night archiving is done of the previous day just after midnight.
The recommended command to run just after midnight is wget -m http://host:port/cqm/`date +%F`/z/ as this will create a directory for the server, cqm, date, and z, and then the files. The use of z clears text off the graphs to make them clean.
Full URL formal
The full URL format allows several variations. These are mainly to allow sensible directory structures in overnight archiving.
| /cqm/ | All CQM URLs start with this |
| 32-hex-characters/ | Optional authentication string needed for untrusted access. Can be used with trusted access to test the authentication is right |
| YYYY-MM-DD/ | Optional date to restrict output. Can also be in the form YYYY/MM/DD, YYYY-MM/DD, YYYY/MM-DD if preferred. Can also have /HH or -HH on the end to get data for just one hour, and /HH-HH, or -HH/HH on the end for a specific range of hours. |
| options/ | Optional graph colour control options. Useful when extracting a list of images as the all must have the same options as the list is just graphname.png as a relative link thereby ensuring all graphs appear in this directory. The options list can include / separators rather & separators to make apparent subdirectories. |
| ext/ | The file extension can be included on the end of the options, this is used only for making the index of all graphs for that type (see below) |
| graphname | Graph name |
| .ext | Extension for file type required |
| ?options | Options can alternatively be included as a html form get field list |
Where no graph name or ext are provided, i.e. the index page of a directory then an html page is served. An ext/ can be included after any options to make a list of files of that type. Otherwise the index is an html page explaining the options.
A blank graph is available by accessing simply .png (i.e. no graph name
An xml list of all graphs is available as .xml
A csv list of graph name and score is available as .csv and similarly for txt and tsv
load handling
The graphs and csv files are generated on the fly, and only one is generated at a time. Connection requests are queued. As part of the normal web management system, the trusted IPs queue is always processed first so constant access from untrusted sources will not stop access from trusted sources. If the queue is full the connection is not accepted. The most load applies when archiving, but tools like wget fetch one linked file at a time which is ideal.
Graph scores
Graphs are scored based on settings in the config. Each 100 second sample has a score which is included in the csv and xml lists for any graph. The score is also totalled for a graph as a whole and included in the csv and xml list of all graphs. This total is done by multiplying the last score by 864, the previous by 863, and so on for the previous 24 hours.
Creating graphs, and graph names
Graph names are text and up to 20 characters. Only letters, numbers, @, -, and . are allowed. All other characters are removed. It is recommended that names complying with this are used.
Graphs can be defined in some configuration settings such as interface names.
Graphs can also be created dynamically in some cases, e.g. L2TP based graphs are made based on the Chargeable-User-Id, Calling-Station-Id or User-Name for a connected line, and so can be defined from the RADIUS authentication response. It is recommended that the circuit ID is used where available, e.g. from BT platform RADIUS.
The number of graphs is limited depending on memory, but the design is to allow for 100,000 distinctly named graphs. Dynamic circuits simply do not have graphs on them if this number is exceeded. Graphs not used for more than the data retention time are discarded automatically.
SNMP
TBA: We may in future allow some graphs to be polled by SNMP.
Ping
Models of the FB6000 that are designed to do constant pinging, e.g. the FB6102, allow graphs for pings using <ping.../> in the config, but they also allow pings to be started and stopped using a web interface. The Ping menu will show under config as well allowing input using a web form.
To start a ping, send a GET or POST to the /ping URL with form fields. The username or password can be used instead of a normal session tracked login. The user has to be logged in to control ping settings.
The field graph must be specified with the graph name for the pinging
To start a ping also specify ip and optionally table. This can be used on an existing graph to change the ping target. To stop a ping omit ip.