Building and testing the DSP: technical details¶

This is the technical companion to Building and testing, the plain-English overview. Start there; this page keeps the full detail: the code's names, the Recommendations' clauses, the numbers and the test lists.

Building¶

cmake -B build                            # in the top directory
cmake --build build

There is no i386 requirement. D-Modem builds natively on x86_64, ARM and other architectures, and links with -lm. Clang and GCC both build it.

Only the interop tests need a 32-bit build (-m32, with 32-bit libraries: g++-multilib on Debian or Ubuntu), because the original library and the other vendors' modems are 32-bit i386 code. Without it those tests are skipped.

Other C libraries¶

With a C library other than glibc (musl on Alpine, Windows', Apple's), the test programs take glibc's rand() sequence from dsp/test/libc_compat.h. Their noise and random fax pages are then the same draws the tests' thresholds were set with; with a different sequence a couple of the spandsp tests fail on musl. libc_compat.h is included ahead of every test source in dsp/test/; with glibc on Linux it is empty.

CI's musl job builds and runs the offline tests on Alpine. The tests there need coreutils (for stdbuf) and tiff-dev (for spandsp's headers).

Windows¶

A Windows build (MinGW-w64, -DCMAKE_TOOLCHAIN_FILE=platforms/windows/mingw-w64.cmake) runs its tests under Wine: ctest --test-dir build-win -L quick. The interop tests are left out there, since netsim is not built for Windows (windows.md).

Tests¶

Running them¶

ctest --test-dir build -j16         # all of them (-j about four times the
                                    # processors: about 1.5 min on 4)
ctest --test-dir build -L quick     # back-to-back pumps, libdmodem, fax
ctest --test-dir build -R doc_check # every function has a comment above it
                                    # (cmake/doc_check.py; run it on its own as
                                    # python3 cmake/doc_check.py . [FILE...])
ctest --test-dir build -L spandsp   # against spandsp (libspandsp-dev)
ctest --test-dir build -L smartlink -j16  # calls between Smart Link's soft modem
                                          # (its original library) and this one,
                                          # data and class 1 fax (needs -m32)
ctest --test-dir build -L agere -j16      # calls between the Agere soft modem
                                          # and this one
ctest --test-dir build -L flaky     # only the known flaky tests
                                    # (-LE flaky: all but them)

The labels for the interop calls are one per peer:

Other labels: quick (the offline tests), spandsp (spandsp's transmitters and T.30 against ours), alsa (dmodemd on emulated hardware), osmocom (V.110 against libosmocore, when it is configured), flaky and logs (the log fixture's own tests).

The flaky tests¶

The known flaky tests are listed in cmake/flaky_tests.cmake. Each is an open intermittent problem, and most fail only on a loaded machine. They run last and one at a time:

• Every other test requires the fixture dmodem_steady, whose cleanup test dmodem_flaky_gate runs once they have all ended.
• The flaky tests depend on dmodem_flaky_gate and are RUN_SERIAL.
• Each keeps its own labels and has flaky as well.
• Run on its own (-R), a flaky test waits for nothing.
• A gtest program's flaky cases are found by a discovery of their own (TEST_FILTER, CMake 3.22 and later; with older CMake they run with the others).

When one of them is fixed, take it off the list. A new one goes on the list with its task number.

Test names¶

The test programs are in dsp/test/; the link layer's and the core's are in link/test/ and core/test/, and dmodemd's and the Windows driver's in frontend/dmodemd/test/ and platforms/windows/winmodem/test/.

Most of them are GoogleTest programs (the system's libgtest-dev, or fetched while configuring). ctest runs each of their cases as a test of its own, named <suite>/<Test>.<case>/<parameters>, for example Line/PumpLoopback.DataBothWays/v34_echoes. The interop calls are named <label>.<call>, for example agere.agere_v92_qc. The others are scripts and commands added straight to ctest (doc_check, v44_preset_check, compress_bench_check, the dmodemd_* Python tests, and so on).

Run one test by itself with ctest -R v34_echoes, or run the program directly with --gtest_filter.

The test programs¶

The data pumps and their parts (dsp/test/, label quick unless noted):

The fax (dsp/test/):

The details of the fax tests are in Fax.

libdmodem (dsp/test/, label quick):

The link layer and the core:

The interop calls (dsp/test/, one program per label):

dmodemd and the hardware (frontend/dmodemd/test/):

The Windows driver (platforms/windows/winmodem/test/):

Test logs¶

A ctest run with a failure keeps the logs of the tests that failed, in a directory of its own under build/test-logs/, named for when the run started (latest points to the newest). A run without a failure leaves nothing there; skipped tests are not failures.

build/test-logs/2026-10-16_21-34-03/
    run.txt                     when, the ctest command, the commit, the host
    results.txt                 each test as it ends: PASS, FAIL or SKIP, its exit
                                status and seconds
    summary.txt                 the counts, then results.txt, failures first
    open.open_v90a_ulaw/        a test that failed (or was skipped, or killed)
        output.log              what it printed (ctest's output of it)
        info.txt                its command, start, debug levels, environment
        result.txt              its line of results.txt
        modemA.log, modemB.log  a call's modem logs (dmodemd_alsa: exchange.log,
                                dmodemd-<line>.log)
    open.open_v90a_ulaw.2/      the same test again in the run (ctest --repeat)

This works with plain ctest, including -R, -L or --repeat. A fixture does it (cmake/test_logs.cmake):

1. test_logs_begin makes the run's directory before the first test.
2. Every test runs under cmake/test_run.sh, which keeps its output and still passes it on to ctest. A test finds its directory in DMODEM_TEST_DIR.
3. When a test passes, the launcher deletes its directory at once. A test without its result.txt was killed (for example at its ctest timeout) and counts as failed.
4. test_logs_end writes the summary after the last test. When the run ended without a failure, it deletes the run's directory and points latest at the newest run left.

Settings:

• DMODEM_TEST_KEEP_PASSED=1 keeps everything (some 100 MB for a full run).
• The 20 latest runs are kept (-DDMODEM_TEST_LOG_KEEP=<n>).
• -DDMODEM_TEST_LOG_ROOT=<dir> puts them elsewhere.

Debug levels¶

Every run logs verbosely. The launcher sets the debug level of every modem and library the tests run to DMODEM_TEST_DEBUG (5 by default), wherever it is not already set:

• DSP_DEBUG (the pumps), CORE_DEBUG, DMODEM_DEBUG (libdmodem), DBG (the interop calls' modems), DMODEMD_DEBUG (dmodemd's -d);
• LINK_DEBUG and SPANDSP_DEBUG are switched on.

Level 5 is what the modems log of each step of a call, with their equalizers' and retrains' figures as they go. Levels 6 and up log every bit and symbol (DSP_DEBUG=6 ctest -R ... for one test). DMODEM_TEST_DEBUG=0 leaves the tests' own levels. With all of this, a full run takes about a tenth longer.

A call run by hand (interop_call, or a test program outside ctest) writes its modem logs to LOG_DIR, or to build/dsp/test/logs/<test>/ for the test programs.

How the calls keep time¶

The interop calls run as fast as the modems can. Every modem keeps time by its line's samples: the Agere, SM56, PCTel, HSF and Broadcom harnesses count jiffies and timers in samples, and our core, under the open pumps or the original library, does so with PIPE_SYNC. So a call goes as it would in real time, only sooner, and a busy machine only slows it down.

The call's own seconds (STREAM, MIDCALL, WAIT, ANSWER_AFTER, HANGUP_WAIT, and how long the driver waits for what the modems say) are then the line's seconds, from netsim's clock (NET_CLOCK_FD). They run faster or slower than the wall clock as the machine allows. The wall clock is used only while the line stands still (a modem has gone).

Three things keep such a call the same from run to run, however late the driver answers:

• On hook, our modem and the original library run in real time anyway (PIPE_SYNC), so the driver's commands meet the samples they would in real time. The call itself, from the dial or the answer, runs as fast as it can. Their line also runs from the start, as the other harnesses' lines do: the lead silence is written when they start, not at the first ATD or ATA. A line that stood still until then took the commands as they came, and the call began wherever the scheduler had the two modems at that moment.
• The driver waits 100 ms of the line after a CONNECT before sending data (DATA_WAIT). The SM56 says CONNECT a little before it takes data, and echoes anything that comes sooner as a command line.
• The data around a call's mid-call commands (MIDCALL) is kept: before the escape's OK, and right after ATO's CONNECT.

Fax calls¶

The fax calls (interop_call --fax) keep time by the driver's own relay of the line (A's samples at LINE_RATE: 8000, or 9600 with the original library). They go at most ten times as fast as real time (MODEM_SPEED=10, for half_open and half_smartlink):

• The fax modems use so little processor time that the line would otherwise run a few hundred times as fast. The milliseconds the driver takes to answer a modem (it plays T.30 for both machines) would then be most of a second of the line, past T.30's turnarounds.
• The modems take the DTE's commands every 192 samples (PIPE_SYNC=192: 20 ms at 9600 Hz, the original library's frame). The original library's V.17 receiver takes a short train or not depending on how its frames fall on it, so our training has to start at the same point of those frames every time.

So the calls take the same turnarounds the real-time calls did and give the same TCF and pages, in a tenth of the time.

The relay also holds the line while the driver has not yet answered what a modem said (from the modem's output until the driver's next poll or write). However late the scheduler lets the driver run, its answer meets the line where it would have at once. (Before this, the Agere modem's +FTH ended in ERROR now and then, its frame arriving some seconds of line time after its CONNECT.)

Parallel runs¶

The calls run a little nicer (nice 10) than the rest of the machine. Each takes about one processor while it runs, except for the moments on hook in real time, so -j can be about four times the number of processors. On 4 processors the whole suite takes some 75 s at -j16 (105 s at -j8; 13 minutes when every call ran in real time). Much more than that (-j32 on 4) leaves the calls on hook short of real time. None of the calls has to run alone.

MODEM_REALTIME=1 ctest ... runs every call in real time, as they all did before (useful for comparing a failure).

Tools for trying a line by hand¶

The test programs are in build/dsp/test/.

interop_call¶

The calls go through interop_call.cpp; the interop_call program makes one by hand:

interop_call <caller> <answerer> ["ATX3;AT+MS=122"]
interop_call --fax <sender> <receiver> [mod] [AT setup]

The call's settings come from the environment (interop_call.h: NET, STREAM, BULK, CALLS, EXPECT_B, ...; NETSIM for netsim's path and LOG_DIR for the logs, by default the current directory).

A call connects two modem processes with pipes: half_open (this code on the modem core, ../core) and another modem's harness, or a second half_open (the open label). Each goes through the modem's whole controller: AT commands, V.42/LAPM detection and data in both directions. half_open8 is the open modem at 8000 Hz (the original library does not work at 8000 Hz). The other modems, their harnesses and the results of calls with them are in interop.md: Smart Link's original library (half_smartlink), the Agere, SM56, PCTel, HSF, Broadcom, Intel and ESS soft modems, and spandsp's modems.

dsp_loopback¶

dsp_loopback <dp> [snr dB] [seconds] [srate] [line gain] [clock ppm]
             [echo dB] [far echo dB]

dsp_loopback runs a caller and an answerer through a simulated line with delay, a low-pass filter, white noise and an optional sample clock offset. It then checks the bit error rate in both directions. It is the simulation pump_test uses, for trying a line from the command line.

V.34 between two open modems (dsp_loopback 8 ..., with the low-pass filter, no bit errors):

With FAR_DELAY set to 200 ms, it still runs at 33600 bit/s.

netsim: the simulated network¶

NET=<rate A>:<rate B>:<ulaw|alaw> puts netsim, a simulated G.711 network, between the two modems. Each direction goes through the G.711 law at 8000 Hz. A modem at 9600 Hz is reached through a converter that acts like a codec on an analogue line: a raised cosine at 4000 Hz, so that PCM samples stay free of intersymbol interference. NET_PLAIN=1 uses dsp_util's short resampler instead, which is not free of it, so V.90 falls back to V.34 through it.

Analogue modems at 8000 Hz¶

NET_ANALOG=a (or b) with NET=8000:8000 makes A (or B) an analogue modem with its codec at 8000 Hz (the raised cosine sampled at NET_PHASE, NET_PPM). MODEM_ANALOG=<name> (or 1) gives that half_open an analogue line (Host::analog_line(): it dials there, and its pumps are the analogue modem's).

Mobile speech codecs¶

NET_CODEC=amr:12.2 (AMR at 4.75 to 12.2 kbit/s) or NET_CODEC=amrwb:23.85 (AMR-WB at 6.6 to 23.85 kbit/s) puts a mobile network's speech codec on the 8000 Hz line in place of G.711. Each direction is encoded and decoded frame by frame (dsp/test/codec_sim.cpp, on the OpenCORE and VisualOn libraries: Debian's libopencore-amrnb-dev, libopencore-amrwb-dev and libvo-amrwbenc-dev; without them netsim says so).

• :dtx after the rate turns on the codecs' discontinuous transmission.
• NET_CODEC_LOSS=<percent> erases that share of the frames at random (NET_CODEC_SEED), as lost packets would.
• codec_probe runs a file of samples through the same codec, for measuring what it does to a signal.

What the modulations and the codecs make of each other is in cellular/, a proposed Recommendation for data over mobile telephony's speech paths (v_cvm.pdf).

Impairments toward the 9600 Hz modem¶

• NET_GAIN sets an analogue gain on the way to the 9600 Hz modem.
• NET_PAD=<dB> puts a digital loss pad on that way: each code is re-encoded that much lower.
• NET_RBS=<mask> adds robbed bit signalling: in the samples whose count mod 6 has its bit set in the mask, the octet's least significant bit is forced to 1, as on T1 trunks.

Pads and robbed bits are what V.90's DIL finds (v90a.md).

The 9600 Hz modem's codec is on the network's clock, with its samples on the network's symbols, unless:

• NET_PHASE=<fraction>: its samples are that fraction of a network sample late; or
• NET_PPM=<ppm>: its clock is that many parts per million fast, both ways.

This models a converter whose output times move with the clock (netsim's raised cosine toward the modem, a 3900 Hz low pass from it), which V.90's PCM receiver has to follow.

Echoes¶

A modem's own echo from its line's hybrid is the modem's to add, because netsim has the samples in blocks:

• DSP_TEST_ECHO=<dB>:<ms>[a|c] makes an open modem at 9600 Hz receive what it sent <ms> before, <dB> down, through a high pass at 300 Hz and a low pass at 3400 Hz (a or c: only the answerer or the caller).
• NET_ECHO=<dB>:<ms> adds the echo of a hybrid at A's line, in A's own sample clock: what A receives comes back with what it sends, taken from the line before A's codec would clip it.

The NET_ECHO delay has to be more than the modem at A sends ahead of what it has received (some 20 ms for the HSF modem). Otherwise netsim says how many samples went without their echo; an echo with gaps in it is not one a canceller can learn.

The fax calls and the fax modulation tests¶

The fax calls (<label>_fax_*) work like the data calls, with AT+FCLASS=1. interop_call --fax plays the T.30 side of both fax machines: CSI/DIS, DCS, TCF, CFR, one page (for V.17, with the short train), EOP, MCF and DCN, at up to ten times real time (see Fax calls). It accepts a page with a bit error rate below 1e-3 (a fax page without ECM survives that), because the original library makes a few bit errors with some of its own signals.

fax_test's Class1/FaxModulation.* cases send random data through each fax modulation over a simulated line:

• The V.17 short train codes run a long train first, so the receiver has an equalizer to start from.
• A few V.17 14400 cases (_page<seed>) send a test page's MH bits instead, as do spandsp_test's _page cases from spandsp's transmitter.

A page's runs of the same few patterns move the receiver's loops in a way that random data does not, so a page can lose a row where random data gets through. The seeds chosen are pages that did lose a row before the receiver's AGC and timing loop were slowed down in the data.