Coherent PSK Demodulation on HF

This post is rather technical, and assumes a knowledge of PSK demodulator design. I apologise if it is difficult to understand for the general reader. I have spent the last few weeks working on this part time so felt compelled to record the results somewhere. Thanks to Bill Cowley VK5DSP and Peter Martinez G3PLX for their email advice on this work.

I have a background in modems for satellite communications, which use non differential PSK and coherent demodulation. This has a 3dB advantage over DPSK, at the cost of additional complexity. The FDMDV modem used for FreeDV uses Differential Phase Shift Keying (DPSK). This is the usual choice for HF radio channels which have phase distortion due to multipath propagation. However 3dB is a big potential improvement, so I couldn’t help wondering if coherent demodulation would work for HF radio channels. So I wrote some Octave code to try it.

First I needed to develop a phase estimation algorithm that could be bolted onto the FDMDV demodulator, but using the same DPSK modulator and over the air specification. True coherent demodulation requires a unique word to resolve phase ambiguities. This isn’t possible with the current FDMDV modem specification as there are no spare bits. So I went for a pseudo coherent scheme where we coherently demodulate the PSK symbols, then pass them to a DPSK decoder. This has a performance hit compared to coherent PSK, but resolves the phase ambiguity without a unique word.

The function rx_est_phase() in fdmdv.m estimates the phase over a window of Nph symbols.

As a first step I plotted the scatter diagram of DPSK (top) versus pseudo coherent DPSK (bottom) for data from a real HF channel.

The psuedo coherent scatter plot looked a bit better to me so I decided to go a little further. I implemented a demodulator simulation that could measure bit error Rate (BER) for AWGN channels. During development I found that the phase estimator couldn’t be used to track frequency offsets larger than 0.5 Hz. I think this is because of the low 50 Hz symbol rate. So the existing DPSK demodulator was run in parallel to provide frequency offset tracking.

Here are the BER results for two Eb/No values (5 and 7 dB) for the two different algorithms.

Demod 7dB 5dB
Differential 0.0128 0.0487
Pseudo Coherent 0.0068 0.0252

The bit error rates are about half, which is a 1dB improvement. This is not very much, but I decided to “run it to ground” and test using some FDMDV modem data from real HF channels. Mark VK5QI kindly gathered these samples for me. Rather than Codec 2 data, a known test sequence is transmitted so BER can be determined at the receiver. The fdmdv_demod_coh.m script implements a demodulator that uses sample files as input.

Here are the results for 50W Tx power (VK2-VK5_20m_50W_TX.raw) using 1400*10 bits. The “Nph” parameter is the size of the window used to estimate the phase. More symbols means a smoother estimate but a slower response to phase variations.

Demod BER
Differential 0.0541
Pseudo Coherent Nph=5 0.0517
Pseudo Coherent Nph=7 0.0484
Pseudo Coherent Nph=9 0.0442
Pseudo Coherent Nph=13 0.0458

Not a very impressive improvement, at best a 20% improvement in BER. The results with 18W of Tx power (VK2-VK5_20m_18W_TX.raw) using 1400*10 bits are also marginal:

Demod BER
Differential 0.0991
Pseudo Coherent Nph=9 0.0926

On the HF channel we have multipath pushing FDM carriers down into the noise. My guess is that the HF channel can be modelled as either good, where the SNR is high, or bad, where a fade knocks out one of the FDM carriers entirely. A small improvement in demodulator performance only affects the transition between the two states.

So, the next step in improving FreeDV performance is to explore Forward Error Correction (FEC).

My First FreeDV Contact

A very pleasant Ham Radio day. My friends Joel and Mark (VK5QI) visited my home to build Peter Parkers (VK3YE) “Porta 40” DSB receiver (from the November 2012 issue of “Amateur Radio” magazine). Joel did the assembly work, with Mark and I helping test the receiver.

We started with the local oscillator, and checked it could be heard on a nearby SSB radio. We then built the RF Amp, mixer, and AF Amp. We tested the mixer with the use of a signal generator (an Arduino controlled DDS) and an oscilloscope and verified the mixer loss was just a few dB. An oscilliscope was used to verify each amplifier stage had gain. At the end of the day we connected the receiver to an outdoor antenna and were surprised to hear Ham radio signals on the 40m band! It felt very satisfying – I think we were all a bit surprised that it worked! I enjoyed working with Joel and Mark – we all brought different skills to the project and worked well as a team. I took care of catering – cooking a nice curry for lunch and keeping the coffee flowing.

First FreeDV Contact

For the last few days I have been trying to make some contacts on the HF bands using my FT817 5W radio and various antennas, including a long wire (with antenna tuner), a commercial end-fed trap diople, and a magnetic loop. The long wire and dipole are propped up at one end with a 7m “squid pole” type fibreglass fishing rod:

I could hear quite a few signals, but no one could hear me. I think the problem was low power and low antenna height. This WebSDR receiver located about 800km away has been very useful. It lets me visualise big chunks of the band for signals I could transmit CW and SSB to it to test various antennas, listening to the results through my laptop.

Mark brought along his 100W Icom HF SSB Radio to try a little more power. While connecting his radio to my antenna we noticed some activity on the 20m band that sounded like FreeDV. We connected his radio to my laptop and could visualise the BPSK sync part of the FreeDV signal, but it was too weak to decode. However we did learn that the band was open to VK2 (about 1500km away), so Mark IM-ed Brenton, VK2MEV, and we started a FreeDV contact, using about 10W of power at our end.

We experienced SNRs in the 5-10dB range with about 80% copy. Some speech was lost when the fading got really bad. The fading on the channel was changing all the time, fast to slow and back again. I think the other Hams running FreeDV just before us were Peter and friends, as described in Peter’s FreeDV blog post today.

Being a speech coding guy I was initially unhappy with the coded speech quality but after a while I seemed to adjust and it seemed just fine. The microphone EQ was useful in improving Brenton’s voice. FreeDV was easy to operate and the colourful GUIs make it interesting and fun.

The tx/rx switching was slower than Push To Talk (PTT) SSB, due to internal latencies and modem sync times, plus the VOX delay. It would be hard to do any real time break in, but didn’t affect our conversation. We switched to SSB to compare and were struck by the “hiss and crackle”, with fading on the analog audio also obvious.

Later in the day Mark and Brenton sent some test data frames over the same channel. I will use this test data to start optimising FreeDV for low SNR channels. Having known data means I can measure the bit error rate, and even extract patterns of bit errors that I can apply to off-line simulations of the system. I can then compare different demodulation and FEC schemes before returning with the best candidates for some real world testing.


For the last 2 months I have been working with Dave Witten KD0EAG, coding a GUI application called FreeDV. It combines Codec 2 and the FDMDV modem into single, user friendly application that runs on Linux and Windows. It enables anyone with a SSB radio start using digital voice.

It works really well. FreeDV uses just 1100 Hz of bandwidth, much less that the 2400 Hz required for an analog SSB signal. Compared to SSB it provides a “noise free” audio experience, and continues to work during fades and multipath at quite low SNRs. Mel Whitten has experimented with many Digital Voice systems over the years. This practical experience has led to the current design – a fast sync, no FEC, low latency system that gives a “SSB” type feel for operators.

Here is a video showing FreeDV in action, with analog SSB for comparison:

It’s been a long time since I did any GUI programming and I found it a nice change from the command line signal processing work that I usually do. The programing problems I had to solve didn’t involve maths or complex signal processing algorithms. However bringing FreeDV to life has it’s own special problems, for example spending hours messing with wxWidgets “sizers” to get a check box positioned just right! It was also much larger than the usual program I work on, so there was a certain complexity navigating large files and keeping several balls in the air at once.

I have also really enjoyed working with a nice team of guys, including Dave Witten, Mel Whitten and Bruce Perens. Also involved were a wonderful group of alpha testers and kind people helping us document, support, and improve FreeDV. One example is this fantastic FreeDV Getting Started video produced by Tony, K2MO.

I also feel a sense of importance in our work – FreeDV is the only open source digital voice system for Amateur Radio. It’s an opportunity to prevent Ham Radio (and digital voice over radio in general) being “locked down” to proprietary codecs.

Over the next few months we will gradually improve FreeDV. In particular I would like work on improvements to the low SNR performance. In the medium term I am interested in other applications for narrowband digital voice over radio, such as telephony in the developing world. Ham Radio is an ideal test bed for refining the algorithms and experimenting with integration of the various buildlng blocks.

Codec 2 wins the ARRL Technical Innovation Award

I’ve won the ARRL Technical Innovation Award for 2012 for my Codec 2 work.

When I first became interested in Ham Radio as a 12 year old in the late 70’s my grandfather bought me the 1979 ARRL handbook. Quite an honor to one day be contributing back to this fine hobby that was my start in a communications and electronics career.

That version of the handbook even had a chapter on “Narrow Band Voice Modulation” – an esoteric analog technique to compress speech by removing chunks of audio bandwidth. Who would have thought that 30 years later I ‘d be contributing in the same area…….

The Codec 2 project is moving along nicely. Recently I have been working on integration of the FDMDV modem with Codec 2, and have written a GUI program called fl_fdmdv to help me debug the combined system. Here is a screen shot (click for large version):

This looks really cool as the graphics update in real time, a static image doesn’t really do it justice. Displaying parameters in real time has helped me spot a few bugs, which I missed with the static plots I get from the Octave simulations. I have used fl_fdmdv to send the Codec2/FDMDV signal over an audio cable between two laptops. It’s really exciting to see the bits being modulated onto the waveform on the GUI while listening to audio flowing over the system! Next step is to replace the audio cable with SSB radios and do some over the air testing.

I am also working on a high quality version of Codec 2 at about 4000 bit/s. The target is speech quality similar to CELP type algorithms such as g.729/Speex/Opus that run at 8000 bits/s. The main application is VOIP, but it might also be useful for a “FM quality” mode for VHF digital radio. The key to high quality is quantising and transmitting the phase of the sine waves used by Codec 2 to model speech. The challenges are working with phase (modulo 2-pi hurts my head) and the time varying number of sine waves and hence phases that must be transmitted. The technique I am currently working on is “sparse vector quantisation of phases”. Tough work but I am slowly making progress. This work is being generously supported by a company who wishes to remain anonymous – but I wanted to thank them anyway!

FDMDV Modem Page

I have added a FDMDV modem page to this web site, plus typed up a README_fdmdv.txt that explains all the files related to the modem and how to use them.

This modem was “lost” a few years ago as the initial implementation was closed source. So I felt compelled to put a fair bit of effort into documenting the open source implementation I have been working on. Building this modem was fun, just hard enough to be challenging but no real show stopper bugs and didn’t hurt my head like Codec 2 algorithm development.

Next step is to tweak Codec 2 to make it interface cleanly to the modem for some initial on-air tests.

Testing a FDMDV Modem

A key use for Codec 2 is digital voice over HF and VHF radio. A few months ago I figured we needed to get Codec 2 on the air. With a PC, codec and modem software, two sounds cards, and a Single Sideband (SSB) radio it is possible to send and receive Digital Voice (DV) signals over HF radio.

This requires a HF modem optimised for digital speech, in particular fast sync, no multi-second training sequences, the ability to recover quickly after a fade, and no automatic re-transmit of “bad” packets. FDMDV was a working system for HF Digital Voice from a few years ago, so seemed like a good starting point. It embodies a lot of experience from Digital Voice pioneers like Mel Whitten.

FDMDV stands for Frequency Division Multiplexed Digital Voice. A FDM modem is a basically a bunch of slow modems running in parallel. For example FDMDV has 14 carriers spaced 75 Hz apart, each running at 50 symbols/second. Due to multipath problems on HF this approach works better than one carrier running at 14×50 = 700 symbols/second. On each symbol is encoded two bits using differential QPSK, so the bit rate is 1400 bit/s.

A few months ago I started experimenting with GNU Octave simulations of parts of the FDMDV modem. One thing led to another and I ended up writing an open source version of the FDMDV modem, based on the FDMDV spec.

I am in the final stages of the C version of that modem, currently writing command line demo programs. I am not sure what the “best” HF DV system would look like (Codec/FEC/protocol/modem) but I feel the best way to find out is build something and iterate on it. Rather than concentrating on the Codec alone I wanted to get some real world HF DV experience to tune and evolve the system as a whole.

The cool thing about open source is it attracts the best in the field. I have been in regular contact with HF modem gurus like Peter Martinez G3PLX (PSK31), and Rick Muething KN6KB (WINMOR) who have been very helpful with suggestions and support as I re-implemented the FDMDV modem. Rick also has some great ideas for more advanced modulation schemes (trellis coded PSK) that would be nice to try later. I have also had some great help from Bill Cowley, who has 25 years of PSK modem experience.

Testing the Modem

After developing the modem algorithms for about two months using GNU Octave I was ready to test over a real HF channel. So a few days ago I sent a wave file of the modem signal to Mel and Tony (K2MO). They kindly played the tones over a 925 mile HF channel and sent me a recording of the received signal.

I ran the files through my FDMDV modem code. On the first pass the scatter diagram was a mess and the Bit Error Rate (BER) was about 10% – suspiciously high.

Then I noticed the timing offset was changing very quickly, as you can see in the plot below:

The demod estimates the best time to sample the received symbols. This is known as the “timing offset”. In the real world the sample clocks used at the transmitter and receiver tend to be a little different, for example 8000 and 8001Hz. In our case the sample clocks are in the sound device hardware used to play and record the modem signals. So we expect the timing offset to drift a little. I had been simulating just such problems during the modem development, for example testing clock differences of up to 2000 ppm (16Hz at an 8000 Hz sample rate).

Now the demod code keeps an eye on the drift in the timing estimate, and reshuffles buffers every now and again to keep them from overflowing. Hence the saw-tooth effect.

If we count how many “teeth per second” in the saw-tooth, we can estimate the difference in the transmit and receive sample clock. I estimated about 2.5, of 40 samples each. So in every second that’s 2.5×40 = 100 samples, or a 100Hz difference, or 12500ppm! It’s like the PC playing the signal was at 8000Hz and the sample rate of the PC receiving the signal was at 8100Hz.

I re-sampled the signal to correct the large sample clock offset using Sox:
sox -r 8100 -s -2 for_david.raw -s -2 for_david_8000hz.raw rate -h 8000

and the results were perfect – 0 bit errors except for when there was SSB interference across the signal! This was very exciting for me – the first verification that my modem actually worked over real HF channels.

Turns out some sound cards can’t accurately sample at 8000Hz. This was something I had been warned of by my HF modem brains trust. The solution is to use the 48000Hz sound card rate, which most soundcards seem to be better at.

FDM Modem in Action

Here are samples of the first 5 seconds of the transmit and receive modem signal. Now look at the spectrogram of the received signal:

Time is along the x axis, frequency along the y axis. The “hotter” the colour, the stronger the signal. Our FDM signal is the parallel red lines between 600 and 1700Hz. Above the modem signal is some analog SSB. You can hear this as the high frequency “Donald Duck” sound in the received signal. Now around 2.5 and 3.3 seconds there are strong bursts of SSB right on top of our signal, in the 0 to 1100 Hz range.

So how does our modem do? The “Bit errors for test frames” and “Test Frame Sync” plots below tells the story:

Look at the centre plot, it is a measure of bit errors for each test frame received by the demod. Between 2.5 and 3.5 seconds you can see several error bursts. However the demod recovered quickly after the SSB interference. The BPSK sync and test Frame sync plots are unbroken, indicating our demo didn’t “lose it” during the interfering burst. If this was Codec (digital voice) data we would hear some degraded speech, but the system would soldier on between interfering analog SSB bursts. Just what we want.

This sample also shows some of the accumulated wisdom that went into the FDMDV system design. It is a narrow signal (just 1100Hz), so less sensitive to interference to adjacent users on the busy HF bands compared to a system using a full SSB bandwidth of say 2400Hz. Narrow band means we can pack more energy into fewer carriers. The signal to noise ratio is relatively high, and the BER due to gaussian type channel noise (AWGN) practically zero. Rather bit errors come from adjacent users and mutipath fading effects (the latter not illustrated here).

Next steps are to integrate the Codec and Modem into an easy to use GUI program for Windows and Linux. This will help us obtain some real world experience which we can use to tune and further develop the entire system.