In the previous post on Codec 2 2200 I described a prototype speech codec for a new, higher quality FreeDV mode. In this post I describe the modem development and initial over the air tests.
Bandwidth, not SNR limited
Turns out I am RF bandwidth limited rather than SNR limited. My target bandwidth over the channel is 2000 Hz, as FreeDV needs to pass through the various crystal filters in Commercial Off the Shelf (COTS) SSB radios. Things get messy near the edge of those filters. I want to stick with QPSK for now as QAM means a 4dB hit in SNR at the same bit rate. By messing with my waveform design spreadsheet it turns out I can get between 2000 and 2400 bit/s of coded speech through the channel by reducing the FEC code rate. This is acceptable for high-ish SNR channels with light fading.
This gives us an AWGN channel operating point of 4dB SNR, so we have 6dB margin for fading if we design for an average channel SNR of 10dB. That feels about right.
For the speech quality target I wanted to used 20ms frame updates (the lower rate Codec 2 1300 and 700C modes use 40ms updates). However that was a bit of a squeeze. I messed about with Vector Quantisation for a while but the quality was slipping beneath the Codec 2 2400 bit/s target I had set for this project. Then I hit on the idea of 25ms updates, and that seems to work well. I can’t hear any difference between 20 and 25ms updates.
Note the interaction between the speech codec frame rate and the modem bandwidth. As the entire system is open source, I can adapt with the codec to help the modem. This is a very powerful engineering technique that is not available to teams using closed source codec or modem software.
FreeDV 2200, like FreeDV 1600 has some “unprotected bits”. The LDPC forward error correction only covers part of the frame. I’m not sure about this approach, and will talk to Bill, VK5DSP sometime about a lower rate code that protects the entire frame.
A digital voice system is complex. Building the real time C code for a new codec, FEC, modem stack can take a lot of time and effort. FreeDV 700D took around 1000 man-hours. So I tend to simulate the whole thing in GNU Octave first. This allows me to prototype the entire system with the least possible effort. If it works, I then port to C, and I have a working reference to test the C port against. The basic idea is find any bad news early.
I modified the Octave version of the modem to handle the different bit rate through changing the number of carriers, symbol rate, and FEC. I started by modifying the raw, uncoded OFDM modem simulations (ofdm_tx.m, ofdm_rx.m), then moved to the versions that include the LDPC FEC.
After a few days of careful programming, testing, and refactoring I was able to transmit and receive test frame using the prototype “FreeDV 2200” waveform. Such a surprise when something complex starts working! Engineers are not used to success, we spend most of our time chasing bugs.
Using the new waveform I ran some coded speech through simulated AWGN and HF channels at the calculated operating points. Ouch! That hurts. Some really high amplitude excursions in the decoded speech, especially when the frames are messed up by a fade or during initial synchronisation.
These sorts of noises used to be common in the early days of speech coding. In contrast, my recent codecs have a “softer” response to bit errors – the speech gets messed up, but not in a way that causes pain.
OK so my initial tests showed that even with 10dB average SNR, sometimes that nasty old HF channel wipes out the codec and it makes loud cracks. What to do? I don’t have any bandwidth left over for more FEC and don’t want to dramatically change the Codec design.
The first thing I did was change the variable bit rate allocation Huffman code to a fixed bit allocation, as explained in the Codec 2 2200 blog post. That helped. The next step was to spent a few days developing some “error masking” ideas. These don’t actually fix the problem, but make it sound more acceptable.
The LDPC decoder kindly tells us when it can’t successfully decode a frame. When that happens, the masking algorithm swings into action. We know some of the bits are rubbish, but not exactly which ones. An AGC algorithm adjusts the energy of the current frame to be similar to the last good one. This helps with the big spikes. If we get a burst of errors the level is smoothly reduced, effectively squelching the signal.
After a few days of tweaking I had this working OK. Here are some plots of the speech signal before and after masking:
Here are some plots that illustrate the masking algorithm. The top plot is the number of errors/frame, they appear in bursts with the channel fading. The lower plot is the energy of the decoded speech before and after masking. When there are no errors, the energy is the same. During an error burst we reduce the gain, exponentially dropping the level if an error burst continues.
Here are some speech samples so you can hear the effect of bit errors:
|HF Channel Masking||Listen|
|HF Channel SSB||Listen|
|HF Channel FreeDV 1600||Listen|
The FreeDV 1600 sample takes a while to sync, perhaps due to that deep fade at the start of the sample. The 2200 modem was derived from 700D, which can sync up at very low SNRs. Much to my annoyance the SSB sample doesn’t seem very bothered by the fading – a testament to robustness of SSB and why it’s so hard to beat. The first few seconds of the SSB sample are “down in the mud” and would probably be lost to untrained listeners.
Over the Air Testing with the Peters
So now I had simulations of every part of FreeDV 2200 ready to roll. The modem simulations work on wave files, which can be played over the air through real HF radios. This is a very important test, as real radios have crystal filters, power amplifiers, and audio filtering that can affect the passage of bits through them.
My first test was across the bench. I played the OFDM signal at low power through my IC-7200 tuned to 7.177MHz, and connected to an external dipole. I received the signal on my FT-817 without an antenna, while recording the signal using FreeDV. I took special care to make sure the IC-7200 wasn’t being over driven, adjusting the transmit drive so the ALC wasn’t moving at all.
The scatter diagram was a bit messed up, and SNR about 13dB, which is pretty typical of tests through real radios, even with no noise. I measure SNR from the scatter diagram, so any distortion in the signal will lower the SNR reported by FreeDV. An important figure of merit is the effect on Bit Error Rate (BER), which can be measured in dB. So I generated a transmit signal with additive noise that had an Eb/No of 3dB. When passed directly to the Rx script I measured a BER of 3%. When the same signal was passed through the radios, then to the Rx script, I measured a BER of 5%.
Looking at the PSK BER versus Eb/No curves, this is a difference of 1.2dB. The distortion of the path through the radios is equivalent to 1.2dB more noise in the system. That’s quite acceptable to me; given the wide bandwidth of the signal compared to the crystal filter skirts, PA distortion, likely rx overload as the radios are so close, and other real world factors.
The next step was some ground wave and DX tests. Peter, VK2PTM, happened to be staying with me. So we hopped in his van and set up a receive station about 4km away, where Peter, VK5APR joined us for the tests. We received a very nice signal on Peter’s (2TPM Peter that is) KX3, in fact a nicer scatter diagram than with my FT-817. Zero BER, despite just a few watts Tx power.
Peter has documented our FreeDV experiments on his blog.
For the final test, we radiated the same test signal from Adelaide about 700km to Peter, VK3RV, about 700km away in Sunbury, near Melbourne. Once again we were on 40M, and this time I was using a few 10’s or watts. Hard to tell exactly with these waveforms. Peter kindly recorded the FreeDV 2200 signal, and emailed it back to me.
You don’t have to be called Peter to be a FreeDV early adopter. But it helps.
This time there was some significant fading, and bursts of bit errors. The SNR averaged about 8dB over our 60 second sample but being a real HF channel the SNR wandered all over the place:
Note the deep fade at the start of the sample, which is also evident in the second plot of uncoded and coded errors above. The Scatter diagram is the classic cross shape, as the amplitudes of the QPSK symbols fade in and out. The channel noise is evident in the width of the cross.
When played back through the speech codec it worked OK (Listen), and sounds similar to the simulated HF channel samples above. The deep fades are gently muted and the overall effect was not unpleasant. Long distance HF conditions are poor here at the moment, so I didn’t get a chance to test the system on more benign HF channels.
I’m very pleased with these tests. It’s a very complicated system and so many things can go wrong when it hits the real world. The careful work I put into test and simulation has paid off. I am particularly happy with the 2000 Hz wide modem waveform making it through real radios and real channels as I felt that was a major risk.
These two posts have described the development to date of a new FreeDV mode. It’s a work in progress, and I feel another iteration on the speech codec quality and FEC would be useful. I’m quite pleased with the higher bit rate version of the OFDM modem. It would also be nice to get the current work into real time, so we can test it on air – using the open source principle of “release early and often”.
The development process for a new FreeDV waveform is not linear. I started with the codec, got a feel for the bit rate, played with the modem waveforms design spreadsheet, then headed back to the codec design. When I put the waveform through a simulated channels I found bit errors made the speech codec fall over so it was back to the codec for some rework.
However the cool thing about open source is that we can consider all these factors at the same time. If you are stuck with a closed source codec or modem you have far more limited options and will end up with poorer performance. Ok, I also have the skills to work on both, but that’s why I’m publishing these posts and the source code! It’s a great starting point for anyone else who wants to learn, or build on this work. Yayyy open source!
An important next step is to put FreeDV 2200 into real time, so we can test with real over the air conversations. This means porting some code from Octave to C, unit tests, and a bunch of integration with the FreeDV API and FreeDV GUI program. Please contact me if you have C programming skills and would like to help. I’ll help you with the Octave side, and you’ll learn a lot!
Appendix – Command Lines for testing the new Waveform
Here are the command lines I used to test the new FreeDV waveform over the air. I tend to prototype in Octave, to minimise the amount of time it takes to get something over the air that I can listen to. A wrong turn can mean months of work, so an efficient process matters.
Use c2sim to generate files of Codec 2 model parameters from an input speech file:
$ ./c2sim ../../raw/vk5qi.raw --framelength_s 0.0125 --dump vk5qi_l --phase0 --lpc 10 --dump_pitch_e vk5qi_l_pitche.txt
Run an Octave script to encode model parameters to a 2200 (ish) bit/s second bit stream:
octave:40> newamp2_batch("../build_linux/src/vk5qi_l","no_output","mode", "enc", "bitstream", "vk5qi_2200_enc.c2");
Run an Octave script to generate OFDM modem transmitter audio. The payload data is known test frames at 2200 bit/s:
Three hundred seconds are generated in this example, about 5 minutes.
This raw file can be “played” over the air through a HF SSB radio, then recorded as a wave file by a HF radio receiver. The received wave (or raw) file, is run through the OFDM modem receiver. Using the known test frame data, it measures the Bit Error rate (BER) and plots various statistics. It also generates a file of errors, that shows each bit that was received in error:
This C utility “corrupts” the Codec 2 bit stream using the file of errors, simulating the path of real Codec 2 bits through the channel:
$ ../build_linux/src/insert_errors vk5qi_2200_enc.c2 vk5qi_2200_sunbury001_dec.c2 subury001_error.bin
Now decode the bit stream to a set of Codec 2 model parameters:
octave:43> newamp2_batch("../build_linux/src/vk5qi_l","output_prefix","../build_linux/src/vk5qi_l_dec_sunbury001", "mode", "dec", "bitstream", "vk5qi_2200_sunbury001_dec.c2", "mask", "subury001_error.bin");
The “mask” parameter uses the error file to implement an error masking algorithm. In a real world system, the LDPC decoder would tell use which frames were bad, so this is a mild cheat to expedite development.
Finally, run C part of Codec 2 decoder to listen to the results:
$ ./c2sim ../../raw/vk5qi.raw --framelength_s 0.0125 --pahw vk5qi_l_dec_sunbury001 --hand_voicing vk5qi_l_dec_sunbury001_v.txt -o - | aplay -f S16