FreeDV Robustness Part 2

Since the last post I have been working on two new FreeDV modes (1600 and 2000 bit/s), designed to improve the performance of FreeDV over HF multipath fading channels. This involved modifying the Octave and C modem code to support a variable number of carriers, a new 1600 bit/s Codec 2 mode, and some C code to implement FEC. Once again I’ve included the command lines I am using for those who want to repeat the simulations. It’s also useful for me to write the commands down so I can find them again!

The 1600 bit/s mode uses simplified scalar pitch and energy quantisers. This makes it more robust to bit errors, but at a cost of increased bit rate. The increase equates to a 10log10(1600/1400) = 0.5dB drop in SNR, which is pretty modest. The 2000 bit/s mode uses the 1400 bit/s codec, plus 600 bit/s of FEC to protect the first (and most sensitive) 24 bits of the Codec data.

Here are the results over a simulated “CCIR moderate” channel with 10dB SNR:

Sample
Baseline 1400 bit/s with no FEC
1600 bit/s Codec 2 with no FEC
2000 bit/s, 1400 bit/s Codec + 600 bit/s FEC
Analog SSB

Out of the digital modes, I think 2000 bit/s is best, with 1600 bit/s quite close. Compared to the analog, the digital has some annoying R2D2 sounds, but lacks the constant hiss. At 2000 and 1600 bit/s my goal of intelligible speech over a HF multipath channel is, I think, achieved. Here is a typical waterfall, bit error pattern, and SNR against time for this channel:

Note the sample files only play the decoded audio for the first 12 seconds, the plots cover 30 seconds.

Here are some results on a “CCIR poor” channel at 4dB SNR, followed by the waterfall, bit error pattern, and SNR plot against time. The poor channel has faster fading and with the low SNR we have a much higher BER (which averages 8% before FEC).

Sample
Baseline 1400 bit/s with no FEC
1600 bit/s Codec 2 with no FEC
2000 bit/s, 1400 bit/s Codec + 600 bit/s FEC
Analog SSB

Although they are all pretty bad, in this case I think the 1600 bit/s version performs best. A QSO might “just” be possible. Actually it’s quite remarkable that we can hear anything given an average BER of 8%! I think the poorer performance of 2000 bit/s on this channel is an example of FEC breaking down at low SNRs/high BERs. The FEC is actually making more bit errors than it corrects! Digital voice is unique in the low SNRs/high BERs it operates in compared to regular data.

The fluttering at the start of the 1600 bit/s sample might be caused by bit errors in the frame energy parameter causing the level to jump around, or possibly bit errors in the voicing bit. If we know the channel is bad (the modem can tell us), we could “lock down” a few bits to enhance speech on poor channels. For example force all frames to be voiced, or smooth rapid changes in the energy. This could even be a “DV noise reduction” checkbox the operator can enable on poor channels. Try doing that with your closed source Codec!

The SSB sample is still very intelligible, perhaps better than the digital if you don’t mind the hiss. We still have a long way to go (maybe 6dB?) to do better than SSB in poor channel conditions!

Command Lines

The samples above were generated by some magical command line incarnations that run simulations of various parts of the system. Pathsim is used to generate modem files corrupted by the simulated HF channel. I then use a GNU Octave simulation of the demod to generate the bit error patterns for that mode/channel combination. For example the 1400 bit/s mode:

octave:18> fdmdv_demod("../src/mod_test_1400_poor_4dB.wav",1400*30,14, "mod_test_1400_poor_4dB.err")
38640 bits  2773 errors  BER: 0.0718 PAPR(rx): 16.20 dB

This means take the mod_test_1400_poor_4dB.wav file as input, demodulate 30 seconds worth of bits at 1400 bit/s, using a 14 carrier modem, and save the error patterns to mod_test_1400_poor_4dB.err. Running the simulation also generates a bunch of nice plots to help me work out what’s going on, some of which are above.

Next step is to insert the bit errors into a Codec 2 bit stream and decode the results. I do this with a bunch of piped command line tools:

~/codec2-dev/src$ ./insert_errors ve9qrp_1400.c2 - ../octave/mod_test_1400_poor_4dB.err 56 | ./c2dec 1400 - - | play -t raw -r 8000 -s -2 -

The “56″ above refers to the number of bits/frame for the 1400 bits/s mode. When testing the 2000 bit/s mode I use additional command line tools for the FEC, for example:

~/codec2-dev/src$ ./fec_enc ve9qrp_1400.c2 - | ./insert_errors - - ../octave/mod_test_2000_poor_4dB.err 80 | ./fec_dec - - | ./c2dec 1400 - - | play -t raw -r 8000 -s -2 -

And finally to generate a 12 second output wave file (this time for the 1600 bit/s mode):

~/codec2-dev/src$ ./insert_errors ve9qrp_1600.c2 - ../octave/mod_test_1600_poor_4dB.err 64 | ./c2dec 1600 - - | sox -t raw -r 8000 -s -2 - mod_test_1600_poor_4dB.wav trim 0 12

Simulating communications systems on the command line. Who needs a GUI?

Off Air Examples

Simulated channels are useful as the channel is the same for each run, so a direct comparison of the 3 modes on exactly the same channel can be made. However I wanted to backup these simulated results with some bit errors from real world, off air HF channels.

So last Saturday I hooked up with Mark, VK5QI, on 40m. It was a pleasant Summer (well early Autumn) evening for us. Mark and Andy VK5AKH set up a 40m dipole up the back of the Morialta Conservation Park in Adelaide, and were using a Codan 2110 Manpack for most of the testing.

I was operating “portable” in a backyard in Geelong, with an end fed dipole strung from an 8m squid pole. I was happy to find that my FT-817 with 2.5W SSB could reach 800km to Mark so we could coordinate the tests. At my end the analog SSB was noisy (beneath my urban S8 background noise level) but intelligible. Just the sort of channel I’d like FreeDV to work over.

Some pictures of Mark’s portable station are here.

I generated some modem files with test data sent using 14, 16 & 20 tones. Mark played these files to me over his tx at about 4W average power and I received and recorded them using the FreeDV “record from radio” feature on the Tools menu. We recorded several samples of each mode – and each sample was quite different due to the ever changing nature of HF radio channels. Later, I ran the recorded files through the simulations described above to produce samples of the bit error patterns and decoded voice over the three modes. Here are four examples of the 2000 bit/s mode, taken a few minutes apart, followed by the usual plots for one of them (serial 003). These are longer samples, about 30 seconds, to make sure we experience a few fades.

Sample Average BER
2000 bit/s 001 0.010
2000 bit/s 002 0.013
2000 bit/s 003 0.020
2000 bit/s 004 11W 0.062

If you listen to sample 3, you can hear errors creep in at times corresponding to the low SNR (high BER) points in the plots above.

Sample 004 is an example of too much Tx drive. To illustrate this, Mark intentionally cranked the average power up from 4W to 11W, on a radio that is rated 25W PEP for SSB. This distorts the modem waveform, the BER goes up, and it sounds bad.

Unlike the simulations, the waterfall for off air samples shows the effect of the SSB rx AGC bringing the noise level up during fades, as it seeks to maintain the same average output level. The noise at the start and end of the waterfall is the channel noise before and after the 30 second test wave file.

This was a noisy but intelligible SSB channel for me, and FreeDV sounds OK over it. That’s pretty much what I am aiming at – for FreeDV to work about as well as SSB over HF multipath channels.

Next Steps

So now it’s time to try some real QSOs with these new modes and see if we have made any real progress. There is danger in getting too far ahead of oneself with simulations. A common mistake is getting a good result, declaring “victory”, then moving on when some bug is actually means you are fooling yourself. To support the new modes in FreeDV means quite a bit of coding which will keep me busy for the next week or so!

Once I am ready for testing the first step will be to establish the flatness of the tx response using this swept sine wave. Then ensure SSB works OK over a given channel, and start trying the new modes, in particular looking for examples of fading that break the modes.

Further down the track it would be nice to try different FEC and modulation schemes, lower Codec bit rates with strong FEC, attenuating or masking the annoying R2D2 sounds when there are bit errors, and (Peak/Average Power Ratio) PAPR improvements to increase our average transmitted power.

How You Can Help

I am interested in finding some more examples of channels that break the new modes. If you and a friend are set up for digital modes, then you can play and record wave files over a HF channel. By making off-air recordings of my modem test files, you can help me gather examples of real world HF channels, and help improve FreeDV.

Here is how you can help:

  1. If you are using FreeDV and find it’s not decoding, switch to SSB and confirm you can still communicate OK.
  2. Download and play these files (mod_test_1400.wav, mod_test_1600.wav, mod_test_2000.wav) over the channel, while your partner records the received audio. Make a separate recording for each file. The files are 30 seconds long, so I set the record time for 40 seconds. The FreeDV Tools-Record From Radio feature is a convenient way to record. It doesn’t matter if there is some noise at either end of the recording.
  3. Name the recorded files mod_test_1400_yourcallsign.wav, mod_test_1600_yourcallsign.wav, mod_test_2000_yourcallsign.wav.
  4. Email them to me.

Thanks!

11 comments to FreeDV Robustness Part 2

  • The URL for the sweep is wrong.

  • Nice work. How have you interleaved the bits when using FEC?

    • david

      I haven’t introduced an interleaver yet, but the parity bits for the FEC code are at the “opposite end” of the frame to the data they protect. This means the parity bits map to the higher frequency carriers, the data they protect to the lower freq carriers, which will hopefully make the FEC effective with frequency selective fading.

  • For those of us not from Oz, the “Squid Pole” is a telescoping fiberglass pole. Its original purpose is fishing (except that fish are vertebrates, so maybe we should call it cephalopoding).

    • They also get called the “tele-pole” amongst other names.

      ( Perhaps the people in that area have waters that are heavily infested with television sets. :-) )

      SpiderBeam is one manufacturer that produce them for antenna-mast purpopses, some as long as 20m. A very handy temporary mast. I’ve used mine in the past for a portable VHF antenna mast on a few WICEN events.

      Those who are looking for something that’ll work in constricted space, these are well worth a look.

  • Nicely done on the robustness work. Interesting that the 2kbps mode outperforms the 1.6kbps mode in some circumstances with the FEC doing more harm than good.

    I’d be interesting to try cranking the bit rate up to 2400 and seeing how the regular Codec2 fares. Getting that extra 6dB is going to be a challenge though.

  • Matt Robert VK2LK

    Its pretty awesome to see the BER graphs correlate with the fading on the FFT waterfall.

  • Robert

    What exact method of FEC are you using? I suspect there is some optimal method for this application.

  • john

    Hi David and all,

    Most impressive :) :) .

    I wonder if a scheduled transmission like W1AW of freedv of various types from various stations would allow more of us to try to receive the same freedv transmission from different parts of the world ?

    Maybe scheduled in sequence on the same frequency so one long recording could be made? 14236? And maybe send the sample files to you?

    I bet at this point freedv is more useable than any other voice mode?

    John

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>