This modem was “lost” a few years ago as the initial implementation was closed source. So I felt compelled to put a fair bit of . . . → Read More: FDMDV Modem Page]]> I have added a FDMDV modem page to this web site, plus typed up a README_fdmdv.txt that explains all the files related to the modem and how to use them.

This modem was “lost” a few years ago as the initial implementation was closed source. So I felt compelled to put a fair bit of effort into documenting the open source implementation I have been working on. Building this modem was fun, just hard enough to be challenging but no real show stopper bugs and didn’t hurt my head like Codec 2 algorithm development.

Next step is to tweak Codec 2 to make it interface cleanly to the modem for some initial on-air tests.

Keynotes and Open Source DSP

The Bruce Perens keynote had many good points on . . . → Read More: linux.conf.au (LCA) 2012]]> Well, it’s that time of year again – my annual geek-week at linux.conf.au. Every day there were many interesting talks to attend and many I had to miss. I am currently catching the ones I missed by watching the LCA 2012 videos.

Keynotes and Open Source DSP

The Bruce Perens keynote had many good points on why open source is becoming essential to our security and well-being in the 21st century. These themes were expanded by the other keynote speakers. Bruce stated that “open software is the only credible producer of software”, we can choose to be “slaves to tools or the people who control the tools”. Watch the talk for more information on these memes.

I am interested in “the art” of presentation (as distinct from the content) so was also interested in Bruce’s presentation style from that perspective. He appeared on stage in a suit, which is uncommon in a geeky crowd. It raised a few eyebrows. However Bruce then explained that “a suit at LCA” was a theatrical device to underscore the point that we, as an open source community, should be facing outward. Open source communities are good at talking to people within our community, but our image outside of that community is poor and misunderstood (e.g. not many people understand their email is relayed via open source, or how it helps security problems and can help preserve democracy). Our external image must be improved.

Another great point by Bruce was how Open Source is now solving tough, previously opaque problems that were traditionally considered too hard due to patents or specialised knowledge. All you need is one guy to really get into and understand the problem. Suddenly the voodoo evaporates. People then know it’s possible – a problem that our peers have solved always appears easier. This one guy publishes source and shares what he has learnt. Others start to hack the code. Bruce, much to my embarrassment (!), actually cited my DSP work as an example, for example Oslec for echo cancellation and projects like Codec 2. When Oslec started I had many people tell me “it can’t be done”, “you need a DSP chip to do it in hardware”, or “it’s all covered by patents”. The truth is that Open Source DSP algorithms are now out performing closed source competitors. For example the Opus guys have developed a world-beating open source audio and voice codec. More on that below.

Actually I really enjoyed all of the keynotes. Paul Fenwick spoke on how our mind works, including topics like “decoy choices” and the “planning fallacy”, and how playing Tetris can help with traumatic experiences. I also recommend you watch the keynotes by Jacob Applebaum and Karen Sandler.

It was great to see Jean-Marc Valin and Timothy B. Terriberry in person, presenting on the Opus Codec. I wonder if this will be the “last” audio codec. It’s open source, royalty and patent free, will be an IETF standard, codes speech and music signals from 6,000 bit/s up, and outperforms other codecs like MP3.

Codec 2 talk

As I mentioned above I am interested in how conference presentations work. Like a lot of my work, I am inclined to experiment. Try something different. Hack it. A presentation on Codec 2, like many presentations at LCA, has a strong technical component that is hard for the average LCA attendee to follow. My Codec 2 work is an extension of my PhD research in speech coding. It took me 3 years to get my head around speech coding for the PhD. So how do I communicate Codec 2 topics to a smart, but non-speech-coding aware audience?

One way to handle this is “tutorial” style. You spend about half the talk bringing people up to speed on your technical topic. Enough to explain in the second half how you applied Linux or open source to this field. This is a common approach at LCA. It can work well, but also means a lot to absorb for the audience. This can be a challenge after a day (let alone a week) of great ideas and intellectual stimulation at a conference like LCA.

Instead of the tutorial approach I hit on a different idea. Rather than confine my talk to Codec 2 and DSP theory, I tried approaching Speech Coding from a variety of tangential topics that matter to LCA attendees. I talked about codec patents, how Ham radio relates to Open Source, and finally a really easy to grasp graphical explanation of how the sinusoidal model used in Codec 2 works. I left out a bunch of DSP topics, and didn’t even put up a block diagram of the codec. I wanted the audience to walk away knowing 3 or 4 things about speech coding really well, rather than try to cover the entire, technically deep, acronym rich subject at a shallow level.

This worked well. Really well in fact – my Codec 2 talk was voted the best of the conference and I was asked to repeat it later in the week. Wow! This was especially amazing for me as the voting is done by the attendees. A nice way to start 2012 for me, after working through some tough personal issues in 2011. Here are the slides for my Codec 2 LCA 2012 talk in Open Office format.

How Good Was Your Conference Talk?

It is important to me that my talks are well received. For me it’s part of my job and I take it seriously. Here is what I look for. This applies mainly to conferences with multiple parallel threads where people have a choice in what they attend:

• Did I fill the room (or nearly so)?
• Were people still asking questions at the end of your allocated time? Extra points if people come up to you afterwards and ask more questions. Even better if you get hustled off by the conference organisers because the next speaker is overdue to start.
• Did some of the more popular speakers/major contributors to the conference attend the talk?
• Was the applause loud and enthusiastic?

Oh, and I also like to make my talks short and leave more time than usual for questions. For example in a 50 minute slot I will time my talk to be 30 minutes rather than the nominal 40, allowing 20 minutes for questions. I feel strongly that the audience should drive a good chunk of the talk through their questions. This feels much better to me than running over time and not allowing enough time for questions.

21 Second Lightning Talk

Lightning talks are a fun part of LCA. These usually last 5 minutes. I had an idea for a lightning talk on my Electric Car that I wanted to try. I figured I could get my talk done in 10-15 seconds. Yes, I was experimenting with presentation styles again. I wanted to use lots of slides connected with just a few words (normally we are encouraged to do it the other way around). This year I managed to get a lightning talk slot and presented the talk. You can see it on the lightning talk video starting at 50:20. From when I start to when I stop talking is 21 seconds, not quite the sub-15 seconds I was aiming at. I always was a bit talkative. However the applause was pretty loud so I think the idea worked!

Codec 2 Hacking

While at LCA Jean-Marc did some great LSP vector quantiser work for Codec 2 and explained some of the techniques involved. This was very useful, and will be part of Codec 2 soon. Thanks Jean-Marc!

At the end of the conference Bruce Perens (left), Timothy and Jean-Marc (right), came to stay for a few days at Nuria’s (centre) house. It was really nice to have them all, we did some good work on Codec 2, and the dinner-time conversation (fuelled by Nuria’s fine lasagne and BBQ) was fascinating. As Bruce pointed out, there is great value in the small number of open source speech coding guys meeting face to face.

I was a bit nervous travelling in the same car with Jean-Marc and Timothy. People working on open source voice codecs are rare – so we figured we had 60% of the world’s open source codec guys in one car!

It’s been two years since I installed Lithium batteries and . . . → Read More: 28,000 Electric km]]> I haven’t written much about my EV for a while as nothing much has happened. It just goes and goes, and has just clocked over 28,000 electric km since it was converted. This post is a collection of notes from my EV driving in 2011.

It’s been two years since I installed Lithium batteries and they have operated faultlessly. I haven’t touched them. It’s a bit deceiving actually, guess I should check the terminals or something. But when it just goes and goes you get a bit complacent. I don’t even have an ammeter or voltmeter at the moment.

This picture says it all – spider webs where the petrol used to go!

However I realised I hadn’t taken the car in for a “service” since I finished the conversion to electric drive 3 years ago. This is because there isn’t much to service in an EV, no oil, water, spark plugs, timing belts, heads to crack, water pumps, hoses, fuel pumps, or exhaust systems to corrode. The only wearing parts are brakes and tyres. So I took it down to my friends at Woodville Park Autos and they rotated the tyres, checked the brakes and replaced the wiper blades. That’s the maintenance for 3 years and 28,000 km on an EV!

Earlier this year I did 108km of city driving on one charge, and the car still felt just fine. So I am not sure what the range actually is. One of those things I don’t really want to find out!

The 6.7 inch Advanced DC motor is adequate for a 60 km/hr commuter vehicle on the Adelaide plains. It’s quick off the line on the flat but struggles up hills at say 80 km/hr. OK for our terrain but I think I would put a 8 inch motor in next time. However this would require bigger Lithium batteries to provide enough current.

The feeling of “electric cruising” on a warm summer night is quite magical. Windows down, a warm breeze in your face and no motor noise.

My daughter turned 16 recently and had her very first driving lesson in the EV:

It’s much like an automatic to drive so a nice easy way to start. I am sure EVs will play a much bigger part in her life time than ICE vehicles.

Here are . . . → Read More: Codec 2 at 1400 bits/s]]> I’m in the process of releasing a 1400 bit/s version of Codec 2. Through efficient quantisation of the LSPs I have reduced the bit rate from 2500 to 1400 bit/s with only a small change in quality. This bit rate makes the codec very useful for digital voice over HF radio channels.

Here are some samples:

GSM full rate is what you might have been using on your mobile phone a few years ago. It’s a good example of “communications quality” speech. Compared to GSM, Codec 2 does a reasonable job at just 10% of the bit rate. There are some more samples on the Codec 2 page.

I think it’s possible to eventually push Codec 2 beneath 1000 bit/s with the same quality level. Improvements in the speech quality of Codec 2 at 1400 and 2500 bit/s are also possible with further algorithm development.

Some factoids:

• At 1400 bit/s you can send 45 phone calls in the same bandwidth required for a standard 64 kbit/s phone channel.
• 1400 bit/s is 175 bytes/second.
• A 30 second voice mail can be stored in 5250 bytes.
• A 30 minute pod cast can be stored in 308 kbytes.
• At 1400 bit/s Codec 2 uses 56 bit (7 byte) packets, sent every 40ms. If used for VOIP the RTP+UDP+IP overhead is 40 bytes/packet. So the payload is just 15 % of the total VOIP packet.

History

Building a 1400 bit/s communications quality speech codec is a highlight of my career.

My interest in speech coding started in 1989 just after I graduated from engineering. I was working with a team or researchers on Mobilesat – one of the first satellite-based mobile phone services. We were working mainly on the modems for the Mobilesat system. Just at that point in history, it became possible to use digital speech rather than analog systems such as FM or SSB. During the 1980′s breakthrough speech coding algorithms were developed that could deliver communications quality speech at less than 10 kbit/s. At the same time, the invention of DSP chips meant we could (just) run these complex algorithms in real time. Prior to that it took 30 minutes to process 3 seconds of speech on the PCs or workstations of the day.

Although I was meant to be working on sat-com modems, I was fascinated by speech coding; first the DSP hardware, then the challenges of real time implementation, then the speech coding algorithms themselves.

The CELP based speech codecs I built in the early 90′s could deliver communications quality speech at 9600 bit/s, or (with a significant drop in quality) run at 4800 bit/s.

So managing to get about the same quality at 1400 bit/s is a nice personal achievement for me. Giving it away to the world is even cooler.

2500 to 1400 bits/s

Here is the bit allocation of Codec 2 running at 2500 bit/s:

The Line Spectrum Pairs (LSPs) dominate the bit rate so I focused my attentions there for a couple of weeks. Here is the bit allocation of Codec 2 running at 1400 bit/s:

A Graphical Explanation of LSPs

Rather than going into the math of LSPs let me explain graphically. Here is a plot of 10 LSPs over 400ms of male speech:

The LSPs were extracted from this section of male speech:

The segment is the word “force” from the male sample at the top of this post.

In our case there are 10 LSPs. They are spread over 0 to 3500Hz. Together they represent the spectrum of the speech signal at any given point in time. They evolve over time as the speech signal changes, so we have to keep sending updated versions every 20ms or so.

Speech coding is the art of “what can we throw away”. So the idea is to send each LSP frequency across the channel to the decoder with the minimum number of bits, but still maintain good speech quality.

Closely spaced LSPs represent peaks in the speech spectrum. Our ear is very sensitive to these peaks so we must take special care with closely spaced LSPs. In the example above, you can see LSPs 1 and 2 close together between frames 7 and 25, at the same time we have a high energy vowel. You can also see LSPs 8&9, and 9&10 coming together during a consonant between frame 25 and 30. This indicates two peaks in the spectrum at high audio frequencies.

The ear is more sensitive to low frequencies, so it turns out we can use a coarser representation (less bits) for higher frequency LSPs.

There is another property that helps us. Perceptually important frames of voiced speech like vowels tend to change slowly. This suggests that coding the difference between frames will lead to coding efficiencies, as the frame-frame differences are very small.

Scalar and Vector Quantisation

The 2500 bit/s version of Codec 2 uses scalar quantisers. For example LSP 8 is “quantised” to one of 8 values represented by a table or array:

To quantise LSP 8, we find the closest value in the table, then send the index of that value. For LSP 8 this requires 3 bits/frame for the 8 possible values. The 2500 bit/s version of Codec 2 uses 36 bits total, with one table for each LSP. Because each LSP has it’s own quantiser table, it is known as scalar quantisation.

A Vector Quantiser (VQ) can be more efficient as it quantises several values at once, which can be referred to with just one index:

This example vector quantises LSPs 1 to 4 using a 12 bit (4096 entry) table. VQs can be very efficient, as they quantise several values at once and can take into account correlations in the input data. The trade off is that VQs tend to be noisy, as the table entries may not quite match all of the values in the input vector. Also I have found designing a VQ that works well to be quite a challenge.

From 2500 to 1400 bit/s

Here are some highlights of a couple of weeks of trial and error. I am not claiming any of this is particularly original, just new or important to me so worth logging here:

I developed a 25 bit/frame quantiser using scalar quantisers for LSPs 1-4, then a 12 bit (4096 entry) Vector Quantiser (VQ) for LSPs 5-10. I used VQ for the higher LSPs, as they are less sensitive to quantisation noise. I freely admit I don’t completely understand VQ, so there may be room for improvement here.

I found that I could “pre-quantise” or bandwidth expand the LSPs without any drop in quality. For example if LSPs 5-10 are quantised to 100Hz steps there is no perceptual difference in the decoded speech. This suggests that quantising the LSPs to any finer resolution is a waste of bits – we can’t hear the difference. I used this effect to design the 12 bit Vector Quantiser (VQ) for LSPs 5-10, that (for me at least) worked better than the same size VQ I designed with traditional minimum mean square error (MSE) training methods.

Then I started playing with delta-time quantisation of LSPs. During high energy, strongly voiced speech, the LSPs change slowly from frame to frame. So I experimented with just transmitting this small change.

Perceptually important, closely spaced LSPs are really sensitive to quantisation noise. Small changes in closely spaced LSPs can have a big effect on the decoded speech quality. Fortunately, during this sort of speech the frame to frame changes are very small. So for coding delta changes in LSPs I designed a VQ codebook by hand with the properties I wanted. I took the approach of constraining the VQ codebook to very small changes.

Here are some of the delta-time codebook entries:

This full codebook is here.

It’s a bit like counting in binary, except the base changes for each element of the vector. It’s probably not optimal, but it works. As there are 81 values in the codebook it can be transmitted with 7 bits (entries 82 to 127 are not used).

Another twist – I discovered was that I could get away with just updating the lowest LSPs 1-4 on the delta frames. On odd delta frames I just copy the previous values for the LSPs 5-10. This means just 7 bits/frame are required on the odd frames for LSPs.

A further reduction came from delta-coding the pitch (fundamental frequency), as it also changes slowly during perceptually important voiced speech frames. Just 3 bits on odd frames resulted in no loss of quality.

Next Steps

Over the next few weeks I will release a separate encoder/decoder version of 1400 bit/s Codec 2. At the moment you can run the same algorithm using the “c2sim” simulation program:

$ svn -r 306 co https://freetel.svn.sourceforge.net/svnroot/freetel/codec2-dev
$ cd codec2-dev && ./configure && make && cd src
$ ./c2sim ../raw/hts1a.raw --1400 -o hts1a_1400.raw
$ ../script/playraw.sh hts1a_1400.raw


I’d really like to see Codec 2 combined with a modem and running over the HF bands. Some early experimentation to get real world user feedback and rapid development of an “open” digital voice mode would be great. This mode could be implemented as a Linux or Windows PC application that uses two sound cards to connect to a SSB radio and head set.

A key issue to explore is robustness to bit errors. I favour unequal error protection modes, for example just a small amount of FEC that protects just a few key bits.

There are many areas where Codec 2 could be improved. The LSP quantisation could be developed further to improve the quality or lower the bit rate. I’d also like to work on the model used to synthesis phases at the decoder, and track down some issues with different speakers.

Links

Codec 2 Project Page