Solar Boat 2

I’ve just spent 6 days on the Murray River, cruising about 200km on solar power alone, sleeping and cooking on my solar powered Hartley TS16 sailboat. I’ve always wanted to do a longer trip based on the Solar Boat design from 5 years ago (thanks again Gary for your help with mounting the panels). I’ve made a few minor improvements – a better (75AH) battery pack, automatic charge cutout, and cleaned up the wiring (thanks Jacqui for your fine soldering). That was fun too, sometimes the project work to prepare is as satisfying as the trip.

This photo shows the “Solar Hartley”, the Honda 5HP outboard (which didn’t get started on this trip) is obscuring the Torqeedo 2kW unit. The electric motor is inside the black oval visible under the rear of the boat, with the orange propeller.

It really is a nice way to see the river, I travel at “houseboat speeds” – about 5-7 km/hr depending on wind and currents, at a motor power of 300-500W. It’s very relaxing compared to the sailing I normally do. On the big wide river I just nudge the tiller every minute or so. With sailing I’m forever tweaking ropes, making sure I don’t run out of wind, worrying about tipping, and trying to avoid hitting Australia.

The solar power peaks at about 380W (14A at 26-27V) and maintains that for 5 hours, so most of the cruising is “energy neutral”. I was actually surprised by how much power I was generating, I guess because it’s Summer here (the previous trip was in Autumn). I’m generating greater than 100W solar from 0800 to 1800. After 0900 the solar power means the battery draw is so low you can effectively drive all day.

Some of this solar boating takes a bit of getting used to – for example when I stop for lunch the boat is “refueling”. Vehicles don’t usually refuel themselves when they sit around parked. This has some practical benefits. I’ve read about other Murray travelers in small boats who walk several km from the river to find fuel for their thirsty outboards. That fuel might weigh 10-20kg, hard to carry by hand. That’s if you can get fuel – some of the places I went to were remote, with no fuel available for several hour travel in either direction.

It’s very quiet. Most of the noise is the trickle of water, just a slight whine from the motor. If I stand up it’s just water noises. People on the shore don’t look up when I pass as they can’t hear me. I can sneak up on kayaks. When other boats go past their motor noise is louder than mine – until they go around the next bend 1km away.

I think this would scale to a larger boat, e.g. a houseboat with more panels but the same speed. Not sure if solar boating would work at faster speeds, I might try my motor with a new prop on a smaller boat and see if I can get it up on the plane. Opening the throttle I got up to 10 km/hr at 1200W, but why bother? Three times the power for 25% more speed. My sailboat is a displacement hull (4.8m waterline) and travels at about the same speed with the 5 HP Honda outboard, which consumes about 1l/hr.

The trip was very pleasant, I had the river to myself most of the time. Cruising past lovely tall cliffs and quiet holiday home settlements. You can camp almost anywhere you can find a reasonable spot to park the nose of the boat. I just had to be careful I could get off without sinking knee deep into mud, and not too flat – you want the back of the boat in “prop deep” water. My boat is light enough that I can just push or paddle off with an oar if I have to so. The boat has enough room for me to sleep in and for all my stuff. I also have a 12V battery to charge my toys, run the echo sounder and provide lighting.

The mornings were the nicest, I tended to start at dawn, running on battery alone on for a few hours, mist on the river.

By mid afternoon I would find somewhere to moor for the night, nose into the bank, a couple of ropes from the stern to keep the back in “prop deep” water.

Occasionally I’d find a pontoon to tie up to, and even more occasionally be able to park the boat properly.

I do need a better seating arrangement, the deck chair is comfortable but takes up too much room and I can only just reach the tiller. It is nice to be high up though, so see over the panels. I used the wonderful Murray River Pilot book as a reference, and the Wikicamps app to help me find camping spots and toilets.

The Torqeedo Cruise 2.0 outboard motor I purchased from Eco-Boats (6 years ago) performed flawlessly, running up to 10 hours/day. No fuss or mess refueling it either, and no hearing loss – imagine sitting next to an internal combustion outboard 10 hours a day!

The Doc Wattson meter was pretty handy, I could watch the solar power climb as the angle of the sun rose, and monitor the battery voltage. It tells me I generated 500Ah and 13.5kWhr over the trip. That’s about 6 full charges of the 75AH battery pack.

It’s a pretty cheap way to holiday. Some great scenery, on the water, and all I really spent was $50 for fuel to tow the boat to the river (two hour drive each way from my home). I made my own meals on board on a little stove. All the camping is free.

Next goal to travel the entire Murray from the South Australian border to the sea, about 800km. I imagine that will be a few weeks in total, which I can do in chunks, a few days at a time.

FreeDV Update – Sep 2023

This post is summary of the work I have performed in September for our Enhancing HF Digital Voice With FreeDV ARDC grant.

OFDM Modem Deep Dive

I’ve been continuing my “deep dive” into FreeDV modem algorithms, working on a new low SNR FreeDV waveform that will hopefully replace FreeDV 1600, 700C, D & E, and give us improved performance on multipath channels. It’s a work in progress which I’m documenting in this report.

I’m taking a fresh look at modem algorithms that were developed several years ago, putting a little maths around the code, and writing up the results. I’m also hoping to get the modem reviewed – please contact us if you have HF OFDM modem experience and would like to help out for a few weeks on this task (either as a volunteer or paid consultant).

In September I finished up the equalisation work with pleasing results – a single algorithm that can track phase offsets on a variety of channels from benign AWGN to fast fading multipath with performance roughly the same at FreeDV 700D. This is a nice step towards our “single mode” goal. This work is documented in Section 3.1 of the report.

I then moved on to strategies for improving performance on multipath channels (Section 3.2), exploring diversity in frequency and time. By sending two copies of a FreeDV 700D like waveform separated in frequency, the simulations indicate a useful 3dB improvement, with up to 5dB if we also add interleaving to spread the signal in time. However time domain diversity also adds algorithmic delay, which is problematic for PTT speech.

Towards the end of the month I started looking into acquisition – the algorithms used by a FreeDV receiver to lock onto off air signals. I’m working on coarse timing estimation first, revisiting the experimentally derived algorithms I built up a few years ago. We’d like to improve acquisition so we can automatically sync up to signals with +/- 200 Hz frequency offset to make tuning FreeDV signals easier.

After I’ve prototyped the most important new modem algorithms, I’d like to set up some automated tests to send test signals over the air to check the simulation results match reality. It’s quite easy to “get ahead of yourself” with the math and simulations and miss something important in the real world.

Administration and Documentation

But it’s not all R&D. On the administration side I’ve progressed our DSP Engineer job descriptions, including the comments from several reviewers. However I’ve hit a wall here, as I need some help with employment of DSP engineers and project management of the DSP work. I’m at the limits of my available time and brainpower with the DSP R&D. Our Project Leadership Team (PLT) is working on a way forward with this little hurdle.

I was also interviewed by Stefan DH5FFL from DARC, the interview will be appearing in an upcoming issue of CQ DL. This is part of our work in documenting Codec 2. I’ll be following that up with a more technical article, and a mathematical algorithm description of Codec 2.

We’re setting up policies and process for paid work on the FreeDV project- part of the transition from a hobby to paid engineering work on the project. We have been discussing what maintenance work we will commit to and what external contributions we will accept into the codec2 repo. For example with new modems being developed, there is not much point spending ARDC funds on maintaining older FreeDV modes that may soon be deleted.

We are also standardizing on gcc/C99 and encouraging people who want to use non-standard compilers like MSVC and certain embedded compilers to maintain their own Codec 2 forks. Our focus needs to be on what’s unique about our project – the speech codec and modem waveforms – rather than consuming time and resources on non-core activities that others can do equally well.

There are so many rabbit holes we could go down and potential distractions – but we need to stay focused in order to deliver value and meet our stated goals with grant funds that we are lucky enough to have been awarded.

SM1000s are back

Lastly, thanks Edwin and Jerry from Dragino for making a new batch of SM1000s available, after a long break due to the stm32 chip shortage over the last few years.

FreeDV Update March – August 2023

In March of this year (2023) the FreeDV project (in partnership with the Software Freedom Conservancy (SFC)) was lucky enough to be awarded a grant based on our project proposal Enhancing HF Digital Voice With FreeDV by the kind people at the ARDC.

This post is summary of the work I have performed under this grant between March and August. There has also been lots of other fine work going on, in particular by Mooneer Salem K6AQ and other members of the FreeDV community.

First of all – as the FreeDV organisation has grown, there is quite a bit more overhead involved. We have monthly meetings of a project leadership team, which spins off various actions. There is also project planning, management, budgeting, contracts to review and sign, working with with our financial sponsor (the SFC), and job descriptions to write. So it’s not all coding and DSP, which is where I like to focus!

Just as the grant kicked off, I was finishing up a study I had been working on for about 8 months which involved a deep dive into several of the core algorithms used in Codec 2. The output was some new algorithms, a journal/report documenting the algorithms, and candidates for new 700 and 1200 bit/s Codec 2 modes. I’ve posted some speech samples in this repo.

The document was a journal that I brain dumped my work into, as I had no one else to discuss it with. Using the grant funds we’re hoping to employ one or more DSP engineers experienced in speech coding to help out with this work – the first step will be to review the new algorithms (and Codec 2 in general).

I should point out the candidate modes are not ready for real world use, they require review and refinement before progressing to release. Unfortunately there is only so much I can do by myself, so the codec work is blocked until we get some extra DSP brain power (hence the need for the grant).

In April and May I developed the datac4 and datac13 raw data modes to support FreeDATA, that Simon immediately integrated and is now in daily use. The mode is capable of sending data down to around -8dB on poor quality HF channels, which has resulted in messages being sent all over the world using HF Radio and no other infrastructure. Here is a map of FreeDATA users. Here is some more technical information on the FreeDV raw data modes that have been developed.

In May I went back to basics with some maths for efficient filters for FreeDV, for example ways we can generate filter coefficients on the fly and the most efficient way to implement the filters in real time.

In July I performed a clean up of the codec2 repo. Over the last decade it had accumulated quite a bit of cruft, personal projects, and code to support various blog posts. The FreeDV Project Leadership Team (PLT) have developed policies for what is allowed in the codec2 repo from now on:

  1. Only code that is required to build libcodec2, or to test libcodec2 goes in codec2.
  2. Experimental stuff, WIP R&D reports, David’s random code from blog posts, code used for development, goes in some other repo.
  3. Only widely used “production” code goes in codec2. If it has an user base of < 2 (e.g. for personal projects, early R&D) - it should probably be application code or a fork.

The old repo (with all the ancient cruft and mad scientist experimental code) has been renamed codec2-dev. Still a bit of clean up work to go and (with the support of the ARDC grant) I’d like to do some more documentation of the algorithms, in particular for the voice codec. However it’s a fine starting point for new development.

Since August I’ve been working on a new low SNR FreeDV waveform. The goal is a single mode that can work on a variety of channels, from benign AWGN channels to fast fading, and replace FreeDV 1600, 700C, D & E. We would also like it to outperform SSB at 0dB SNR. These are tough goals. To start with I’ve deep dived into what physics says is possible, and where our current “losses” are – for example overheads due to synchronisation. It’s all work in progress which I’m documenting work in this report.

At the time of writing the report and simulation is work in progress. The current focus is equalisation of the HF channel – trying various algorithms using pilot symbols to correct the symbol phase. About five other algorithms to dive into after that.

Improving FreeDV 2020

It’s been a few years since FreeDV 2020 was released. On-air experience suggests FreeDV 2020 is less robust to low SNR and fading than the 700x modes, which means it can only be used on benign HF channels, with slow fading and a relatively high SNR. Factors include the high bit rate required for the codec [3], and the modem waveform design.

The goal of this project is to improve the robustness of 2020, and provide intelligible speech at the same operating point SNR as FreeDV 700E. In numerical terms, this can be expressed as the SNR where 700E achieves a coded bit error rate of 0.01, and a packet error rate (PER) of 0.1.

To explore these goals I ended up developing two new modes 2020A, and 2020B. Here are the key innovations, and the theoretical improvements:

  1. Compression (clipping) of the 2020x modem waveforms has been added, which is worth about 4dB. Should improve vanilla 2020 straight away.
  2. 2020A uses the same waveform as 2020, but an unequal error protection scheme. The most important 11 bits of the 52 bit LPCNet codec payload are heavily protected, the other bits not protected at all. This changes the “slope” of the speech quality against SNR curve. Compared to 2020, it may work (with poor speech quality) at lower SNRs, but still have a few audible errors even at higher SNRs.
  3. 2020B is like 700E to 700D – it works with fast fading but requires a few more dB. This will make it usable in European Winter (or over the South Pole Argentina to Australia) type channels – if you have enough SNR. The big challenge here was squeezing all the info we need (enough pilots symbols for fast fading, LPCNet, FEC bits) into a 2100 Hz channel – we are pushing up again the edges of many SSB filters. It also uses unequal FEC, just the most important 11 bits are protected, but not as well as 2020A.
  4. Index optimisation of the LPCNet Vector Quantiser [7].

Simulations

To explore the experimental ideas above I developed a script to test the new modes on simulated channels. I used FreeDV 700E and compressed analog SSB as controls. Tests were conducted at the same peak power/noise level. The multipath channel (multipath poor MPP and multipath disturbed MPD) peak power was set 5dB higher than the AWGN.

Here are the results.

The reported SNR (right column of table) varies because:

  1. With clipping the average power S increases.
  2. Different waveforms have different Peak to Average Power Ratios (PAPR), so S will vary slightly across waveforms. SSB SNR is dependant on the compression and source material.

Comparing 5 & 8 (AWGN) and 6 & 9 (MPP), it appears that compression helps 2020. We get about 4dB SNR increase and improved speech quality with the same peak power.

Comparing 10 and 19, we can see that 2020B does indeed allow us to operate on fast fading channels. However it’s not error free – due to lack of FEC.

Comparing 12 & 13, index optimisation [7] helps a bit, there are less large amplitude pops and clicks. This is not as obvious on 19 & 20, so the improvement is not large.

Comparing 5, 11, and 16, that lack of FEC on 2020A and B can heard, 2020 is nicer on the AWGN channel as FEC mops up the errors.

In 24-26 the noise level is increased (SNR decreased) by 3dB, the 700E PER is 3%. In SNR terms this is just a fraction of a dB away from the 10% PER metric mentioned in the introduction. One of the goals of this project was to make a 2020 variant usable at the same low SNR as 700E. Both 2020 and 2020A get there, but they do take “one sentence” to sync up. I don’t think 2020A (with it’s partial but strong error protection) is any better than 2020 (which now has compression/clipping).

My take aways from these samples:

  • Waveform compression (clipping) helps on all 2020x waveforms, and gets us intelligible speech at SNRs similar to 700E, albeit sync is a bit slow.
  • The effect of index optimisation is marginal when applied to this codec.
  • 2020B works as advertised and can handle the faster fading MPD channel.
  • However the unequal error protection on 2020A and B means audible errors, even on relatively benign channels.
  • 2020A with it’s unequal error protection scheme isn’t any better than 2020 at low SNRs.

Controlled Over The Air (OTA) Tests

Next step was to test on real HF channels, using a similar approach to [6]. The samples we sent in groups of 4 (700E/2020/2020A/2020B) every 30 minutes over the course of one day.

Here are the results. Key:

  1. I used two KwiwSDRs, one less than 100km away (iron) to get a fast fading NVIS path, and one 800km away (am).
  2. Rx – off air sample. First my station ID, analog (SSB) test sample, digital 2020x test sample modem signal.
  3. AnDV – like Rx, but the 2020x modem signal as been decoded. Lets you compare SSB to DV.
  4. DV – just the decoded 2020x – useful for comparing one test to another.
  5. You can get bigger versions of the plots by opening them in another browser tab.

I note (i) The station ID SSB at the start is a bit louder than the SSB test samples, I need some more work on my SSB compressor tool (ii) 700E messes up the mans voice, that codec need some work. A project for later this year.

Sample 24 is a good example of the quality of 2020 compared to SSB.

Samples 59-62 are an example of fast fading and large delay spread, the 700E and 2020B modems obtain a better SNR (10dB compared to 5dB) as they are designed to handle this channel.

Samples 75-78 show 2020A/B doing a little better than 2020, but we still have some errors. Once again, the SNR of 2020B is quite a bit higher as it handles the NVIS channel better.

In samples 79-82, 2020 does better than 2020A/2020B. Once we reach a threshold SNR, the FEC kicks in and squashes any bit errors. It’s quite a bit better than SSB too.

Conclusions

With respect to the goals in the introduction:

  1. We have indeed improved the robustness of FreeDV 2020, waveform compression seems to be the most useful innovation. This can simply be applied to the existing 2020 mode with no compatibility issues. No new mode is required.
  2. We have shown 2020 providing intelligible speech at SNRs close to 700E, although in these tests sync was a bit slow.

Some of the OTA samples show speech quality that is competitive to SSB at similar SNRs.

2020B can handle fast fading channels, however I’m not convinced the partial protection techniques used in 2020A/2020B are worth it. The residual errors are annoying, even at high SNRs. Better to have FEC on everything, just like 700E and vanilla 2020.

Further work

Rather than original goal – would it be nice to set a goal of “noise free”, as that is a key potential benefit of DV over SSB. The residual errors on 2020A/B are audible and annoying. 2020B does work well on fast fading channels, but suffers from lack of FEC.

This testing was conducted with a very small set of samples. It would be useful to test wider, for example with different speakers and different channel conditions, and different radios. Can the average Ham radio station also achieve higher quality with FreeDV 2020 than SSB? If not, why not?

LPCNet has moved on a lot since I forked it for experimental FreeDV use. It would be useful to test the latest version. I am not sure if any of the recent changes are relevant to this work.

Some other ideas:

  1. In the automated OTA tests it would be useful to detect if anyone is using the frequency before starting to transmit. This is tricky as I can’t hear anyone on my local radio due to HF EMI, so it would need to be through the KiwiSDR
  2. Not sure if I have levels right – I need a better SSB compressor tool
  3. Sometimes we get a false sync during analog speech sections.
  4. Performance with lots of lightning crashes (impulse noise) is worth looking into. This really mess’s up SSB.
  5. It would be interesting to repeat the OTA tests using regular SSB radios, to see if the results are repeatable.
  6. Faster sync for 2020 would be nice.
  7. A way to squeeze FEC coverage of all bits into the 2020B signal. Unfortunately we use up a lot of bandwidth to deal with the fast fading (more pilots, higher symbol rate, larger cyclic prefix), so we don’t have “room” for a strong FEC code. We might be able to cover all bits with a high rate code, which would at least mop up any remaining errors on high SNR channels.
  8. We might be able to improve muting when there are channel errors, for example make it FEC based rather than a SNR based squelch.
  9. Further work on compression to reduce PAPR. For example we could design an interleaver to minimise the probability of high PAPR frames, using similar techniques to index optimisation [7].

Links

[1] FreeDV 2020 improvement brainstorms
[2] Experimental 2020A Mode
[3] Experimental version of LPCNet, forked for use with FreeDV (now quite out of date).
[4] FreeDV mode overview
[5] FreeDV 700E
[6] Controlled FreeDV Testing
[7] VQ Index Optimisation

FreeDV Activity Day

On the weekend of 18-20 Feb I took part in the FreeDV Activity Day, kindly organised by Mooneer, K6AQ [1]. I was particularly interested in trying to receive FreeDV signals from other countries. To maximise my chances, I went camping at “Point Parnka” a quiet location about 200km SE of Adelaide. My station was a simple end fed dipole and IC7200 running 30W.

It was indeed radio-quiet there – my noise floor on 20m was S0, which on my IC7200 means less than -135dBm/Hz [2]. The 40M band was a bit noisier, about S1, and I could hear some 100 Hz buzzing on top of SSB signals. I looked around – in adjacent camp sites were caravans which I guessed had 12V to 240V inverters, and solar chargers (100Hz is the 2nd harmonic of 50Hz AC). Sure enough, the next day the caravans left, and took their EMI with them – 40M was now down to S0 and the 100 Hz buzzing gone.

Having such a low noise floor is interesting – I could hear many stations, and SSB was quite pleasant to listen to without the usual urban EMI. However not everyone could hear me. I had several one way contacts – they sounded great to me but I was lost in their local noise floor.

When I pressed the “Start” button on FreeDV, I could hear the USB link to my IC7200. I guess the USB bits are being modulated at HF frequencies. I messed around with some ferrites on the USB cable which improved, but didn’t entirely remove this EMI. Next time I will position my antenna further away, and try a “pure analog” connection to my laptop.

The highlight of the weekend was decoding Mel, K0PFX, long path from the US using FreeDV 700D on 20M. That’s about half way around the world. Until now, I wasn’t sure if there was a modem issue with FreeDV that made international contact difficult. However I think it’s just down to SNR – with enough power, the right antennas and band conditions, our FreeDV waveforms can make the trip just fine. We have evidence from simulation and automated Over the Air (OTA) tests [3] that suggests FreeDV is competitive with SSB in terms of low SNR performance. Nice to support these results with a real QSO.

I had a few FreeDV (and SSB) contact with Hams over the weekend, including some 700D and 2020 across 1000-2000 km paths in Australia. However I am a rare user of FreeDV. The problem is every time I use it I get frustrated about some aspect and end up kicking off another project to fix it! Oh well, the experimentation is fun.

Quite a bit of activity in other parts of the world, too:

A few observations:

  1. To date I’ve spent a lot of time working on the low SNR performance of FreeDV (where communications is marginal). Now I would like to see if I can improve speech quality on high SNR, “armchair copy” channels.
  2. I also discovered a problem with wind noise upsetting the codec. Making the codec more robust to acoustic background noise is an interesting project.
  3. High SNR SSB sounds great, but the quality drops off when the channel has impulse noise (e.g. due to lightning crashes). In theory FreeDV can correct these, but I haven’t carefully tested the modems to see how they perform. We only test against simulations of AWGN noise and multipath. The effect of radio AGC should also be studied – it make fast jumps when there are static crashes.
  4. It’s hard to beat the low latency of PTT SSB for quick back and forth overs, especially compared to using a laptop to run FreeDV. However the inherent latency of the FreeDV modes is pretty good (e.g. 80ms for 700E). With a real time implementation on a microcontroller (or integrated into a HF Radio) the PTT experience could be pretty close.
  5. I was pleased with the performance of the FreeDV modems. I monitored one 40M QSO 700E where I could see “notches” about 300 Hz apart, indicating a delay spread of 3ms, and the modem was hanging on fine. This is very pleasing, and I am happy we are making progress in this area – multipath channels are tough on modems

I’m looking forward to the next FreeDV activity day in May.

Links

[1] FreeDV Activity Day
[2] Measuring Urban HF Noise with a Loop
[3] Controlled FreeDV Testing

Measuring HF Noise Around a City

I recently took my loop antenna [1] to a national park on the edge of Adelaide, and tried to measure the noise power at 7.1MHz. I was expecting a low level. Curiously, the noise seemed directional, there was a 10dB null off the loop in one direction. This made me wonder if I was “DFing Adelaide”, and seeing the ground wave signal [5] from the sum of all noise sources in the city.

So I hatched a plan to take my loop antenna to a bunch of national parks around the perimeter of the city, and see if I could get sensible bearings back towards the city. This time I used a spectrum analyser so I didn’t have to mess around with s-meters. I spent a pleasant day circumnavigating the city, and hacked up a Python script [2][3] to plot the results on a map. The base of each line is the measurement location, and the height of the line the noise relative to the noise floor of the system.

Here’s a plot of noise power against the distance from the center of the city.

These results suggest noise levels drop as we get further away from the city, and is some evidence for ground wave propagation of noise from the city. I had wondered if being sited in a valley just outside the city would be enough to get a low noise floor, but perhaps distance is a factor as well.

I’m surprised a straight line fit of log(power) against linear(distance) works so well, as it should be a curve due to the inverse square law. Oh well, I guess it’s just a small amount of data.

The outlier at 30km was a site between two large sheds that were making machinery noises. I could see harmonics on the spec-an so suspect switch mode power supplies or similar nasties. Country sites can have local noise too.

The loop antenna/pre-amp/spec-an combination has a noise floor of -146 dBm/Hz. Some of my measurements were close to that level (e.g. -143). The measured noise power is actually the sum of the noise power collected by the antenna and the noise of the receiver. So if the received noise is the same level as the receiver (say both -146), the total noise power we measure is twice that (3dB higher) or -143. So I added some code to correct for the receiver noise power, this makes a few dB difference at low noise levels.

Reference [6] Figure 4.9 has some curves of expected noise powers, calculated from an ITU-T document on the subject. Here is a comparison with my measurements at 7 MHz, using my loop antenna and pre-amp system [1]:

Condition Expected Noise Factor (dB) [6] Measured Noise (dBm/Hz) Measured Noise Factor (dB)
Residential 50 -120 54
Quiet Country 30 -146 28

For the residential case I measured between -115dB/Hz and -127dBm/Hz around my suburb so used -120 as the average. Much to my surprise my results are a reasonable match with [6]! Of course rather than being fixed values, the absolute noise levels vary quite a lot over different sites, time of day, and atmospheric conditions, which is noted in [6] and the ITU-T document.

There is a 20dB difference in noise power from the best case in my suburb (-127 dBm/Hz) to the quietest location I visited (-146 dBm/Hz). This is consistent with [6]. That’s a factor of 100 if the noise power in expressed in Watts. Someone could use 1W to talk to me in the country, but require 100 watts to achieve the same SNR if I was listening in my suburb. It’s not possible for the average 100W Ham station to increase their power to 10kW which explains why HF reception from urban locations is getting so hard.

My experimental aim of “DF-ing” the noise was busted. I couldn’t find good nulls from most sites, especially the very low noise level sites.

Layers of Urban Noise

I have some vague hope of using DSP techniques and multiple antennas to cancel some of the noise and actually be able to receive 40M HF radio signals at home. These hopes are fading – it seems like there are several layers of noise:

  1. A city wide urban noise fog made up of the sum of millions of switch mode power supplies, VDSL [4], and high speed digital hardware. This is the best case S5-6 noise I see at certain times of day, and that I have established [1] is fairly uniform (+/- 5dB) around my suburb. An antenna with directivity could possibly help by rejecting noise from some directions, but I don’t really want to put up a 40M beam. I’m not sure if this noise can be reduced using two antenna phase cancellation techniques as the “urban fog” is probably the sum of energy collected from many directions, and phase cancellation effectively puts a null in the pattern of a two antenna array at one (or perhaps two) bearings. More thought required.
  2. High energy, very local stuff that sits on top that urban noise fog, like my neighbours lights or power line noise. This is often impulsive, and pushes the noise over S9. There may be several of these local sources coming from different directions, complicating the use of phase-difference subtraction techniques.
  3. Some times you get all of these noise sources at the same time, which explains why the analog phase-cancellation based boxes aren’t always effective. They can only cancel one noise source.

Further Work

Some ideas for further work:

  1. Try a wire antenna with directivity.
  2. Try 20m, that seems to have a lower quiescent noise floor (S0 at times), but still gets hammered by the S9+ local noise sources at various times of day. However that might reduce the problem to just the local noise sources, as the “urban noise fog” is at an acceptable level. I think. Time to retune the loop!
  3. Think about noise reduction techniques for the strong local impulsive noise.

Links

[1] Measuring Urban HF Noise with a Loop
[2] Easy Steps To Plot Geographic Data on a Map
[3] GitHub repo for this project
[4] VDSL versus HF Radio
[5] Radio Noise W8JI
[6] A High Performance Active Antenna for the High Frequency Band

Measuring Urban HF Noise with a Loop

For the last few years I’ve been interested in the problem of urban HF noise. As a first step I’d like to characterise the problem by taking a few measurements of noise at my home, around my suburb, and a few other locations. So I need a small, portable antenna. As I’m interested in high noise levels on HF, it doesn’t have to be very efficient.

On a “quiet” day – I see 7MHz noise from my dipole at S5 on my IC7200. I think the S meter is adding up all the noise in the IF bandwidth of the radio. Hooking up my spec-an and referring this to 1 Hz bandwidth, I measure the average noise at about -120 dBm/Hz which is 54dB above thermal (-174dBm/Hz). So even a low gain antenna will receive plenty of noise without hitting the noise floor of the receiver. Ideally I’d like to see -135 dBm/Hz, which is S0 on my IC7200.

Small Loop Antenna

I decided to build a loop, as they don’t need a ground plane, and are relatively insensitive to low permeability surrounding objects (like me holding it) [2].

Using the equations from [2] I decided to use 3 turns of RG58 (about 3m) with a 0.3m diameter. The braid is the conductor, I don’t use the inner at all.

My loop is fed using a toroidal transformer [5]. I like the idea of using a transformer to ensure good balance. All three turns of the loop coax pass through the toroid (the primary), and the secondary is a bunch of turns I wind myself. Using the equations on [2], I estimated the radiation resistance Rr at just 0.006 ohms. To match this to 50 ohms means a lot of turns on the matching transformer.

I tried 50 and 100 turns but for some reason the return loss was still really poor. Also it didn’t seem to be receiving anything, and touching antenna or capacitor terminals had no effect. On a whim I reduced the number of turns and the return loss started to improve, I could see a noticeable dip of a few dB as I moved the tuning capacitor. If I touched the capacitor or the loop it would de-tune, indicating it really was starting to become an antenna!

Turns out that for small loops, the resistive loss Rl is often greater than Rr. I confirmed this by connecting the loop in series with a fixed 100pF capacitor across the spec-an input. When driven by the tracking generator I can see a dip at resonance. By measuring the bandwidth I could estimate the Q and hence the loss resistance Rl of about 2.8 ohms, much greater than Rr of 0.006 ohms.

At about 15 turns I obtained a return loss of better than 10dB – good enough. However it’s really just a match to Rl, consequently the antenna is quite inefficient, I estimate a gain of 10*log10(D*Rr/Rl) = -25dB (D=1.5, the directivity of a small loop). But that still not bad for a 30cm loop on 40m, and good enough for my mission.

The ratio of 15 turns secondary, 3 turns primary (15/3)^2 would transform 2.8 ohm into around 70 ohms, and lead to an expected return loss of 15dB, about what I’m seeing.

EMI receiver

I decided to use my FT817 as a portable receiver to log noise values as I wander about with the loop. So I added a 26dB pre-amp [3] to compensate for the negative gain of the loop and to boost the Rx level high enough to move the S-meter. The pre-amp is increasing the signal and noise together, so unlike a regular pre-amp the receiver will not be more sensitive. Just louder.

I placed the loop on a 1m tripod at the front of my yard, away from the house as that seemed to reduce the noise a bit. Using distant SSB signals, I compared the dipole to the loop over a period of a few days. I used a RTL-SDR SDR running gqrx on the 40m band, with a 7MHz bandpass filter to prevent overload. At times the levels were quite close, at other times the loop down about 10dB. Sometimes the SNR was better on the loop, other times the dipole. It was hard to get exact measurements. I guess there are differences in pattern, polarisation, and the signals received at the slightly different sites, and the power of SSB bounces around.

However it’s fair to say the loop (with it’s pre-amp) is roughly as sensitive as the dipole. If the loop detects a signal (such as noise) the dipole will also detect the same signal at roughly the same level. So the loop is a useful antenna for portable EMI measurement.

By connecting my spec-an in parallel with the loop antenna, I created a calibration curve of noise density against the S meter reading:

My FT817 S-meter seems to have a very narrow range, and is wildly different from the IC7200 S meter. However if I can measure less than S1 on the FT817 (blue, attenuator off), it looks like I will hit my target of -135dBm/Hz.

Blowing up my FT817

During some portable measurements I blew up my FT817! The fix and fault (shorted filter EMI choke and blown internal battery fuse) were exactly as described in [4]. I managed to repair the Rx but the Tx power is still low at 200mW, it’s almost like it’s folding back due to high SWR (although the SWR into the dummy load is Ok). So I’ll need to think about that fault some more. I’ve added external fuses to both the positive and negative FT817 supply leads now, as well as the Lithium battery I use for external power.

Noise measurements

I hopped on my bike and logged the noise at various sites around my suburb. They all bent the FT817 needle with the attenuator off (blue), so I used the “attenuator on” curve (red). The actual measurements are the red circles in the figure above.

So it’s not just my house, this seems to be pervasive problem in my neighbourhood. The best case was walking out into the middle of a large sports ground. I guess that makes sense, as the EMI source must be around the perimeter, and we get some inverse square law fall off. Take that far enough and we have portable operation from a country site – putting some distance between us and the cloud of urban noise.

Discussion and Conclusions

I’ve been reading about antennas this year, and can recommend “Antenna Physics” by Robert Zavrel [1]. The technical level was just right for me – between typical Ham texts and deeply mathematical tomes. This project complemented my reading, and give me some practical experience in the concepts of loss resistance (Rl) and radiation resistance (Rr).

It was pretty cool to see the return loss improve as I adjusted the matching network, and understand “why”. I really enjoy the learning experiences with Ham Radio.

I now have a portable, sorta-calibrated EMI receiver, that I can use to measure the noise levels at various sites, and compare it to my target. As a bonus the loop is directional, so I can DF noise. The range of the FT817 S-meter is a bit limited, but it’s calibrated well enough to tell me if the noise level is similar to my house or getting near my target level of -135 dBm/Hz. Nowhere in my suburb is close, and the noise is not localised to my house.

I’m quite surprised a 30cm loop can receive 7 MHz signals at all, in a quiet location it works just fine as a receive antenna.

The occasional improvement in SNR with the loop is worth looking into. Maybe it’s the location away from the house. Or the lack of near B-field noise, compared to the near E-field noise that the dipole will respond too. It’s difficult to put my dipole in that location (or move it at all), as it’s so big and there are power lines nearby. The compact size of the loop is very useful for repositioning.

Time is a factor, my noise level varies over the day as various noise sources come and go. Over the course of the day I can see at least three different noise sources, for example (i) high energy clicks 20ms apart that seem to be coming from power lines (ii) some sort of (lighting?) noise from a neighbour in the evening, plus (iii) some other noise with a 30us period. Urban HF noise is a complex problem.

Reading Further

[1] Antenna Physics – an Introduction
[2] Antenna Theory Loop Antenna Page
[3] Experimental methods in RF Design, Ch2, Fig 2.34, Class A amplifier
[4] FT817 short circuit power supply repair – the exact fault I induced in my FT-817. This post was very helpful in fixing it!
[5] VK6CS Magnetic Loop Antenna Matching

Open IP over VHF/UHF 5

I’ve been having fun testing my data radio system over the air for the first time. This involved a few false starts, careful testing, tracking down a few bugs, and tuning the system to handle local EMI and strong pager signals. The good news is – it works! Using 10mW of transmitter power I have established a 1 kbit/s link over a 15km urban path using a RpiTx transmitter and RTL-SDR receiver. Plus lots of software.

My aim for this phase of the project was: don’t expect miracles with 10mW, but it should work down to the predicted Minimum Detectable Signal (MDS) over real world urban paths.

The configuration was similar to the previous post:

Terminal 1 (T1) uses a HackRF Tx and RTL-SDR Rx. Instead of a T/R switch I just use a hybrid combiner. I’ve also added a small “PA” to boost the HackRF signal to 10mW. Terminal 2 (T2) is the prototype pirip terminal. This uses a RpiTx transmitter (10mW output after the low pass filter), and RTL-SDR receiver, and a home made T-R switch.

I installed T2 at my home, connected to a vertical dipole about 8m high via some low loss coax. I installed T1 in my car with a commercial 2m antenna, and took it mobile so I could try to establish a link from various sites.

The frame repeater service on T2 allows me to “ping” it from T1. T2 responds to any valid packet it receives and sends one right back to T1. The system also gives me signal and noise power estimates from both ends of the link. I worked up some maths to estimate the Rx signal power in dBm from the FSK detector output. This lets me monitor received signal levels – and the local noise floor (noise spectral density) in real time.

So off I went for a drive, pinging away at 10 kbit/s. However the system was a bit deaf, I couldn’t get further than 500m before losing the signal. Even given a non line of sight path, that didn’t feel right. I tried reducing the data rate to 1 kbit/s. This was a bit better, out to 1.5km if I chose my location carefully. However the reported receiver powers didn’t look right, and I could see a big difference in the sensitivity at the two ends of the link.

Over the Cable Tests

A radio system has a lot of moving parts. A lot can go wrong, and when you do over the air tests it’s really hard to spot the problems. So I’ve learnt to perform Over the Cable (OTC) tests first. The basic idea is you connect the Tx and Rx via coax and enough attenuators to reduce the Rx power down to the predicted MDS. You keep working through the bugs until it works at the MDS.

So I took T1 and T2 back to the test bench to perform a bi-directional MDS test. At 1 kbit/s, this system should have a Minimum Detectable Signal (MDS) of -132dBm. I placed the two terminals in different rooms, and connected them via coax and lots of attenuation. The usual fun and games occurred, it’s quite hard to attenuate even 10mW (10dBm) down to -132dBm without some RF leaking around the attenuators.

The RF from the Pi was particularly hard to contain. I think it’s pretty poorly coupled into the coax, so the coax shield and Pi itself radiates a lot. So I installed T2 into a metal box, powered it from a battery, and provided just one hole for the SMA connector. With the lid firmly screwed down, this formed a nice Faraday cage.

That helped, I could now attenuate the signals right down to -132dBm. However T2 was quite deaf – about 8dB off theory. I traced this to EMI from the Pi getting into the antenna switch, which was stuffed into the box close to the Pi. To fix this I built a little shielded box from PCB material for the switch. Yayy – now both ends of the link were behaving, with packets getting through right down to the MDS of -129dBm for T1 and -132dBm for T2. T1 is 3dB less sensitive as it has a hybrid splitter instead of a T/R switch. This means the Rx signal is attenuated by 3dB before it hit’s the LNA of the RTL-SDR.

I found another problem that was making the system deaf. When T1 transmits a packet, some of that transmitted signal leaks through the hybrid to the T1 receiver. Turns out the leakage is really quite strong, and is enough to make the FSK demodulator deaf to any off air signals for the next few hundred ms. So T1 becomes deaf to the weak replies from T2. This is due to some averaging code in the FSK demod frequency offset estimator that gets “pumped up” to the mean received level. The work around was to put a delay in the T1 frame repeater service. When it receives a packet from T1, it waits a second before sending back the reply. This is not a practical solution for a real world use, so I’ll need to have another lap around the acquisition code in the near future. However good enough to let me proceed with the aims of the current round of tests.

Urban RF Nasties

My receiver has a fixed gain, set by the “-g” parameter on the RTL-SDR. A high gain means a low noise figure, but makes the receiver prone to overload. I hooked my antenna up to my spec-an and could see two problems that would impact my experiments:

  1. A very strong signal: -22dBm on 148.33 MHz, I think a pager from a multi-story hospital just 700m away.
  2. A lot of noise – the antenna “noise” temperature was equivalent to a noise figure of 20dB. I presume this is urban EMI. That really sucks, as it’s a big hit to my link budget. At the input to the RTL-SDR at full gain we have a 6dB noise figure. Added to that LNA noise is now the urban EMI that is 20-6 = 14dB higher. So the urban EMI will dominate, meaning roughly 14dB off my link budget, the equivalent of dropping my MDS from -132 to -118dBm.

I hooked up my antenna to my RTL-SDR and fired up gqrx to get a feel for what the RTL-SDR was “seeing”. Sure enough at full gain it was suffering a lot of overload problems when that pager fired up. I could actually listen to a local AM radio station quite clearly, that was being mixed up to VHF by the overload distortion. I could also see the level of a beacon being shifted down when the pager transmitted, a sign of receiver desensitisation. So I backed off the RTL-SDR gain to “-g 40” for T1 and T2. Based on my earlier measurements this means a RTL-SDR noise figure of 11dB. Note that at the T1 end (installed at my home), the urban EMI noise from the antenna (20dB) dominates, so the receiver noise figure doesn’t matter much. We would expect a MDS of around -118dBm.

So I hopped in my car and went mobile again. The system was breathing a little easier now. I parked at a few places around my suburb and by moving the car back and forth a few metres could often find a place where multipath worked for me and establish a link. However at these power levels I really need a line of site path, and where I live is quite flat. So I drove across town to the hills that overlook Adelaide. I started the system “pinging” from my car and sure enough I found a few spots that have a clear view over the plain and the link came up at a range of 15km!

Now 15km at 144MHz is a very neat 100dB attenuation over a line of site path (assuming 0dB gain antennas). So we would expect +10dBm-100dB = -90dBm at the receiver. Here is the log from my ping service:
1634971678 Rx Frame Rxloc: -116.08 Noloc: -157.56 snrloc: 11.48 Rxrem: -115.22 Norem: -153.21 snrrem: 7.99
1634971699 Rx Frame Rxloc: -112.90 Noloc: -157.50 snrloc: 14.60 Rxrem: -115.14 Norem: -153.46 snrrem: 8.32
1634971706 Rx Frame Rxloc: -114.00 Noloc: -157.30 snrloc: 13.29 Rxrem: -111.90 Norem: -153.30 snrrem: 11.40
1634971720 Rx Frame Rxloc: -120.02 Noloc: -157.54 snrloc: 7.52 Rxrem: -115.07 Norem: -153.36 snrrem: 8.29
Rxloc and Noloc are the local Rx and No (noise density) power estimates (the T1 terminal in my car). Rxrem and Norem are the Rx and No power estimates at the remote T2 terminal (at my house), 15 km away. The Rx power levels are nice and symmetrical. You can see the noise levels are bit higher at my house (Norem), due to all that urban EMI. This makes the remote SNR lower. The MDS of the system was estimated at -118dBm, a good match to our Rx power.

I didn’t have a perfect line of site path, so instead of the expected -90dBm, it was about -115dm, which means an extra 25dB of loss somewhere. However the link budget stacks up, and we have a working link over the top of a noisy urban RF environment. Yayyyyyy!

This is not a practical system yet, there are still quite a few building blocks to stitch together, and some more code (like a better acquisition system) to be written. However it’s clear the concept works, we really can send data over meaningful distances with simple hardware and open source software. A 2V red LED with 5mA of current consumes 10mW – the same Tx power as this system.

I’m really enjoying the sheer breadth of technologies in this project (propagation, antennas, EMI, shielding, RF hardware, link budget and strong signal analysis, modems, software decimators, power estimation maths, service scripts, Raspberry Pi). It’s fun being in control of and actually building so many layers in the system. A lot of moving pieces, a chance to learn many skills, and a lot that can go wrong too! Makes you appreciate what is inside commercial radios and chipsets, that we take for granted when we press the PTT button, or send an IP packet on our phones.

Next steps:

  1. Rework the acquisition system so it can handle strong signal immediately followed by a weak signal.
  2. A semi-permanent installation that runs for a month to gather long term stats and make sure nothing breaks in real world operation.
  3. A better link that can sustain 100 kbit/s. We need about 20dB more link budget for that. A directional antenna would be useful to try. My current antennas collect EMI power from all directions. A directional antenna would suppress EMI, and reduce the power from local strong signals, allowing me to bump up the RTL-SDR gain and enjoy a lower noise figure. A directional antenna would also increase the wanted signal receive power, further improving the link margin.

Reading Further

[1] Open IP over VHF/UHF Part 1 Part 2 Part 3 Part 4
[2] GitHub repo for this project with build scripts, a project plan

Sox play Desktop Application for Ubuntu File Manager

Every time I install Ubuntu I get stuck on this one. When browsing files on Ubuntu, I want to be able to click on a wave file icon and have it play with sox “play”. I don’t want it to launch some heavyweight audio editor or MP3 player. Just play my 3 second file without any fuss.

The problem is I only install Ubuntu every few years so I forget how I did it last time. A side effect of Linux being so reliable is you only need obscure set up steps rarely. So documenting the steps here so Google can augment my memory.

Place this code in ~/.local/share/applications/play.desktop

[Desktop Entry]
Name=play
Comment=play wave files
Exec=/usr/bin/play %F
Terminal=false
Type=Application
Categories=Audio
MimeType=audio/x-wav

Then in the file manager, select a wave file, right click Open With Another Application and “play” should be one of the selections: