Radiofax Tips: Modify kiwifax.py for Better DX Use, Slant Correction Factors & Status

The past few weeks have been quite busy with radiofax monitoring – a small project inspired by the existence of KiwiSDRs and WebSDRs and a desire to preserve what little “analog” utility stations we might have left on the HF/shortwave bands. This post looks at some of the tips I have for other potential monitors and how I’ve been going about my little hobby.

The Problem with kiwifax.py

As much as kiwifax.py has been a godsend to me, there have been a few niggles about its use for DX or monitoring purposes.

For one, it is optimised for use with APT start/stop tones and relies on its detection to initiate and stop printing. Some stations are known not to use the standard tones, resulting in either missing charts entirely or not ending when charts actually finish. The other is that the tones are often not long enough to guarantee successful detection during fading propagation conditions – this results in “missing” one chart in a sequence of charts, for example. Finally, I came across RBW41 which doesn’t use APT tones at all, so how can we get that to print?

There is one way at the moment, which is to use the -F force flag, but this only forces printing from the execution of the program up until the maximum number of lines. You could set the maximum number of lines to a fairly large value, but I find values over about 10,000 cause unacceptable wear on the SSD (as it seems to read the whole file, append a line and write the whole PNG back to disk) and slows down the decoding process dramatically.

As a result, modifications to the kiwifax.py script are probably the best way to overcome this issue and instead turn it into a continuously printing system. This way, it would be easy to leave it running and come back and not miss anything no matter how weak – and with a little post-processing, we can still determine rough start/end times and fix the vertical alignment.

I will repeat what I have mentioned in the past – kiwifax.py (printing or not) occupies one of the receiver slots on the target KiwiSDR. As many are operating in 3-user (20kHz), 4-user (classic) or 8-user (maximum) configurations, responsible use of kiwifax.py is encouraged to prevent occupying too many slots (i.e. concurrent instances) or continuously occupying slots denying others access to the SDR. Also note the existence of per-day maximum use times that some SDRs have which will restrict your capability to monitor “around the clock” – careful selection of SDRs which are less-frequently used, with no such time-outs and/or with higher numbers of slots would probably be best.

To “fix” kiwisdr.py and turn it into something useful for monitoring/DX purposes, it’s necessary to make a few changes.

The first change was to remove the whole start/stop score decision-making code. Deleting all the code highlighted above would cause the program to remain in whatever mode it is in and eliminate its ability to detect whether it should start/stop or phase.

To ensure it would be printing, it would probably be worth “unwrapping” the self._switch_state(‘printing’) line from the if statement. By removing the if-statement, the program will start up as if the force flag has been provided.

Then, we need to pay attention to the _process_row function – by looking at the condition for maximum height, the system changes the state to idle. But we don’t want that. If we just removed it, the PNG would keep growing and growing to no end – so we instead want to start a new file – hence add the two lines above: self._new_roll() and self._switch_state(‘printing’). There is probably a lot of unnecessary code with handling the other conditions, but this was the simplest way without breaking the program to make it do what I wanted it to do.

It is probably also worth changing the “default:” max_height value to something larger to ensure that fewer charts are “broken” across .png files. I normally use either 5,000 or 10,000 – coding it here means you don’t have to supply the command line option.

The good thing is that each PNG slice is “continuous” with the next – if a chart breaks across PNG files, there is no lines lost and you can piece it back together in an image editor. Each PNG slice is labelled with the start time – to know what time the actual fax started, just count the number of lines from the top and divide it by the scanning rate (e.g. 120 lines per minute most commonly).

The only downside seems to be that the code still bails on certain types of exception. While you could handle that within the code, I decided to wrap the whole thing in a .bat file that has a “wait” time before re-invoking the script allowing for unattended monitoring sessions. This is necessary as I’m currently connected via LTE where the session breaks very frequently.

With the right paths coded in the .bat file, you could even have multiple instances in separate folders monitoring the same station across different KiwiSDR receivers. I’ve even set the title of the window to something descriptive to help me keep track of all the instances I’m running. Just note that if you are doing so in the same folder, you might find filename conflicts are the result causing one instance of kiwifax.py to overwrite the results from another – separate folders are recommended.

Having the code available and free to modify is one of the great advantages of open source software – I’m very thankful that when I have a problem, I also have the tools to solve it!

My Radiofax DX/Documentation Workflow

It occurred to me that I never really mentioned exactly how I go about doing my radiofax monitoring and documentation, so it’s probably a good time to do so and explain why it sometimes takes so long …

The above basic flowchart was hurriedly drawn on a sticky-note but describes exactly the steps I take.

  1. The first step is to research the frequencies, times, transmission power, scanning rate/IOC and locations for the expected transmission. By knowing these parameters, it becomes easier to plan your reception.
  2. Knowing the location of transmission, one can then look on the map of SDR receivers to find the best candidates by location and see how many alternatives there are. It pays to keep a little notebook to take notes about these receivers. When a known scheduled transmission is on, open up these candidate SDRs and note their web addresses, signal strength, apparent signal to noise and whether there are any “daily use limits”. Sometimes due to out of date schedules, nothing is seen, so we must persist and check back at common fax times (e.g. on the hour) and stepping through the list of known active frequencies.
  3. Once a first sighting of the signal is spotted, then we know the target station is active and at what time. As a result, we can receive a chart or two to determine whether our information is correct regarding start/stop tones and scan rates. This might take a number of days as we may only know of very few known times of activity.
  4. The parameters for reception are refined by understanding which frequencies work best with which SDRs, which SDRs give good signals and the slant correction factor gets dialled in progressively.
  5. All the while, the results are being recorded in separate folders with sometimes overlapping reception via multiple receivers. All of the receptions are then collected and analysed to weed out duplicate receptions by selecting the best result, duplicate chart types by looking at the chart contents and fixing some reception issues (chopped images). Once they have been processed, then they can be uploaded and blogged about along with some research on the side.

As a result, it’s a fairly simple workflow but one that can easily take several weeks for a station especially when signals are being unkind or stations follow erratic schedules. The biggest impact to this invariably comes from stations where only very few candidate receivers exist and they are unreliable – going online and offline very frequently, resulting in a significant amount of frustration as I attempt to find a substitute (rarely possible) or wait it out until they’re online at just the right time with a free slot.

Slant Correction Factors

It is a bit of a miracle that analog radiofax works as well as it does, but one of the key problems often encountered is image slant. This is due to the synchronization of clocks being slightly different on the transmitter versus the receiver – for longer charts to maintain a straight alignment, the synchronization requirements can be quite stringent.

For example, a chart is 1809 pixels wide and each line is 500ms long in 120LPM mode, making each pixel approximately 276.4us. If a short chart like the one NWS/NOAA sends which takes 10 minutes to transmit is to be received in perfect alignment, then the allowable error is only around half-a-pixel or 138.2us in 10 minutes or 0.23ppm! Although to be honest, we could probably tolerate slight imperfections in the slant as long as the margins are not breached.

Lets be a little more generous – lets say we can “spoil” the complete margin area. From my calculations, the margin is 5% of the image line area based on BoM transmission parameters (475ms image, 25ms margin). We can now allow around 91px of slippage or 25ms, which means for a 10 minute fax, a stability of about 42ppm. When you consider the length of the Kyodo Fax news broadcasts, the allowable error might be even less, which is pretty amazing.

This is why slant correction factors are required which allow the decoding software to compensate for differences in timing. Traditionally with a conventional radio and sound card, sharing these figures are nearly meaningless as the error was frequently dominated by the sound card’s own clock error unless it had been calibrated out.

But in the case of kiwifax.py, as many KiwiSDR receivers are GPS-disciplined, this means they are pretty much as accurate as an absolute time reference. As a result, the same slant correction factors can be used across multiple GPS-locked receivers for the same results.

To aid other radiofax enthusiasts – I share my slant correction factors for stations I have received. I do so for your convenience to reduce the slant to acceptable levels out-of-the-box, but I cannot guarantee the accuracy of the factors due to differences in KiwiSDR and transmitting station drifts. These factors have been derived mostly from trial and error to get “close enough”, and are not 100% guaranteed to remove all slant. Note that in my observation, all stations within a given network typically exhibit identical slant behaviour.

Callsign	Correction
--------------- ----------
CBM/CBV		-90.0
DDH/DDK		-12.0
GYA		+34.0
HLL2		-18.0
HSW64		-11.0
NOJ/NMC/NMF/NMG	-4.0
KVM70 		-4.0
JFX		 0.0
JJC/9VF		+3.2
JMH		 0.0
RBW41		-17.0
VCO		-24.0
ZKLF		+11.0

Radiofax Station Status [10th February 2019]

I recently came across the listings from hffax.de from 2002 and a few random frequency lists. Owing to the unreliability and age of the data, I did cross-reference stations which are currently listed, removed those known to be no longer in existence or where more recent data is available.

This now leaves me with a new list split into two – below is the latest compiled list of the stations I know about with the activity status based on my recent monitoring efforts over the December 2018 to February 2019 period. The second portion are based on the historical data obtained and are very unlikely to still be alive – I just haven’t taken the time to confirm they are definitely offline.

Station Status as at 10th February 2019 (based on my observations)
CALL  COUNTRY           STATUS
----  ----------------- --------------------------------------------
ZSJ   South Africa      ? Unknown, Not Heard
JMH   Japan             - Alive and Well (Published)
???   Chukota Peninsula ? Unknown, Not Heard
BMF   Taiwan            X Confirmed OFFLINE (see above)
HLL2  South Korea       - Alive and Well (Published)
HSW64 Thailand          - Alive and Well (Published)
JJC   Japan             - Alive and Well (Published)
9VF   Singapore         ? Unknown (Relay for JJC)
GYA   Persian Gulf      X Reported Inactive, Not Heard
PWZ33 Brazil            ? Unknown, Not Heard
CB?   Chile             - Alive and Well (Published)
CFH   Canada            X Reported Inactive, Not Heard
VFF   Canada            ? Seasonal, Not Heard
VFR   Canada            ? Seasonal, Not Heard
VCO   Canada            - Alive and Well (Published)
VFA   Canada            ? Seasonal, Not Heard
NOJ   USA               - Alive and Well (Published)
NMC   USA               - Alive and Well (Published)
NMG   USA               - Alive and Well (Published)
NMF   USA               - Alive and Well (Published)
VM?   Australia         - Alive but (Temporarily) Unwell
ZKLF  New Zealand       - Alive and Well (Published)
KVM70 USA               - Alive and Well (Published)
SVJ4  Greece            - Alive and Well (Published)
RBW4? Russia            - Alive and Well (Published)
DD??  Germany           - Alive and Well (Published)
GYA   United Kingdom    - Alive and Well (Published)
???   Argentina         ? Unknown, Not Heard
LSB   Argentina         ? Unknown, Not Heard
CAN6D Chile             ? Unknown, Not Heard
OXT   Denmark           ? Unknown, Not Heard
ATP?? India             X Reported Inactive, Not Heard
5YE?  Kenya             ? Unknown, Not Heard
6VU   Senegal           ? Unknown, Not Heard
RTH   Uzbekistan        ? Unknown, Not Heard
JFX   Japan             - Alive and Well (Published)

* Published indicates a posting has already been put online for
with reception in 2019. For other "alive" stations, a posting will 
arrive once the fax images have been processed and sorted for 
publication.

HISTORICAL STATIONS from hffax.de (Most Likely Inactive)
CALL  COUNTRY           STATUS 
----  ----------------- --------------------------------------------
YMA20 Turkey            ? Unknown
R??7? Russia            ? Unknown
BAF?? China             ? Unknown
VLM   Argentina         ? Unknown
???   Argentina         ? Unknown
3MA?? Taiwan            ? Unknown
HM??? North Korea       ? Unknown, Not Heard
R??7? Ukraine           ? Unknown
CZW   Canada            ? Unknown
3SD   China             ? Unknown
LOR   Argentina         ? Unknown
BDF   China             ? Unknown
VNA   Vietnam           ? Unknown, Not Heard

That being said, there is one station on this list I am trying my best to receive and I have some evidence for it being still active. However, I have not yet updated this to reflect this because it’s not an easy station to catch with very limited possibilities to receive – so I don’t want to cause an “SDR traffic jam” just yet until I’ve concluded my investigation.

But as usual … there is more to look forward to – they just won’t be easy stations to find.

Posted in Radio | Tagged , , , , , | Leave a comment

Radiofax: RBW41 (?) Murmansk, Russia

Since we’re probably on the “tail-end” of the radiofax technological lifecycle, I’ve actually been quietly hoping to receive something from Russia as on historical information, it seems they’ve been rather unique.

While most stations today have settled on 120LPM with 576IOC, with Kyodo News using 60LPM, apparently many Soviet era stations operated 90LPM with 576IOC. There were other stations which operated IOCs including 288 and 352 but it seems rather unlikely to meet them now. Indeed, the last NWS/NOAA radiofacsimile schedule document seems to claim that RBW41 and RBW48 both are transmitting 120/576 with only the schedule broadcast at 90/576.

Thankfully, a reader by the name of Vitor informed me of the whereabouts and the transmission characteristics of RBW41 – namely 6328.5kHz (center frequency) with an inverted/negative shift with charts between 1300-1500 and 2000-2100UTC. With this tip-off, I decided to go searching … and alas, I can report some success!

Charts presented represent snapshots in time and are not kept up to date. Charts are not to be used for navigation purposes. Copyright in the faxed materials belongs to the transmitting agency in Russia, the exact entity is unknown. Charts are reproduced for documentary purposes to illustrate the technical characteristics of transmission, the formatting of the faxed information and as proof of activity of the station, propagation conditions and the reception of the charts at the stated times and dates.

Faxes

The station, believed to be RBW41 of Murmansk, Russia, required a change to my normal kiwifax.py-based workflow. For one, its location meant that it was hard to find a receiver that could get a good signal, so some signal hunting was required to optimise the choice of receiver. I first waited for a fax to appear based on listening into the closest station. Once certain a fax was being sent, I looked around to find the signal at neighbouring receivers. In the end, I found four receivers that provided visibility of the signal good enough to recognise a fax was taking place – SM2BYC, RZ3DVP, ArcticSDR and Kiwi-IVA.

To most efficiently monitor the station, a number of changes to the kiwifax.py script was necessary as RBW41 does not utilise APT start/stop tones. With some modifications, it became possible to monitor more effectively and in-parallel across receivers. Unfortunately, some receivers had daily limits, so I took care to choose the most advantageous time windows to use them.

A typical semi-decent reception actually looks like the above as the signal isn’t particularly easy to receive. As RBW41 is the only station I know of at present that uses negative shift where white is -400Hz and black is +400Hz, it results in an inverted picture when decoded using regular tools. To resolve this, you can invert the received image in an editor, use the invert feature in your decoder or tune 1.9kHz above the centre frequency and demodulate using the LSB mode.

Slant optimisation was performed incrementally – through trial and error, thus the later charts show reducing levels of slant. Faxes are presented in order of reception date and time.

I first received the station at 1330UTC on 3rd February 2019. Choosing the KiwiSDR closest to Murmansk, I was able to get mostly noisy copy but with a few “strong” lines. This gave me hope that the station was alive. Prior to the start of the image and after the image, tuning tone (-400Hz) was present. It appears that images are sent with a black border around them and a white margin in the image area.

The next day (4th), I tried again, but instead trying a different KiwiSDR nearby. This time, I was able to improve the reception a little more, and Russian text was now evident. The chart is definitely a surface analysis or prognosis of sorts.

On 5th February, I decided to try an SDR that was further but within Russia – I suspected this may provide a better copy possibly as the directionality of antennas may affect reception and this could be in the “rear” of the beam. The copy improved even more, to the point that a date is now visible in the copy. Scheduling at the station appears to be somewhat loose, with this fax coming in one minute early at 1329UTC.

Of note is that the image area of this chart exhibits wobbles which suggests the scanning timebase is not consistent. Most modern radiofax stations generate the signals electronically, resulting in crystal-locked accuracy or better, this may indeed be indicative of a “cold” mechanical machine that is slowly warming up and being consistently adjusted by a feedback loop to attempt to maintain a consistent scan rate. This suggests to me that this could be a modified photofax style machine which was used with telephones in the past to “wire” photographs for press usage just like in this video.

Owing to the better signal, other charts started coming out of the noise. Although QRM affected as well, this chart was received at 1030UTC on 7th February 2019 which is a first glimpse at a different type of chart.

The surface chart came a minute later on the 7th, being broadcast at 1331UTC. A little QRM was present and the white tone post-chart seemed to be a bit shorter than usual. This suggests the charts are sent manually.

Staying on-frequency was rewarding as at 1449UTC, a different type of chart was sent. There is a logo of sorts, but it seems the lack of horizontal scanning timebase stability has corrupted all of the text. It may indeed be IOC288 or IOC384 with its poor resolution.

On 8th February 2019, at 1331UTC, another surface chart is received.

However, it was then discovered that it was retransmitted at 2000UTC and this was received much better. This is seemingly as good as it gets for clarity, which is only barely sufficiently clear to read the larger numbers “if you squint”.

On 9th February 2019, some QRM seems to have “clobbered” the beginning of the fax, but this was received at 0330UTC, so it seems the station is actually active at other times as well.

Later on the same day at 1329UTC, the standard surface chart but reception is very poor. But you should never give up … since things can change.

Finally, at 2001UTC, the chart is again received rather pristine – it seems that propagation favours this time to the KiwiSDR receivers I chose.

Conclusion

It was interesting that around-the-clock monitoring of the “suspected” RBW41 station’s frequency was able to turn up charts being sent on a regular schedule. While it was not as exotic as I expected, using 120LPM mode, the station does not utilise APT tones and uses a negative shift which makes it unique.

The charts sent are all short-form charts of a fixed size. That, along with the lack of APT tones, low clarity, sometimes wavering edges and inverted images makes me suspect this is being “scanned” by a modified photofax machine formerly used to transmit photograph prints over telephone lines. Apparently, this was common amongst “negative” printing stations.

As a result, receiving the station is tricky due to weak signals, propagation issues, lack of APT tones – even when all the stars align, the text on the charts are almost unreadable, clumping together. It is, however, still noteworthy that the station does still operate and it’s all thanks to Vitor for letting me know where and when to look for it.

Posted in Radio | Tagged , , , , | Leave a comment

Radiofax: JFX Kagoshima (Japan Fisheries)

Having already pursued the majority of the rather “well known” fax stations, I decided to move along and try my luck at some of the lesser-known stations with spotty data as seen from some older listings and SWL schedule sites. The station that stood out as being the most promising was a Japanese Fisheries station, JFX Kagoshima Fishery Radio Station.

Frequencies I’ve found listed included 6414.5, 8658, 13074, 16907.5 and 22559.6kHz which according to a recent posting from Utility Planet, are shared between JFC and JFW.  I also saw 8616 and 17231kHz as potential frequencies on older lists as well.

As a result, I went trawling for faxes and to my surprise, I caught some. I had to use a slightly modified technique (more on this in a future posting) but I managed to use the AKKY and JH1PGF KiwiSDRs to receive faxes on 8658, 13074 and 16907.5kHz. The other frequencies did not have sufficient signal to be received possibly due to poor propagation or inactivity.

The faxes shown represent a snapshot in time from 7th to 9th February 2019 and are not updated. Faxed material should not be used for navigation or fishing purposes. Copyright of the faxed material belongs to the transmitting agency, believed to be Kagoshima Fishery Radio or JAFIC. The faxes are reproduced for documentary purposes illustrating the technical characteristics of the transmissions, formatting of the content, activity schedule of the station, propagation characteristics of their signal and proof of reception at the given dates and times.

Faxes

The station was monitored for a number of days to optimise frequency and receiver choice, as well as obtain the best copy that I could given my resources. While I was hoping for a schedule, there doesn’t seem to be a transmitted schedule. I received most of the charts clearest on 8658kHz, with exceptions noted. Note that chart dates are all in Japanese format (Heisei 31 representing 2019). Chart transmission timebase was very stable with no slant correction required.

One of the most notable charts transmitted is a surface analysis/prognosis chart which is broken into eight sub-figures. This chart is sent as a rather tall chart and demonstrates JFX’s regular transmission characteristics. Similar to JJC, there is generous white-tone prior to the commencement of the fax allowing for easier tuning to the transmission. Prior to the start tone, inverted phasing ticks appear. Start tone is relatively long and a stop tone is also used. Image margins are black. This fax was received at 0020UTC on 9th February 2019. It should be noted that a number of these charts seem to be repeated throughout the day – times are given for the received example only but it is possible to receive the same chart at other times.

A different type of weather chart is also available, this appearing to be a sea surface temperature chart (in the noise which seems due to poor propagation and QRM). This chart makes reference to the JAFIC website and was received 0700UTC on 8th February 2019 on 13074kHz. Unfortunately, I only received one sample during the monitoring period, thus perhaps it may be sent depending on day of the week (as many Japanese stations tend to have odd schedules).

Navigational warnings or NX are sent, the above example being a Japanese Coast Guard NX received 7th February 2019 at 0930UTC.

In addition, specific warnings are provided for regions, the above is for Nagasaki received 2200UTC on 7th February 2019.

A shorter version also was received covering Kagoshima, received 2240UTC on 9th February 2019.

A third variant covering Okinawa was received 2300UTC just following the above.

JFX’s unique value appears in providing market pricing information to fishermen. This example is from 8th February 2019 at 0830UTC and appears to be Kagoshima Market Prices.

The chart does change from time to time – in this case, it seems that one of the markets is closed. This example from 9th February 2019 at the same time.

This chart also appears to be a market price chart, but instead for deep sea tuna (I’m guessing). Received 0330UTC on 9th February 2019.

There is also a chart titled JFX NOAA with a number of cryptic codes and brands of marine equipment (JRC/Furuno) named. I wonder if this represents data for fish finder equipment for marking favourable fishing zones. Received 1800UTC on 8th February 2019.

There appears to be some evidence for sharing of the channels, as some rather “dirty” faxes were barely received, which is seemingly uncharacteristic of the JFX faxes above. I suspect these originate from another station (JFC/JFW) at a different location resulting in poor propagation but I thought I’d include them here (pending further investigation in the future). The above appears to be some sort of form, received 0235UTC on 9th February 2019.

The above appears to be a notice of sorts, received 0535UTC. Both these charts were received at xxx5UTC and both were received on 13074kHz, perhaps providing key indication that it is a different station.

Conclusion

Just as with telephony fax, it seems that radiofax is most alive in Japan, where different types of fax services still persist. JMH provides the weather, while JJC provides the news and JFX provides fishing-oriented information. JFX appears to have a rather limited selection of charts which provide predominantly textual information to fishing vessels including market prices, safety warnings, specific codes for navigation and basic weather information. The format is not unlike JMH with black margins in the image border and generous tuning tones, however, as seen in the charts above, all are in Japanese.

Posted in Radio | Tagged , , , , | Leave a comment