Category: Stratoballoon

Balloon Mapper

markegilbert1 Comment

Our receiving station for the Stratoballoon project consisted of three major pieces of software:

I’ve gone into great depth about how the first two are configured on my technical blog (“Stratoballoon Radio Details”).  Today, I want to talk about the third one, BalloonMapper.  Here is the basic interface:

10

We would copy a dataline out of dl-fldigi and past it into the “Data Line” box (#1).  We could then hit Enter, or manually click the “Find Position” button in the upper right corner.  This would extract the altitude, and the lat/long from the data string.  It would convert the former into feet and display that in the “Altitude” box (#2).  It would also create the Google Maps URL that would show the lat/long as a pin on the map.

What did the data strings look like?  A picture-perfect data string that came off the radio would look like this:

KD8VZA-KD8VZA-KD8VZA,120057.00,234.11,4214.7460,8533.3750,*4FF4

This is a comma-delimited string, and is made up of the following:

  • My callsign, repeated 3 times
  • A timestamp, in hhmmss.xx format (where the decimal places are ignored)
  • The altitude, in meters
  • The latitude (format explained below)
  • The longitude (format explained below)
  • A termination string, made up of an asterisk, followed by four alphanumeric characters

In most cases the callsigns came out a bit garbled, so it would look more like this:

a8czZA-KD8VZA-KD8VZA,120057.00,234.11,4214.7460,8533.3750,*4FF4

The first part of the string got chewed up because it took us a second to tune dl-fldigi to lock onto the signal.  That’s the main reason I start the string with three copies – I wanted to give Katherine or I (whoever was working the radio) a second or two to get ahold of the signal.

Extracting the altitude was very straightforward.  Simply grab the 3rd piece of data in the string, multiple it by 3.28 to convert it from meters to feet, and display it in the box.

        public static String GetAltitude(String DataLine)
        {
            String[] DataLineComponents;
            String RawAltitude;

            DataLine = (DataLine ?? "").Trim();
            if (String.IsNullOrEmpty(DataLine)) { return ""; }

            DataLineComponents = DataLine.Split(',');

            RawAltitude = DataLineComponents[2];

            return String.Format("{0} ft", (double.Parse(RawAltitude) * 3.28).ToString("0"));
        }

The lat/long was a bit tricker.  First, I had to get them into a format that Google Maps would understand.  You can browse directly to a specific lat/long point on Google Maps like so:

https://www.google.com/maps/place/42°17’44.76″N+85°43’22.50″W

The lat/long values, however, always come off the radio in the 4-dot-4 pattern.  Here is how they broke down:

4214.7460 = 42° 14.7460′

8533.3750 = 85° 33.3750′

So, I would need to split the degrees from the rest of the string, then convert the fractional arc-minutes into arc-seconds, before I could drop it into Google Maps:

        public static String GetUrl(String DataLine)
        {
            String[] DataLineComponents;
            String RawLat, RawLong, FormattedLat, FormattedLong;

            DataLine = (DataLine ?? "").Trim();
            if (String.IsNullOrEmpty(DataLine)) { return ""; }

            DataLineComponents = DataLine.Split(',');

            RawLat = DataLineComponents[3];
            RawLong = DataLineComponents[4];

            FormattedLat = String.Format("{0}°{1}'{2}\"", RawLat.Substring(0, 2),
                                                          RawLat.Substring(2, 2),
                                                          (double.Parse(RawLat.Substring(4)) * 60).ToString("00.00"));
            FormattedLong = String.Format("{0}°{1}'{2}\"", RawLong.Substring(0, 2),
                                                           RawLong.Substring(2, 2),
                                                           (double.Parse(RawLong.Substring(4)) * 60).ToString("00.00"));

            return String.Format("https://www.google.com/maps/place/{0}N+{1}W", FormattedLat, FormattedLong);
        }

So, now I had my URL.  I needed a way to embed a browser into BalloonMapper, which was a Windows desktop app.  I looked at a few options, but eventually settled on Gecko Effects: https://bitbucket.org/geckofx/geckofx-29.0/downloads.

I created a Gecko.GeckoWebBrowser object on my form called “GoogleMaps”.  To browse to a URL, I simply called the .Navigate() method on the GoogleMaps object, and passed it the URL that I generated above:

                this.GoogleMaps.Navigate(GetUrl(this.DataLineBox.Text));

 

Using it was easy.  Getting the control installed and working was a little more difficult, and I had a few false starts.  In the end, here is what worked:

  • I downloaded Gecko Effects 29.0 (see link above).
  • Gecko requires another application called “xulrunner”, which I obtained from here: http://ftp.mozilla.org/pub/mozilla.org/xulrunner/releases/29.0.1/runtimes/ .
  • I found that I needed to match the version of xulrunner to the version of Gecko, otherwise I got a “Specified cast is not valid” error.
  • I also found that the program has to be installed to a folder called “xulrunner” – all lowercase – or it wouldn’t work.

This made finding the capsule’s current location extremely easy:

20

For full source code and binaries, please visit http://TinyURL.com/MarkGilbertSource, and look for the BalloonMapper.zip archive.

Advertisement

Come in ground control: Stratoballoon sketch updated for radio transmitter

markegilbert

It’s been a long time coming, but the main Balloon sketch has finally been updated to include code to control the radio transmitter.  And true to form, the transmitter had what I hope are the final few instrument pack monkey wrenches for us.

I detailed in Episode 22 the work needed to get the NTX2 board attached to the instrument pack.  Once I had that in place, I could incorporate the logic for transmitting over it from the RTTY test radio sketch we had been playing with for the last few months.

Monkey Wrench #1

Before I could drop those pieces in, however, I needed to relearn how to do string concatenation.  I wanted to transmit my call sign, a timestamp, altitude, latitude and longitude in a single comma-delimited string.  These values are a string, and four floating point values, respectively.  If this were straight C, I could use sprintf and format the floating point values to whatever precision I needed.  However, the Arduino C libraries don’t support sprintf for floats.

I could, however, use another built in function call “dtostrf()” which would convert my floats into a character array.  From there, I could use sprintf to concatenate the strings together into my “TransmitBuffer”.

Monkey Wrench #2

Doing that, however preserved all of the whitespace in the character arrays.

KD8VZA,            21515.00,              314.28,              0.0000,              0.0000,*9395

It’s at points like these where I realize how spoiled I’ve gotten using a language like C# that supports things like “.Trim()”.  Unfortunately, Arduino C doesn’t, so I had to roll my own.

What I ended up with was a routine that would copy one array into another, one byte at a time:

…

// This copies the SourceBuffer into DestBuffer starting at DestIndex,
// but trims out any leading spaces in SourceBuffer
int TransferBuffer(char* SourceBuffer, int SourceBufferSize, char* DestBuffer, int DestIndex, boolean ShouldAddComma) {
int BufferIndex = 0;

// Skip all characters from 0-32, but don’t count carriage returns, chr(13)
while (BufferIndex < SourceBufferSize-1
&& SourceBuffer[BufferIndex] <= 32 && SourceBuffer[BufferIndex] != 13) { BufferIndex++; }
while (BufferIndex < SourceBufferSize-1) {
DestBuffer[DestIndex] = SourceBuffer[BufferIndex];
DestIndex++;
BufferIndex++;
}

if(ShouldAddComma) {
TransmitBuffer[DestIndex] = ‘,’;
DestIndex++;
}

return DestIndex;
}

…

The TransmitBuffer method takes the source and destination arrays, the size of the source array, and the current position in the destination array where it needs to start writing.

The method starts by skipping past any whitespace or non-printable characters in the source array, except for carriage returns (character 13).

Once I know where in the source array I need to start copying, I copy the bytes into the destination array one by one, until I reach the end of the source array.  In that main loop, I’m keeping track of where I am in the destination array – this index will be the function’s return value.

The very last thing I do it check to see if I should append a comma to the end of the new string.  That is determined by the last parameter to the TransferBuffer array, ShouldAddComma.

Putting Monkey Wrenches 1 and 2 together

My original version of the TransmitDataToGround function uses these two solutions – dtostrf and TransferBuffer – to combine the call sign, timestamp, altitude, latitude, and longitude into a single string that can be transmitted over the NTX2:

void TransmitDataToGround(float TimeStamp, float Altitude, float Latitude, float Longitude) {

// Build the string to transmit
dtostrf(TimeStamp, 20, 3, TimeStampBuffer);
dtostrf(Altitude, 20, 3, AltitudeBuffer);
dtostrf(Latitude, 20, 5, LatitudeBuffer);
dtostrf(Longitude, 20, 5, LongitudeBuffer);

int TransmitIndex = 0;
TransmitIndex = TransferBuffer(CALL_SIGN, sizeof(CALL_SIGN), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(TimeStampBuffer, sizeof(TimeStampBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(AltitudeBuffer, sizeof(AltitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LatitudeBuffer, sizeof(LatitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LongitudeBuffer, sizeof(LongitudeBuffer), TransmitBuffer, TransmitIndex, true);

unsigned int CHECKSUM = gps_CRC16_checksum(TransmitBuffer);  // Calculates the checksum for this datastring
sprintf(TransmitBufferChecksum, “*%04X\n”, CHECKSUM);
TransmitIndex = TransferBuffer(TransmitBufferChecksum, sizeof(TransmitBufferChecksum), TransmitBuffer, TransmitIndex, false);

rtty_txstring (TransmitBuffer);
}

The logic for calculating the checksum, and the rtty_txstring() function which does the actual transmission, come from the UKHAS site.

Monkey Wrench #3

Once I had TransmitDataToGround() in place, I needed to determine how frequently to call it.  I ultimately decided that I would transmit every 10th reading (where a “reading” is a sampling of the temperature, pressure, and GPS sensors).  I updated the last section of the main loop logic (which checked that it had a good temperature and GPS reading before logging the reading) to include this call.

// Now that we have a good GPS and temperature reading, grab
// the rest of the data, and log it.
if(HasGoodGPSReading && HasGoodTempReading)
{
CurrentPressure = GetPressure(bmp085ReadUT(), bmp085ReadUP());
CurrentAltitude = CalculateAltitude(CurrentPressure);

PrintToSerialOutput(CurrentTimeStamp, CurrentTemp, CurrentPressure, CurrentAltitude, CurrentLatitude, CurrentLongitude);
WriteDataToLogger(CurrentTimeStamp, CurrentTemp, CurrentPressure, CurrentAltitude, CurrentLatitude, CurrentLongitude);

if(TransmitNumber > TRANSMIT_EVERY_Nth_READING) {
TransmitDataToGround(CurrentTimeStamp, CurrentAltitude, CurrentLatitude, CurrentLongitude);
TransmitNumber = 0;
} else {
TransmitNumber++;
}

HasGoodGPSReading = false;
HasGoodTempReading = false;
}

 

I added a new loop index called “TransmitNumber” that I would increment with every reading, and reset when I invoked TransmitDataToGround().

All appeared to be fine at first – the data was coming through exactly as I had expected.  However, I noticed that the radio wasn’t ever turning off in between transmissions.  During the nine readings that weren’t being transmitted, it would generate a solid tone.  I needed to be able to programmatically enable and disable the transmitter.  That led me to rewire the NTX2 – for full details, see the Episode 22 Addendum.

Once I had digital pin 5 wired up properly, I reworked TransmitDataToGround() to turn the transmitter at the beginning, and turn it off again at the end.

void TransmitDataToGround(float TimeStamp, float Altitude, float Latitude, float Longitude) {

// Build the string to transmit
digitalWrite(RADIO_ENABLE_PIN, HIGH);

dtostrf(TimeStamp, 20, 3, TimeStampBuffer);
dtostrf(Altitude, 20, 3, AltitudeBuffer);
dtostrf(Latitude, 20, 5, LatitudeBuffer);
dtostrf(Longitude, 20, 5, LongitudeBuffer);

int TransmitIndex = 0;
TransmitIndex = TransferBuffer(CALL_SIGN, sizeof(CALL_SIGN), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(TimeStampBuffer, sizeof(TimeStampBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(AltitudeBuffer, sizeof(AltitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LatitudeBuffer, sizeof(LatitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LongitudeBuffer, sizeof(LongitudeBuffer), TransmitBuffer, TransmitIndex, true);

unsigned int CHECKSUM = gps_CRC16_checksum(TransmitBuffer);  // Calculates the checksum for this datastring
sprintf(TransmitBufferChecksum, “*%04X\n”, CHECKSUM);
TransmitIndex = TransferBuffer(TransmitBufferChecksum, sizeof(TransmitBufferChecksum), TransmitBuffer, TransmitIndex, false);

rtty_txstring (TransmitBuffer);
  digitalWrite(RADIO_ENABLE_PIN, LOW);
}

Monkey Wrench #4

I found that when I enabled the transmitter, I was losing several characters off of the beginning.  This was because dl-fldigi needed 2-3 seconds between when the transmitter booted up until it actually started sending data to find the signal again, and start decoding it.  I added a 3-second delay to TransmitDataToGround, right after I enabled the transmitter, to provide this buffer.

void TransmitDataToGround(float TimeStamp, float Altitude, float Latitude, float Longitude) {

// Build the string to transmit
digitalWrite(RADIO_ENABLE_PIN, HIGH);
  delay(3000);
dtostrf(TimeStamp, 20, 3, TimeStampBuffer);
dtostrf(Altitude, 20, 3, AltitudeBuffer);
dtostrf(Latitude, 20, 5, LatitudeBuffer);
dtostrf(Longitude, 20, 5, LongitudeBuffer);

int TransmitIndex = 0;
TransmitIndex = TransferBuffer(CALL_SIGN, sizeof(CALL_SIGN), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(TimeStampBuffer, sizeof(TimeStampBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(AltitudeBuffer, sizeof(AltitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LatitudeBuffer, sizeof(LatitudeBuffer), TransmitBuffer, TransmitIndex, true);
TransmitIndex = TransferBuffer(LongitudeBuffer, sizeof(LongitudeBuffer), TransmitBuffer, TransmitIndex, true);

unsigned int CHECKSUM = gps_CRC16_checksum(TransmitBuffer);  // Calculates the checksum for this datastring
sprintf(TransmitBufferChecksum, “*%04X\n”, CHECKSUM);
TransmitIndex = TransferBuffer(TransmitBufferChecksum, sizeof(TransmitBufferChecksum), TransmitBuffer, TransmitIndex, false);

rtty_txstring (TransmitBuffer);
digitalWrite(RADIO_ENABLE_PIN, LOW);
}

Now when the radio turns on, the waterfall in dl-fldigi looks like this:

10

The blank horizontal line in the waterfall above shows this 3-second delay.  The red line that starts out horizontal, and then quickly curves to be vertical is the data coming over the airwaves.  That red line, and the accompanying one to the left, need to be lined up with the two thin red lines that make up the dl-fldigi cursor.  Until those two pairs of lines line up, nothing is decoded.

I also found that the transmitter will drift slightly while it is transmitting, and that the dl-fldigi cursor will drift in between transmissions.  The 3-second delay gives me a chance to adjust the cursor if I need to when the next transmission starts.

Monkey Wrench #5

Even with the delay, I found I was still losing characters off the beginning of the transmission string.  I solved this by simply appending more than one copy of my call sign to the beginning.  As a result, the transmitted strings usually ends up looking like this now:

jf7&VZA-KD8VZA-KD8VZA,21515.00,314.28,0.0000,0.0000,*9395

Finally, I adjusted the squelch control in dl-fldigi to avoid having so much garbage between transmissions appear in the output window.  I still get some, but it’s not the constant barrage that it was when I started.

***

It feels good to be able to put this piece of the balloon to rest.  I may end refactoring the code a little, or adjusting it based on our instrument pack endurance/distance test, but otherwise I think its basically there.

You can get the newly completed sketch from GitHub.

Stratoballoon Radio Details

markegilbert

Towards the end of our last podcast, I did a bit of hand-waving when it came to how we sent and received the test beacon data over the radio.  I quickly walked through the setup of the major pieces of software involved, but skipped over all of the details.  Today, I’ll be walking through those weeds.

On the transmission side, we have the second sample sketch available from the UKHAS site (link: http://TinyURL.com/UKHAS-NTX2 ) running on the Arduino sending the test beacon data through the NTX2 transmitter.

On the receiving side, there are actually three pieces of software at work:

  1. SDR#, pronounced “SDR-sharp”, which is used to pull the raw radio signals off of the NooElec receiver
  2. dl-fldigi, which is used to translate the audio signals back into data
  3. VB-Audio Virtual Cable, which as the name implies is a virtual audio cable between SDR# and dl-fldigi

When I first got the SDR in the mail, I didn’t try transmitting the test beacon data right away.  My first test was actually to get the base software installed and try tuning in a local radio station.  For that test, I found some extremely helpful comments on the Amazon page that I purchased it from.  Those instructions involved going to NooElec’s site, downloading the Zadig driver installer and SDR# from there.  I did, and was in very short order up and running.  Here are the exact steps I took:

  • Went to NooElec’s site, and the product page for the R820T SDR Receiver: http://www.nooelec.com/store/sdr/sdr-receivers/tv28tv2-sdr-dvb-t-usb-stick-set.html#.UzbdOahdXNk
  • Went to Downloads, and downloaded the SDR# AutoInstaller
  • Extracted the downloader, and ran the Install.bat that came with it
  • Plugged in the USB stick. When Windows prompted me to install drivers, I cancelled it.  I can’t remember if it asked me to do this more than once, but if it did, I cancelled both attempts.
  • I ran Zadig.exe.  This is one of the EXEs that was installed to the /sdrsharp folder created by Install.bat.  It should have had WinUSB selected by default. I clicked “Install Drivers”.  I closed Zadig when it finished.
  • I plugged in the antenna.
  • I ran SDR#.
  • To initially test it, I first selected “RTL-SDR / USB” from the drop down (my device), then clicked “Configure”.

10
And configured it as follows:

20
I think the key things that I tweaked on this dialog were the Sample Rate and the RF Gain.

  • I clicked Close to return to the main screen.
  • I set the frequency to 107.7 MHz, a local rock station, checked “WFM” (Wide FM) and hit “Play”.

30

That worked – I could hear the radio station coming in fairly clearly, even in my basement, and I could then use the mouse wheel to change frequencies to other radio stations.  I could also change it manually.

Now that I had the receiver and SDR# working, it was time to try it out with the NTX2.  To decode the data, we would be using dl-fldigi, which was a free download.  While the source code is available off of Github, there are installers and binaries available off of the UKHAS site here: http://ukhas.org.uk/projects:dl-fldigi.  Once that was installed, I fired it up.

I initially spent 15 minutes or so trying to figure out a way to feed the output of SDR# directly the input of dl-fldigi, but didn’t have any luck.  I began to wonder if I needed something else to bridge the gap.  I found yet another page on the UKHAS site that described just such a bridge: VB-Audio.

After I installed this, I rebooted my machine, and then followed the rest of the instructions found here:

  • Go into Control Panel / Hardware and Sound / Sound / Recording
  • You should see an input called Virtual Audio Cable.  Click properties and rename it to SDR.

40

  • Click the “Listen” tab
  • Check the “Listen to This Device” checkbox

50

  • Close the dialog
  • Stop SDR# and under the Audio section select “[MME] CABLE Input (VB-Audio Virtual C”.  (I also played around with the main settings, and ultimately landed on a Filter Type of “Blackman-Harris 7” and a Filter bandwidth of “2400”.  I honestly don’t know how much of a difference this makes yet.)

60

  • I clicked start, but had to do a couple of things before I could hear the test beacon through my speakers:
    • I had to change the receiver from WFM mode to USB, or “Upper Side Band”

70

    • I had to position the red line just to the left of the signal, otherwise it wouldn’t work.

80

    • Even though the NTX2 reports that it was transmitting at 434.650 MHZ, I could only really pick up the signal if I tuned the receiver to 434.635 MHZ.
  • Next, I started dl-fldigi again.  I clicked Op Mode / RTTY / Custom / Audio / PortAudio / Capture, and selected “CABLE Output (VB-Audio Virtual” from the drop down.  I clicked Close to return to the main screen.

90

Before I could actually see the test beacon correctly, I had to be sure that dl-fldigi was configured to decode it.  The instructions for this are described in http://TinyURL.com/UKHAS-NTX2, and the actual configuration is done under Op Mode / RTTY / Custom / Modems:

95

Finally, once I had everything configured, I had to adjust where the two red “data” lines appeared in the waterfall trace by tuning SDRSharp.  Once I could see that, I could match the two red “tuning” lines with the data lines, and actually start seeing data appear in the data box.

100

I found a few different ways that the tuning lines and the data lines wouldn’t match up:

First, if the two in the latter were too far apart, they wouldn’t match up with the data lines in the waterfall trace.

110
To fix that, I needed to adjust the carrier shift (otherwise, I wouldn’t see the data coming through).

120

 

Second, if the red lines were lined up with the signal in the waterfall, I would only see gibberish in the data window.

130

To fix that, I simply clicked the left data line in the waterfall to force the tuning line to match it.  The other line would stay in lockstep, always the same distance away (as determined by the carrier shift).

Finally, I also found that as I tuned SDR# up or down, the data lines in dl-fldigi would slide left or right.  If I let the signal slide away from the SDR# band completely, it would disappear from the waterfall trace in dl-fldigi.  This is something I need to keep in mind if I have problems getting the data to come in properly when we were out in the field trying to track the balloon while it was in flight.

There you have it.  The basic setup for our transmission and receiving station.  We still have a long way to go to getting the NTX2 fully incorporated into the instrument cluster, but Katherine and I feel we at least have a good foundation now.