Category Archives: Electronics

Postscript

I am a part of all that I have met;
Yet all experience is an arch wherethro’
Gleams that untravell’d world, whose margin fades
For ever and for ever when I move.
–Tennyson, “Ulysses

With the completion of the NASA report on the balloon launch mishap, the last chapters of our Australian saga have been completed.  The report has a hugely detailed accounting of all that happened that day, but I can at last share the missing piece of this blog’s narrative.  The immediate cause of the mishap was a mechanical failure in the launch crane’s release mechanism, which prevented the balloon and our instrument from lifting off of the crane at the proper time.  As the crew positioned to abort, the balloon pulled our instrument off of the crane.  The whole world saw the consequences.

After extensive review, NASA has resumed regular balloon operations with a re-engineered release mechanism and improved launch procedures; two scientific payloads were successfully launched in Antarctica in December.  The HERO team will return to Alice Springs early this year to complete their delayed campaign.

As for NCT, upon its return to the U.S. we were pleased to find that despite the extensive damage to the gondola, the instrument’s key components were remarkably unscathed.  The detectors, electronics, and shields all appear operable.  We hope to rebuild, using our experience to make improvements where possible, and to fly again someday soon.  For now, though, NCT’s future remains uncertain.

I won’t be part of that campaign, though.  I’m finishing up my PhD this spring and looking towards new opportunities.  Still, I will always be grateful for the experiences I had—and the people I shared them with—while I was gone ballooning.

Picking Up the Pieces

Today was a terrible day for a lot of people.  For the NCT team, we’ve poured our hearts into this instrument for years.  It was an almost unfathomable shock to find ourselves cleaning up the wreckage of our gondola rather than watching it lift off towards space.  I’m very grateful for the outpouring of support from friends and colleagues around the world–it really does help.

Given the media attention this incident has brought, I’m inclined not to discuss the crash in too much detail.  Obviously there will be a full investigation into what went wrong today, so it doesn’t seem helpful to add premature speculation.  I have a tremendous amount of respect for the CSBF personnel, and I’d like to avoid complicating the process for them.  In short, NCT came off the launch vehicle badly and hit the ground several times as the abort completed.  The aftermath you can see below.

NCT’s core components appear to have come through remarkably unscathed.  The cradle landed upright, and the detectors and shields appear undamaged.  The card cages were scattered about, but their exteriors show only minor damage.  The electronics bay was destroyed, though, and all of the systems in it suffered some degree of damage.  Virtually all of the cabling snapped.  Many of these systems have been tested and used for decades.  They have become so familiar that their loss feels oddly personal.

We’ve cleaned everything up and brought it back to the hangar.  Sincere thanks are due to the HERO team for their calm and capable help with the gondola recovery.

Tomorrow we’ll–it’s so hard to say–start packing up for home.

Compatibility

Despite lingering overcast, today worked out to be a good enough day to perform our compatibility test with CSBF.  While not nearly as intense as last year’s marathon, it still made for a long day.  Since the compatibility test serves partially as a dress rehearsal for launch, we spent much of the day working through our button-up, roll-out, flight line, and roll-in checklists.

Except for a bad transmitter cable on the CSBF side, the tests went smoothly, with no major problems.  We’ll try to optimize our checklists a bit to make them clearer and faster, but on the whole I think today’s uneventful day is a sign of success.  We have a few minor odds and ends to wrap up, but judging by the weather forecast it seems our next project will be cultivating patience…

Compatibility brings everybody out.

Attaching crush padding.

Jane and Boggs keep an eye on things.

Attaching the ballast hoppers.

Integrated!

Frank and Joseph mind the gondola and the electronics.

A bit of sunshine!

Waiting...

Going for a spin.

Fit

The last few days have been so momentous that there hasn’t been a spare moment to post about it.  Now that leaves me the challenge of summarizing recent events!

With the total system coming nicely together, we set ourselves a goal of performing a compatibility test with CSBF by the end of the week.  Compatibility is the last major task before declaring flight readiness.  It involves hanging the balloon gondola from the launch vehicle and performing mechanical, electrical, and communications tests to ensure that all interfaces will work correctly on the flight line.  Given the science instrument’s reliance on CSBF equipment for commanding, telemetry, ballast, launch, and cutdown, it is a crucial test for ensuring a successful flight.

With that goal in mind, we set about checking off the final few little items on our checklist.  Wednesday morning we rolled outside early to gather a little bit more data from our aspect systems–our differential GPS, our precision magnetometer, and the magnetometer we use to point the gondola relative to the Earth’s magnetic field.  Unfortunately, just when we got lined up with our north-south reference, the venerable back tire on the gondola cart exploded!

Definitely the last campaign for this tire...

The CSBF mechanical crew volunteered to find a new wheel for us over lunch, but without the ability to maneuver the gondola, we had to postpone the pointing tests.  With that task tabled, we asked the CSBF electronics crew if they could mount the SIP, the electronics package which rides underneath our gondola and handles all of the communications.  Despite the extremely short notice, seemingly the entire CSBF crew appeared instantly at our gondola and set to installing the SIP and its many accessories.  Installing the SIP occupied most of the rest of Wednesday afternoon, but the task went smoothly and we confirmed that all the interfaces worked flawlessly.  One of the advantages of legacy hardware and flight code is that the bugs have been worked out many flights ago!

The electronics crew starts the SIP.

Wednesday afternoon and Thursday morning we spent on some thankless, tedious, but crucial jobs:  tightening all the jack screws on all of the cables not yet firmly connected.  While we are testing the system, we leave the cables plugged in through friction alone.  Sometimes this causes a cable to work lose and give buggy readings, but it’s far easier to determine that and plug it in again than it is to screw things in and out every time we need to make a small change.  Still, it leads to a Augean task at the end.  Many of the screws–particularly on the flight computer–are very difficult to access.  Others are bent, twisted, or nearly stripped.  We got everything fastened eventually, though, so the cables should remain secure for flight.

Two other small tasks proved time consuming.  We distributed small temperature sensors throughout the gondola, taking care to place them as on the previous flight.  We also installed the mylar solar shields which surround the instrument cradle and deflect the extremely bright visible sunlight at float.

With the SIP in place, yesterday afternoon we were ready to check the final balance of the gondola.  One more piece remained to be integrated, though:  the SIP cage.  This metal frame mounts under the gondola and helps protect the SIP from damage on landing.  On our last flight, though, we landed directly upright, and both the SIP cage and the SIP were crushed.  When our machinist at SSL rebuilt the SIP cage for this flight, he changed the mounting of the cage as well.  Before, it hung on brackets from the bars at the base of the gondola.  In an effort to strengthen the gondola and improve the cage’s strength on impact, the new version slides over pegs that stick out the bottom of the gondola frame.

I’ve been worrying about those pegs since I first laid eyes on them.  I knew if they bent, we wouldn’t be able to get the SIP cage on, and we’d have precious little chance of repairing the pegs.  We’ve taken great care to keep the pegs off the ground and out of trouble, and we’ve been successful.  The SIP cage itself has mostly been in storage, out of the way.

Yesterday, though, when we went to slide on the SIP cage, it didn’t fit.  The front two tubes were more than an inch out of alignment!  Whether due to the heat in the container or pressure in transit, at some point the cage had warped.  In an instant, we went from planning for compatibility the next day to being unsure if we’d need to rebuild the cage.  Despondent, we went home to sleep on our options.

This morning, we had some lengthy discussions with the CSBF mechanical crew.  The approach we converged on was to jack the cage arms outwards with a 4×4 and a hydraulic jack.  The original hope was to push the arms beyond our desired length and create a permanent bend, but we quickly found that the cage just sprung back to its original dimensions.  Instead, then, the CSBF crew cut a 2×4 to length and wedged it into the arms.

Setting up the jack.

Streeeetch...

After finishing some tests of the instrument electronics, after lunch we took a deep breath and tried to fit the SIP cage again.  We installed tapered tips on the pegs to help guide them in, applied plenty of grease, and then used the gondola weight to lower the gondola onto the cage.  Everything came together so smoothly it was hard to remember that there’d been any problems!  Elated, we finalized the placement of the ballast hoppers before taking the cage off.

It fits!

With the problem of the SIP cage solved, we’re now ready for compatibility.  Unfortunately, another problem looms: the weather.  There have been frequent thundershowers the last few days, and the ground outside the hangar is too wet to drive the crane.  Also, since there are more showers in the vicinity we’d run the risk of getting caught out in the rain.  So, we’ll hold off on compatibility until probably Monday.  It’s no great loss, as the airport is too wet for TIGRE to launch, either!  We’re all in a holding pattern, then.  We’ll use the time to catch our breath, perform calibrations with our recently-arrived sources, and finalize our observation planning.

Cranes live in lakes, right?

On interesting side effect of all this rain: we’re seeing the Todd River in flood a second time!

The Todd flooding through the Gap.

Several roads crossing the Todd were closed.

Down to Earth

The last couple of days have begun with bad news on the launching front–the low level winds have been too high for TIGRE to launch.  While the float winds are great–a hypothetical 48-hour flight would stay quite close to Alice Springs–conditions are the ground are worsening.  (Tonight we had an electrical storm and rain!)  Looks like the next launch opportunity will be late this week at the earliest.

Still, our turn will come, and we’re pushing ahead to be ready for it.  I started the day by pushing the gondola outside to calibrate the antenna positions on our differential GPS.  These four antennas provide our primary reference for our orientation during flight.  The winds were low and the process went smoothly.

Calibrating the differential GPS.

With Prof. Boggs here, this morning provided a good chance for us all to sit down and take stock of the state of the instrument and our preparation for flight.  We discussed a few tests to run over the next few days and got a handle on our priorities.

After lunch, we buttoned up the sides of the ebay, finalizing the routing of a number of communications cables.  We closed the day with a successful test run through the CSBF communications package, the SIP.  All told, a very effective Monday.

A storm is brewing...

Hopping Along

Part of what makes research in other countries interesting is experiencing different customs and cultures.  Here in Australia, we’ve learned about the ubiquitous utility of the phrase “no worries,” the culinary versatility of beetroot, and the sartorial advantages of fly nets.  Yesterday we learned that Australia takes a long public holiday for Easter–Friday through Monday.  Unfortunately for us, our calibration sources, which arrived in Australia on Wednesday, didn’t make it out of customs in time.  So, they’ll stay sitting in customs over the four day weekend, arriving in Alice Springs at the earliest next Wednesday.

We really need to get as much calibration data in hand as possible before we fly, so this is a blow.  For now, I guess we’ll fake the “no worries” thing and keep working on putting the system together.  Yesterday we hooked up the test solar panels and migrated to some improved data formats.  Last night I took the theodolite outside to divine a North-South reference line from some measurements of stars.

Installing the test solar panels--much smaller than last year's!

Today, then, we were able to take NCT outside for a little spin.  After aligning the cryostat with our North-South reference, we rotated the magnetometer that NCT uses to orient itself until we found its zero point.  This effort met with good success.  We weren’t able to calibrate our differential GPS, however, due to communications troubles with the computer running the calibration software.  We’ve figured out how to reset things appropriately and will have another go on Monday.

Lining up the gondola.

Today also brought some personnel changes.  Zach headed back to Taiwan to midterms and his previous life as a theorist.  We will miss his curiosity, good humor, and helpful spirit throughout the rest of the campaign!  On the other hand, today Prof. Boggs arrived from the U.S.  It’s good to have the boss on site!

Talk about burying the lede: tomorrow TIGRE shows for a possible launch opportunity!  The winds on the ground and at float seem quite favorable, but it’s always a tossup.  You can watch at home on the CSBF webcam!    Rollout is scheduled for 2:30 am Alice Springs time; the fill/launch period is likely in the 6-7 am range.  (A useful time converter is here–use Darwin as the city.)  Hopefully we’ll see TIGRE rise on Easter!

On Debugging

Today's guest blog for NCT provided by xkcd.

Powering Up

We students did some real brawn work yesterday–lifting our battery boxes into the bottom of the ebay.  It was a tight squeeze, but we got them settled in and strapped down securely.  Since the gondola was well-balanced with the added weight in the back, we were free to move on to a much-anticipated task: rotation tests.  While we’ve solved the problem that caused our pointing problems on the last flight, we’d really like to be sure that the rotation systems are as stable and repeatable as possible before flight.

As we started wiring up the pointing hardware, though, Steve noticed a problem:  the lights on the computer module which drives the servo motor weren’t on.  The flight computer indicated the board was working, but flipping the external switches didn’t change the lights’ state as it should.  Since we use those switches for flight-line checks–and it’s a crucial module–we needed to figure out what was happening.

Unfortunately, the electronics driving the pointing system were designed by engineers at another university, twenty years ago, and poorly documented.  Steve traced through the schematics and pinouts as best he could, and determined that either a small voltage regulator was blown or that the supply voltage wasn’t coming through the backplane.  Maybe a cable had been pinched in transit?  A bit of testing established that the regulator was fine.  Unfortunately, to test the supply voltage we’d have to pull the heavy flight computer case out of the bay–it’s a tight squeeze even without the delicate signal cables lining the side. As I pulled out some of the impeding cables, Steve asked, “I wonder if there might be a switch that controls that power line?”  With that, he flipped a heretofore-obscure toggle switch–and the lights came on.

With the “light switch” now appropriately labeled, we were able to pick up the gondola and do rotation tests!  All the systems worked flawlessly, and even the fairly crude tests we were able to do in the hangar produced repeatability at the tenths of a degree level–easily better than our needed pointing accuracy.

Zach and Jane watch the rotation test.

Overhead, the winds at float are great for flying.  On the ground, though, the mighty cyclone is still keeping low-level winds too high for launching.  TIGRE might get an opportunity yet this weekend.

Blowing in the Wind

With TIGRE nearing flight readiness–they are planning for compatibility tomorrow morning–today CSBF held the first of what will be ongoing daily weather briefings.  The news was that the upper atmosphere winds will begin turnaround by the end of the week.  Turnaround is the best time to fly balloons, as the prevailing winds are shifting from blowing west to blowing east (or vice versa).  The transition stage tends to have low overall wind speeds, so balloons can float for a long time without going very far.  With the size and emptiness of the Australian continent, catching the peak of turnaround is somewhat less crucial for flight duration than it is in New Mexico.  Nevertheless, we left the meeting motivated to get ready.

Thankfully, Jane and Steve have decreased the noise we were seeing through judicious rerouting of the cables, so tomorrow we will return to integrating the remaining gondola systems.

The regional news station did a report on the balloon launching station and the upcoming campaign recently.  See if you can spot the NCT team member who makes a cameo!

In the Dark #2

Once again today we found ourselves in both the literal and metaphorical dark, as the hangar power went out as we were grappling with a tricky problem.  (This time, though, all of our most crucial electronics were safely on battery backup.)  Since completing what we had hoped would be our final routing of the signal cables, we’ve developed a bit of a noise issue.  There seems to be an interaction been electrical grounds which should be isolated that we haven’t seen before.  Jane and Steve spent a hard day trying to isolate the problem.  It can definitely be a bit of a black art.

With limited means of helping that process, the rest of us worked on other projects.  Alan updated some of the software we use to control the gondola from the ground.  Zach attached cables to all of the newly-installed shield pieces.  I spent some time looking at our science targets.  The Southern Hemisphere sky is rich in gamma-ray sources:  pulsars, x-ray binaries, active galaxies, supernova remnants, and of course our primary target, the Galactic center.  Making the best use of our observing time requires a good deal of planning!  I hope to say more about our science goals in an upcoming post.  For now, though, we’ve got to get ready to get off the ground first!