The last couple of days have begun with bad news on the launching front–the low level winds have been too high for TIGRE to launch. While the float winds are great–a hypothetical 48-hour flight would stay quite close to Alice Springs–conditions are the ground are worsening. (Tonight we had an electrical storm and rain!) Looks like the next launch opportunity will be late this week at the earliest.
Still, our turn will come, and we’re pushing ahead to be ready for it. I started the day by pushing the gondola outside to calibrate the antenna positions on our differential GPS. These four antennas provide our primary reference for our orientation during flight. The winds were low and the process went smoothly.
Calibrating the differential GPS.
With Prof. Boggs here, this morning provided a good chance for us all to sit down and take stock of the state of the instrument and our preparation for flight. We discussed a few tests to run over the next few days and got a handle on our priorities.
After lunch, we buttoned up the sides of the ebay, finalizing the routing of a number of communications cables. We closed the day with a successful test run through the CSBF communications package, the SIP. All told, a very effective Monday.
A storm is brewing...
Today's guest blog for NCT provided by xkcd.
With our large project and small team, at any given time each of us has a to-do list longer than we can read in one sitting. For the most part, we each choose tasks to work on based on mood or momentum. In the last few days, though, I’ve been trying to determine which of our myriad of checkboxes is limiting our progress towards the distant but looming goal of flight readiness.
The two most immediate items on that prioritized path are finalizing the detector connections and installing the shields. Our heavy bismuth germinate (BGO) shields surround the NCT detectors on the bottom and the sides, reducing background from the atmosphere. Installing them is a risky and dicey procedure, though, so we want to make sure our detectors are behaving as they should and the connections are all good before we wall them off. All this takes time, consultation, and reference to archival data. We’re making progress, but it’s not as clear cut as determining that things turn on.
For their part, the shields turned on fine, but we found higher background rates than we expected. With ground backgrounds poorly characterized and influenced by everything from the presence of smoke detectors to the composition of the hangar cement, sorting this issue out has been a headache. So far, though, careful tests by Jane and Zach suggest the shields are working as they should.
Steve has been busy building an electronic isolator to report the cryostat temperature. Alfred and Ming-Zhe are working on the test solar panels. I’ve made a few minor changes to the flight software, and Alan continues to improve our ground support software. Alan and I are also discussing our calibration plans–if and when the sources we need arrive!
Keeping it all straight is a task in itself, but we’re helped by great support from team members still at home.
Sometimes the connections trip you up. We’ve spent the last several days trying to get our unruly Gorgon of signal cables to lie straight and carry signals cleanly. Each of our ten detectors has forty channels, so there are plenty of chances to get our wires crossed. Our signal cables are great–long ribbon cables, each strand of which is actually a coaxial wire with two conductors. The cables are accordingly much lighter and more compact than individual wires would be, but they are rather delicate. Routing them safely through the twists and bends of the journey from the cryostat, around the cradle, through a hole into the electronics bay, and to the appropriate board in the right card cage is a challenge. We’re also finding a few of the connectors are showing signs of fatigue, introducing flakiness into some channels. Replacing bad cables with our few spares can aggravate that fatigue, though. As usual there’s a balancing act between pushing for ideal performance and avoiding causing larger problems in solving smaller ones.
The connections in the back can be tricky.
Still, the system is taking shape. The third payload, HERO, has arrived with their gear, so the hangar is now quite full. We’re all sharing a single crane along the centerline of the building, so we put the gondola on our cart for now to allow them to use that space to build. We also started up the flight computer and are now running the system through it. Since programming the flight software was my main responsibility last flight, it was very satisfying for me to see the flight computer running nicely again. I also connected our differential GPS antennas, which seem to be functioning just fine.
Flight computer is up!
Tomorrow, a rest day for most of us. Soon we’ll be calibrating and testing the system with radioactive sources. Monday also brings our Flight Requirements meeting with CSBF, where we’ll discuss our altitude and duration targets for the flight.
The CSBF personnel have been modifying this crane to use as a launch vehicle.
We have the rotor! Testing will commence tomorrow, once McBride returns to us. In the meantime, we managed to check off lots of little things that we needed to do before flight–strapping cables down, checking the gondola balance, putting on solar shields, etc. My todo list for the flight computer is rapidly waning. I’m happy with how it’s running these days, it seems very solid.
To follow up my pictures from the CREST launch, Daniel found a vintage 1992 video about CSBF, Ft. Sumner, and scientific ballooning on YouTube. It’s an excellent introduction to the launch process!
We’ve had some hiccups this week in our preparation for flight, but we’re still making solid progress. The biggest question at the moment is the status of the rotor; it flew back to Berkeley on Thursday to see if it can be rated to our weight for this flight. Assuming we don’t have to drop weight, our short-term schedule will mostly be shaped by the timing of the rotor’s return to us.
There’s plenty to press ahead with in the interim, though. The flight computer is safely ensconced in the electronics bay, and Yvette routed its myriad cables into something resembling order–a feat I thought impossible! We also put the sides of the ebay in place–they help regulate the temperature of the electronics in flight. Daniel is also fitting the mylar solar shields which keep the detectors themselves out of direct sunlight. Some of the others have been attempting to diagnose and reduce the electronic noise in the system.
The fully harnessed, much neater ebay!
Daniel and Alfred install the mylar solar shields.
For my part, my attention is divided. There are still improvements and bug fixes needed in the flight code. Today I found a nasty little bug in our pointing routines that caused an exponentially growing number of processes to be spawned! Needless to say, that was harmful to the stability of the system. It was a simple fix, thankfully. I also worked on interfacing our computer with the CSBF command link. Finding trouble, McBride determined that two of our serial lines have bizarre a hardware failure…
On the whole, though, the system is rapidly improving in stability and readiness. With most of the detailed harnessing and mechanical work done, we’ve moved the gondola up on its cart. With this mobility comes the ability to start our efficiency calibrations, which I’ll discuss more in a future post.
I fixed the repeated data problem on the flight computer disks! Technical details below, but the short description is that we made some unwarranted assumptions about how fast different processes would execute relative to each other. A small “wait until your partner is ready” flag was enough to fix it. This is a problem I’ve known about since last September and spent a lot of time sweating over, so it feels great to have it resolved!
Technical: Again, we have two buffers (bins, roughly), one of which fills with new data while the other is written to disk. When the buffer gets filled, the two get swapped and the process repeats. Given the speed of our disks and our data rates, the original author of this code and I both expected the disk writes to happen much faster than the filling of the new data buffer. We had code that reserved each buffer for only one of filling or writing at a given time, but nothing other than that protection kept the two tasks in sync. The problem was, due to mysteries of scheduling known only to the Linux kernel, the new data buffer was sometimes filling repeatedly before the write task would even start! Data was getting overwritten that was never commited to disk. Then the write task would start, see that it had several buffers to commit to disk, and end up writing a single buffer several times in a row. The fix was to require that the fill task wait to swap until the write task is ready for a new buffer.
I am greatly relieved to have this resolved! The joy is overshadowed, though: CSBF isn’t happy with our rotor, and wouldn’t certify it even to fly at the weight we had in 2005 (eliminating any chance we have of cutting down weight to meet requirements). Sounds like we are going to have to fly the rotor back to Berkeley for examination by our engineers and/or a “pull test.” Yikes!