Have you ever wondered what data from a spacecraft actually look
like? You may think that the picture you see from the Hubble Space
Telescope or the Chandra X-ray Observatory is exactly what the satellite
in orbit sends down to Earth, like broadcasts from a tv station, but
that's not the case.
Data from a satellite, or "telemetry," is usually in a special
format, like a code, designed so a lot of information can be transmitted
in a very short time. It's the job of the Data Processing group to turn
the telemetry or raw data from a satellite into something that
astronomers can easily read and interpret.
Packets, and Realtime Data
Data is generally sent in packets: specific amounts of bit-sized
data that have a time stamp. Transmission of these packets can be
uneven. For example, part of the relay system may 'drop packets' due to
static or transmission errors. Those dropped packets are retransmitted,
so no data is lost. Also, stored data and realtime data can be
transmitted at the same time, each with their different time stamps.
The end result of these scenarios is that the data received on the
ground, while complete, may not be in the proper order. The packet's
time stamp lets you reassemble the data in the order observed, rather
than the order you receive it. Naturally, it makes sense to wait until
your entire transmission is complete before doing the final sorting.
The sorting can be done by either the ground station, or your local
processing center. For both, the method is the same. You store all the
incoming telemetry on your computer. Then, when the transmission is
complete (or all data tapes have been received from remote sites), you
sort the packets by time stamps to create, essentially, a perfect copy
of what had originally been on the satellite.
What Comes In To Data Processing
A satellite in Earth orbit has several types of data it may send to
people on the ground. It can send science data, like observations of
matter falling into black holes in the case of an x-ray telescope
satellite. It can send "housekeeping" data about the spacecraft's
health, like how the solar panels are doing and whether the battery is
fully charged. And it can send "attitude" data, which tells exactly
where the satellite is pointed and how high above the Earth it currently
For an astronomer to use a satellite, he or she needs to know all
these things at once: *where* was the satellite looking, *what* did it
see, and was everything functioning normally at the time? The problem
is, the satellite can't tell you everything it's doing at once. The
different types of data come from different systems on board the
satellite: the telescope collects science data, the power system keeps
track of the solar panels and any batteries, the "attitude control"
system tells where the satellite is pointed, etc. When it's time to send
data to the ground, the different systems take turns, like tv reporters
on the evening news.
When you watch the evening news, you might hear what the government
did today, then what the weather was like, and then what happened in
sports. You don't hear one reporter say, "Today, at 3pm, the President
gave a speech, the Yankees won the first of a double-header, and a bad
storm ripped through Nebraska. Then, at 5pm, Congress worked late on
some legislation, the Orioles had a great triple play, and there was
flooding in Florida." Instead, one reporter tells you what the weather
was like all over the country, and after that another reporter tells you
everything that happened that day in sports, etc.
It's the same with an orbiting satellite: first, you might get all
the data about where the satellite pointed that day, then all the things
the telescope saw, then a read out of the spacecraft's health. The
Ground System makes sure all the data are received correctly, then sends
everything to Data Processing, where the tricky task of matching up the
separate reports from the different satellite systems begins.
One Billion Bytes A Day
Data Processing is basically one big computer program: it takes in
the individual subsystem reports the satellite sent down, puts them all
together, and resorts them so that things that happened at the same time
are all together in one file. These files are converted into a special
easy-to-read format that astronomers use, called "FITS". FITS stands for
"Flexible Image Transport Standard", in other words, a simple way for
astronomers to write data that can be read on many different systems by
very different programs. Before astronomers started commonly using FITS
files, data from one satellite or telescope often couldn't be read
without someone first writing a special program to read it.
Sorting data from a satellite doesn't sound like too big a task -
what's all the fuss about Data Processing? Satellites today frequently
send a *lot* of data to Earth, so Data Processing has to work pretty
hard to match everything up. For example, NASA's X-ray Timing Explorer
Satellite (XTE) generates about 450 Megabytes of "raw" data a day. The small
"floppy" diskettes that personal computer use hold less than 1 Mb of
data, so that's already more than 450 floppy disks of data coming from
XTE a day. Also, the raw data is in a special compressed format,
and packed into subsystem "reports", to boot.
Once Data Processing finishes putting day's worth of XTE data into
FITS format, the data takes up even more space. Even though FITS is easy
to use, it is not compressed. The data have also been broken up into
individual files, so that everything that occurred on board the
satellite at the same time is now in one file. When a day's worth of XTE
data leaves Data Processing, it takes up over a Gigabyte of disk space,
or more than 1,000 floppy diskettes. That's over 1,000,000,000 bytes of
data! And XTE's Data Processing program is fast enough to process three
times this much data in one 8-hour day.
How big does a computer program have to be to handle this much data?
For XTE, Data Processing takes about 10,000 lines of computer code. As
you can imagine, it took several programmers many months to write and
test that much code. But it pays off - when finished, the Data
Processing programs can run for years, processing all the data that
comes from a satellite, faster than the satellite can send it to the
Next: Tell me about Data Archives