Overview
titanbrowse
titanbrowse is a complete reimplementation of titan_browse, a
database and visualizer for Cassini VIMS observations of Titan. This
documentation is still under construction, as is titanbrowse. Since
the principle is the same as the previous version, this document
only
describes the differences. For more details on how to make queries
to
the database, see titan_browse's
documentation.titanbrowse was written from scratch, around similar functionality as that provided by titan_browse, but with additional features, more data, and more efficient implementation. Most of the many differences are internal to the database, and do not show in the graphical interface (pp_titanbrowse_gui). The GUI is similar enough to make it easy to infer how to use it. In addition to the GUI, now there is an API (pp_titanbrowse, which is used by pp_titanbrowse_gui) to the database, which allows access to the same data, and the same query capabilities, from the command line or from other programs. Most of the documentation that still needs to be made is for the API, since the GUI is easy to interpret.
One major improvement from titan_browse is that now the data files contain all the data of every VIMS cube of Titan of the time period they cover. This allows any band or backplane to be used in pp_titanbrowse for selection or visualization, as well as examining images of bands and plots of spectra, and obtaining selected spectra or cubes directly from pp_titanbrowse, without the need for any other data files or libraries.
The cubes contained in the database have much more information than either the PDS cubes or those from the VIMS pipeline. For each pixel, the coordinates are calculated at the center and at each corner of the pixel, which is necessary for proper use of large pixels, pixels near the limb, and cubes with small number of pixels across one of their dimensions. Also, the geometric data is not limited to those pixels that intercept the surface (as is the case with cubes processed with ISIS, such as those from the VIMS pipeline). For pixels that do not intercept the surface, the calculated geometric data refers to the point where the line of sight gets nearest to the surface (for now, assumed spherical, in the lack of better shape determinations). The minimum altitudes of the lines of sight are also provided. Other additional information are the locations of the Sun, observer and specular point, and distances from the pixels to the specular point. These are all in backplanes, which currently are 52.
The data that at first sight does not seem to depend on pixel position (such as specular point coordinates) is provided for each pixel, because they were calculated at the middle of the exposure time of that pixel. Remember that VIMS does not obtain all the pixels in a cube at the same time (except for the very particular and rare case of a single line VIS cube). This is most relevant near closest approach, or during fast attitude changes, when the pixel directions can change considerably between the time one pixel was taken to the time of the next pixel. See the Data description section below for more details.
Update (20120916):
Source files and a few database files are now kept at git repositories:https://github.com/ppenteado/pp_titanbrowse
https://github.com/ppenteado/pp_titanbrowse_files
Update (20100920):
Improvements to the GUI, including: added the option to only plot on the map the pixels that fall on the surface, fixed the orthographic and perspective projections, and added latitude, longitude and altitude for the cursor position on the cube viewer.Update (20100806):
- New datafiles (up to covims_0036): The standard ISIS
backplanes
should be avoided, as I still need to change ISIS to make it use
the
new kernels properly. This does not affect the other backplanes
(those
that only the cubes I prepare have). So, for instance, to use
the
coordinates of the pixel center it is preferable to use the
backplanes
LAT_0 and LON_0, instead of LATITUDE and LONGITUDE. Also, I
recommend
not overwriting the old datafiles you have, as sometimes the new
sets
of kernels cause worse pointing errors for some periods. This is
a
difficult issue to deal with, and has been a problem for all the
Cassini data.
- A new program, pp_titanbrowse_datacompress, can be used to decompress (and to compress, of course) the datafiles. They are provided compressed, for faster download, and can be used in that way. But if one desires to trade processor load by disk use, they can be decompressed with the above routine.
- In addition to the converter above, the _db files can be
generated already uncompressed, with the new keyword
(compress=0).
- There were several bug fixes, both the the database and the GUI, over the last few months, so the code should be replaced with the current version.
- Soon to come, there will be a new, more intuitive array-like
syntax to use in the functions for cube/pixel selection and
plot. This
will allow to use operations on bands as in regular array
indexing.
These new features will require the use of IDL 8 (which is
already
available, since July/22).
Documentation
The documentation of each routine used by titanbrowse is linked on the frame on the left. In addition, the documentation of the previous version, titan_browse, may also be useful in understanding how to make cube and pixel selections.Installation
The requirements for pp_titanbrowse are IDL, plus, for
pp_titanbrowse_gui, David Fanning's Catalyst and Coyote libraries.
If
you do not already have those libraries installed, instructions are
at http://www.dfanning.com/catalyst/howtoinstall.html.Database files
The files used by the database are divided in (at the time of this writing) 29 sets, from the corresponding PDS datasets from which the data was obtained. All the VIMS data came from the PDS Imaging Node, and all the SPICE data used in the processing came from the PDS Archive for SPICE data. All data and information used to make both the software and the database files is publicly available, from either PDS, NAIF, USGS, or CICLOPS. The only exception is the VIMS background map, from Barnes et al. 2009, which was provided by Jason Barnes.Each set takes two files per VIMS channel, for instance, covims_0004_ir.sav and covims_0004_ir_db.sav, for the COVIMS_0004 set of IR data. The examples below all use file names for IR channel data. The current 58 files for the IR channel take 11GB. However, it is only necessary to download the *_ir.sav files, since pp_titanbrowse can be used to generate the corresponding *ir_db.sav files. This reduces the download size to (currently) 7.6GB.
A sample of the database files can easily obtained from a git repository. It is only a sample, due to storage constraints at that server. To obtain these files from the repository, you can do:
[user@computer ~]$ git clone git://github.com/ppenteado/pp_titanbrowse_files.git
Cloning into pp_titanbrowse_files...
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 1), reused 0 (delta 0)
Receiving objects: 100% (6/6), done.
Resolving deltas: 100% (1/1), done.
This will create a directory pp_titanbrowse_files under the current directory, from which you can start using pp_titanbrowse.
To obtain all the files (not just the sample at pp_titanbrowse_files) , you have to download them directly from another server. The separate files are at http://ppenteado.net/vims/pp_titanbrowse/sav/, but to make the download easier, the *_ir.sav files are packed into 4 tar files, at http://ppenteado.net/vims/pp_titanbrowse/sav/tar/.
Generating the _db files
If you opted to download only the *ir.sav files, the *ir_db.sav files will have to be generated before pp_titanbrowse can be used. This may take a long time (many minutes to a few hours, if all files are present), but it may be faster than downloading them. In case you are wondering, this is actually relatively light work, just done on a lot of data. The heavy work was done when the *ir.sav files were created from the raw PDS cubes, which currently takes over a week to process.From an IDL session in the directory containing those files, first create an array with all the *ir.sav files, for instance, with
IDL> files=file_search('covims_*_ir.sav',count=nfiles)
IDL> print,files
covims_0014_ir.sav covims_0029_ir.sav covims_0032_ir.sav
Then loop over the files, to create each _db.sav file, with
IDL> for i=0,nfiles-1 do begin & a=obj_new('pp_titanbrowse_db',files[i],/build) & obj_destroy,a & endfor
Building db file, this may take a long time and use a lot of memory if there are many cubes
Processing cube 1/498
Processing cube 2/498
(...)Once that is finished, you are ready to start using pp_titanbrowse. Obviously, the db files only have to be built once. If you want the files to be stored uncompressed, use the keyword (compress=0) on the obj_new() call.
Source and documentation files
The source and documentation files can now be found in a git repository.To install the library, you may simply use git to download the current version and any susbsequent update. One way to do it is:
[user@computer ~]$ git clone git://github.com/ppenteado/pp_titanbrowse.git
Cloning into pp_titanbrowse...
remote: Counting objects: 27, done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 27 (delta 1), reused 24 (delta 0)
Receiving objects: 100% (27/27), 8.60 MiB | 156 KiB/s, done.
Resolving deltas: 100% (1/1), done.
This will create a directory called pp_titanbrowse under the current directory, with a copy of the library. Then, add the directory pp_titanbrowse/source to your IDL path. Since it is the same that would be done to install Catalyst or Coyote, see their installation instructions if you need more information on how to add a directory to IDL's path.
Use of pp_titanbrowse
To start using the GUI, from the directory where the database files are located, simply create a pp_titanbrowse_gui object in your IDL session, as inIDL> a=obj_new('pp_titanbrowse_gui')For the IR channel, which is the default. If you have the VIS channel files, then use
IDL> a=obj_new('pp_titanbrowse_gui',/vis)
It will search for all the database files in the current directory, and load them all into the database. The time it will take to start depends on how many files are present. Contrary to titan_browse, this will already have all the data directly available, with no need to manually select subsets to be loaded or unloaded. But despite this (and the much larger amount of data that pp_titanbrowse can handle), its initialization is much faster than that of titan_browse, and it uses far less memory.
Once you see the graphical interface, it should be ready to use. It is divided into 5 tabs:
Main tab
This is used to browse through the flyby hierarchy (in the tree on the left side), to find specific cubes. If a cube is selected on the tree, its main information is shown in the right side. Double-clicking a cube on the tree has 3 effects:- Send the cube pixel locations to be plotted on the Map tab,
- replace the text of the cube selection function (in the Cube selection tab) by a function that will select only that cube, and
- send the cube data to be used by the Cube tab
On the bottom right side of this tab, the displayed cube information can be exported to either a text file, or to a variable in the current IDL session, as a structure of type pp_titanbrowse_cubeinfo, such as:
** Structure PP_TITANBROWSE_CUBEINFO, 20 tags, length=1040, data length=1040:
REV STRING '093TI'
SEQ STRING 'S45'
SEQ_TITLE STRING 'VIMS_093TI_GLOBMAP001_CIRS'
PROD_ID STRING '1_1605808125.14960'
START STRING '2008-324T17:09:20.747Z'
STOP STRING '2008-324T17:20:43.821Z'
NAT_START STRING ' 1.6058081e+09'
LINES LONG 24
SAMPLES LONG 64
PIXELS LONG 1536
SURF_PIXELS LONG 1201
EXPOSURE DOUBLE 420.00000
IR_MODE STRING 'NORMAL'
VIS_MODE STRING 'NORMAL'
BACK_MAX STRUCT -> PP_TITANBROWSE_CMD_BACK Array[1]
BACK_MIN STRUCT -> PP_TITANBROWSE_CMD_BACK Array[1]
DBFILE STRING 'covims_0032_ir.sav'
CUBEFILE STRING 'CM_1605808125_1_ir_eg.cub'
DBIND LONG 2
CUBEIND LONG 1
Map tab
This is used to map data onto Titan's surface, with varied map
projections and background reference maps (other background
options are
yet to be implemented). The data plotted (chosen with the Display
mode
droplist) can be either the location of the currently selected
cube's
pixels (the cube must have been double-clicked, in the Main tab,
for
this), the location of the pixels in the pixel list (in the Pixel
selection tab), or an image of the value of some arbitrary
function of
the pixels in the pixel list (this feature currently is not
working
properly). The pixel location droplist chooses whether the
locations
plotted are the standard center of the pixels that intercept
the
surface (the LATITUDE and LONGITUDE backplanes), or the location
of the
points on the surface that are nearest to the lines of sight of
the
corners of each pixel (the LAT_1, LAT_2, LAT_3, LAT_4, LON_1,
LON_2,
LON_3 and LON_4 backplanes). Note that, as in titan_browse, the
lines
in pixel center mode connect the pixel centers, they are not the
pixel
edges. The lines in pixel corner mode connect the pixel corners,
so are
a more precise representation of the pixel shape, but are slower
to
draw on the map. A bitmap of the currently shown map can be
exported to
a png, jpeg or ps file.Cube selection tab
This tab is used to select a cube from the database, following the same query principle as described in detail and exemplified in titan_browse's documentation. It is important to note that the variable names (those obtained selecting them in the tree) must be separated by whitespace from other elements in the function. For instance, "( cmd.exposure *2.) gt 10." is legal, but "(cmd.exposure*2.) gt 10." is not. This is because pp_titanbrowse must parse the function before giving it to IDL for evaluation, and the parser is still very simple.Note that by default the text output of the list is not updated (it is frozen), though the list is being updated, as indicated by the number of cubes selected displayed. The reason is that selections start from all cubes, which is typically a very large number, and making the text output can be slow in this situation. When the list is filtered down to a smaller number of elements, its display can be unfrozen, to start showing the current list.
At the bottom, the cube list can be exported, as a text file, to a variable in the IDL session (an array of pp_editablecubes), or as the individual (original) cube files to a directory.
Pixel selection tab
This tab works similarly to the Cube selection tab, but for
selecting
pixels of the currently selected cubes. Again, titan_browse's documentation
explains how
to make pixel selections. As the described above for cube
selection,
the variable names must always be separated by whitespace from the
other elements in the expression.At the bottom, the pixel list can be exported, either as a text file, or as a variable to the IDL session. The variable will be an array of pp_titanbrowse_pixeldata, such as:
IDL> help,pixsel,/struct
** Structure PP_TITANBROWSE_PIXDATA, 5 tags, length=2488, data length=2488:
CORE DOUBLE Array[256]
BACKPLANES STRUCT -> PP_TITANBROWSE_CMD_BACK Array[1]
CUBE STRING 'CM_1605808125_1_ir_eg.cub'
X LONG 0
Z LONG 0
IDL> help,pixsel.backplanes,/struct
** Structure PP_TITANBROWSE_CMD_BACK, 52 tags, length=416, data length=416:
LATITUDE DOUBLE NaN
LONGITUDE DOUBLE NaN
SAMPLE_RESOLUTION
DOUBLE NaN
LINE_RESOLUTION DOUBLE NaN
PHASE_ANGLE DOUBLE NaN
INCIDENCE_ANGLE DOUBLE NaN
EMISSION_ANGLE DOUBLE NaN
NORTH_AZIMUTH DOUBLE NaN
LAT_0 DOUBLE -38.839169
LAT_1 DOUBLE -38.757030
LAT_2 DOUBLE -38.933907
LAT_3 DOUBLE -38.921597
LAT_4 DOUBLE -38.743710
LON_0 DOUBLE 160.40419
LON_1 DOUBLE 160.49832
LON_2 DOUBLE 160.23354
LON_3 DOUBLE 160.30919
LON_4 DOUBLE 160.57492
ALT_0 DOUBLE 102.46656
ALT_1 DOUBLE 110.09211
ALT_2 DOUBLE 107.57780
ALT_3 DOUBLE 94.850456
ALT_4 DOUBLE 97.376961
PHASE_0 DOUBLE 97.228409
PHASE_1 DOUBLE 97.247437
PHASE_2 DOUBLE 97.222069
PHASE_3 DOUBLE 97.209389
PHASE_4 DOUBLE 97.234756
INCIDENCE_0 DOUBLE 52.578320
INCIDENCE_1 DOUBLE 52.473488
INCIDENCE_2 DOUBLE 52.740810
INCIDENCE_3 DOUBLE 52.683792
INCIDENCE_4 DOUBLE 52.415230
EMISSION_0 DOUBLE 89.999977
EMISSION_1 DOUBLE 89.999977
EMISSION_2 DOUBLE 89.999977
EMISSION_3 DOUBLE 89.999977
EMISSION_4 DOUBLE 89.999977
AZ_DIF_0 DOUBLE 99.115982
AZ_DIF_1 DOUBLE 99.152992
AZ_DIF_2 DOUBLE 99.088104
AZ_DIF_3 DOUBLE 99.079025
AZ_DIF_4 DOUBLE 99.144127
OBSERVER_DIST_0 DOUBLE 26201.770
SPECULAR_DIST_0 DOUBLE 93.634842
OT_DISTANCE DOUBLE 26338.215
SOL_LAT DOUBLE -3.9179378
SOL_LON DOUBLE -156.21873
SP_LAT DOUBLE 19.903276
SP_LON DOUBLE -197.25456
OBS_LAT DOUBLE 32.293877
OBS_LON DOUBLE 111.95390
Cube tab
This tab is used to examine the contents of the cube selected (by double-clicking) on the Main tab. The image on the left side shows the band selected, and/or the selected backplane and a latitude/longitude grid. If the Plot spectrum button is enabled, a spectrum is plotted for every pixel clicked on the image. If a pixel is clicked with the right button, its spectrum is added to the plot, without erasing the previous spectra in the plot. Through this tab, the selected cube can be exported either to a variable in the IDL session (a pp_editablecube object), or to a cube file. Through the spectrum display window, when it is enabled, the spectra displayed can be exported to a variable in the IDL session as an array of simple anonymous structures such as:
IDL> help,spectrum,/structure
** Structure <4034128>, 2 tags, length=1448, data length=1444, refs=2:
CUBE OBJREF <ObjHeapVar8827(PP_EDITABLECUBE)>
PIXEL STRUCT -> <Anonymous> Array[1]
IDL> help,spectrum.pixel,/struct
** Structure <349e7d8>, 2 tags, length=1440, data length=1440, refs=2:
CORE FLOAT Array[256]
BACKPLANES STRUCT -> <Anonymous> Array[1]
IDL> help,spectrum.pixel.backplanes,/struct
** Structure <349d1c8>, 52 tags, length=416, data length=416, refs=2:
LATITUDE DOUBLE -28.825367
LONGITUDE DOUBLE 228.76825
SAMPLE_RESOLUTION
DOUBLE 14.329811
LINE_RESOLUTION DOUBLE 45.343521
PHASE_ANGLE DOUBLE 96.036674
INCIDENCE_ANGLE DOUBLE 72.675316
EMISSION_ANGLE DOUBLE 69.755333
NORTH_AZIMUTH DOUBLE 147.70184
LAT_0 DOUBLE -29.391121
LAT_1 DOUBLE -29.720751
LAT_2 DOUBLE -29.900677
LAT_3 DOUBLE -29.069536
LAT_4 DOUBLE -28.893679
LON_0 DOUBLE 131.36201
LON_1 DOUBLE 131.69547
LON_2 DOUBLE 131.39299
LON_3 DOUBLE 131.03439
LON_4 DOUBLE 131.33582
ALT_0 DOUBLE 0.0000000
ALT_1 DOUBLE 0.0000000
ALT_2 DOUBLE 0.0000000
ALT_3 DOUBLE 0.0000000
ALT_4 DOUBLE 0.0000000
PHASE_0 DOUBLE 96.036530
PHASE_1 DOUBLE 96.055527
PHASE_2 DOUBLE 96.030121
PHASE_3 DOUBLE 96.017532
PHASE_4 DOUBLE 96.042938
INCIDENCE_0 DOUBLE 72.676826
INCIDENCE_1 DOUBLE 72.419731
INCIDENCE_2 DOUBLE 72.698463
INCIDENCE_3 DOUBLE 72.932205
INCIDENCE_4 DOUBLE 72.653809
EMISSION_0 DOUBLE 69.756859
EMISSION_1 DOUBLE 70.175568
EMISSION_2 DOUBLE 70.263412
EMISSION_3 DOUBLE 69.347862
EMISSION_4 DOUBLE 69.263229
AZ_DIF_0 DOUBLE 103.44067
AZ_DIF_1 DOUBLE 103.40625
AZ_DIF_2 DOUBLE 103.21777
AZ_DIF_3 DOUBLE 103.46878
AZ_DIF_4 DOUBLE 103.66153
OBSERVER_DIST_0 DOUBLE 27941.348
SPECULAR_DIST_0 DOUBLE 94.304420
OT_DISTANCE DOUBLE 28933.352
SOL_LAT DOUBLE -3.9178617
SOL_LON DOUBLE -156.33441
SP_LAT DOUBLE 20.274433
SP_LON DOUBLE -197.57336
OBS_LAT DOUBLE 32.658207
OBS_LON DOUBLE 111.05993
Use of pp_editablecube
All variables with cubes exported to the IDL session are
pp_editablecube objects. If they come from the Cube tab, they are
a
single pp_editablecube with that cube. If they come from the Cube
selection tab, they are an array of pp_editablecubes, one for each
cube
in the selection.pp_editablecube is a class designed to read (from the inherited, read-only, pp_readcube class), edit and write cubes in the ISIS (commonly ".cub" or ".QUB") format. These classes were written from scratch, for several reasons:
- Do not need to install ISIS just to access cubes from IDL.
- Have a better interface than that of ISIS' awkward routines.
- Be able to read an write the entire contents of the cube, not just the core and backplanes.
- Be able to store the cube contents and transfer them between programs, in a way that the cube can later be recreated, the same as the original, or after extensive editing.
- Handle cubes of any endianness.
- Implemented completely in IDL, with no need for DLMs or any other libraries.
Data description
In the current version, each cube contains the following data (all of which can be accessed and edited through pp_editablecube):- Cube metada. From an example:
REV 093TI REV identifier
SEQ S45 Sequence identifier
SEQ_TITLE VIMS_093TI_GLOBMAP001_CIRS Sequence title identifier
PROD_ID 1_1605808125.14960 Cube identifier
START 2008-324T17:09:20.747Z Cube start time
STOP 2008-324T17:20:43.821Z Cube stop time
NAT_START 1.6058081e+09 Cube native clock start time
LINES 24 Number of lines in the cube
SAMPLES 64 Number of samples in the cube
PIXELS 1536 Number of spatial pixels in the cube (lines*samples)
SURF_PIXELS 1201 Number of pixels whose center line of sight intercept the surface
EXPOSURE 420.00000 Exposure time
IR_MODE NORMAL Resolution mode for the IR channel
VIS_MODE NORMAL Resolution mode for the VIS channel
DBFILE covims_0032_ir.sav Database file this comes from
CUBEFILE CM_1605808125_1_ir_eg.cub Cube file
DBIND 2 Index that identifies the database file
CUBEIND 1 Index that identifies the cube in the database file
- Core bands (including wavelengths and units)
- Sideplanes and bottomplanes (pp_titanbrowse does not use those, but they are present in the cubes)
- Backplanes (includes their names and units). From an example:
LATITUDE NaN Latitude of the center of pixel line of sight, if it intercepts the surface (from ISIS)
LONGITUDE NaN Longitude of the center of pixel line of sight, if it intercepts the surface (from ISIS)
SAMPLE_RESOLUTION NaN Sample resolution of the center of pixel line of sight, if it intercepts the surface (from ISIS)
LINE_RESOLUTION NaN Line resolution of the center of pixel line of sight, if it intercepts the surface (from ISIS)
PHASE_ANGLE NaN Phase angle of the center of pixel line of sight, if it intercepts the surface (from ISIS)
INCIDENCE_ANGLE NaN Incidence (solar) angle of the center of pixel line of sight, if it intercepts the surface (from ISIS)
EMISSION_ANGLE NaN Emission (observer) angle of the center of pixel line of sight, if it intercepts the surface (from ISIS)
NORTH_AZIMUTH NaN North azimuth of the center of pixel line of sight, if it intercepts the surface (from ISIS)
For the following backplanes, the
data
refer to the point nearest to the surface in the line of sight,
of the
pixel's center (_0), top-left corner (_1), top-right corner
(_2),
bottom-right corner(_3), and bottom-left corner (_4):
LAT_0 -38.839169 Latitude
LON_0 160.40419 Longitude
ALT_0 102.46656 Altitude to the surface (0 means the pixel intercepts the surface)
PHASE_0 97.228409 Phase angle
INCIDENCE_0 52.578320 Incidence angle
EMISSION_0 89.999977 Emission angle
AZ_DIF_0 99.115982 Azimuth difference between the Sun and the observer
OBSERVER_DIST_0 26201.770 Slant distance from the point to the observer
SPECULAR_DIST_0 93.634842 Angular distance from the point to the specular reflection point
OT_DISTANCE 6338.215 Distance between the target and observer
SOL_LAT -3.9179378 Subsolar latitude
SOL_LON -156.21873 Subsolar longitude
SP_LAT 19.903276 Specular point latitude
SP_LON -197.25456 Specular point longitude
OBS_LAT 32.293877 Subobserver latitude
OBS_LON 111.95390 Subobserver longitude
Paulo Penteado
pp.penteado@gmail.com
Version 20120916
Project statistics
Directories: | 1 |
.pro files: | 16 |
.sav files: | 0 |
Routines: | 104 |
Lines: | 4,773 |